Agent skill · software engineering · nvidia
nemo-mbridge-perf-moe-comm-overlap
MoE expert-parallel communication overlap in Megatron Bridge. Covers dispatch/combine overlap, flex dispatcher backends, and expert wgrad scheduling.
Why this skill is useful
Provides specific configuration commands and tuning parameters for optimizing MoE communication overlap in Megatron Bridge that are not widely known.
What it needs
About 4k tokens when loaded. Last updated 2026-08-07. 2,816 stars on the source repository.
What this skill does
MoE Communication Overlap For the higher-level overview, see: @docs/training/communication-overlap.md @skills/nemo-mbridge-perf-moe-comm-overlap/card.yaml Quick Decision Use MoE communication overlap when: EP > 1 token dispatch or combine time is visible in the profile the run is already correct and you are now tuning throughput Avoid turning it on as an early bring-up step. It is easier to validate after the dispatcher, routing mode, and recompute plan are already stable. Enablement Prerequisites expertmodelparallelsize > 1 nummoeexperts > 1 moetokendispatchertype must be "alltoall" or "flex" Precision: BF16 or FP16 If PP is used, VPP (virtualpipelinemodelparallelsize) must be set (non-None) Flex dispatcher activation Setting moeflexdispatcherbackend alone does not activate flex dispatch. You must also set moetokendispatchertype = "flex". Recompute And CUDA Graph Interaction Full recompute is not a good companion for the overlap path. delaywgradcompute adds further constraints if CUDA-graph scopes include attention or MoE-router work. In practice, selective recompute is the safer pairing when overlap is enabled. Measured Evidence HybridEP production-shape validation A 2026-07-25 controlled Qwen3 30B-A3B pretraining comparison used 16 H100 GPUs, BF16, sequence length 4096, TP=1, PP=1, CP=1, EP=16, MBS=1, GBS=1024, forced-balanced routing, HybridEP, and Transformer Engine CUDA-graph scopes moerouter and moepreprocess. The only performance change was plain EP overlap; delayed wgrad stayed disabled. Case Steady window Step time Model TFLOPS/GPU --- ---: ---: ---: EP overlap off iterations 5-20 24.7138s 244.039 EP overlap on, search run iterations 5-20 21.0725s 286.208 EP overlap on, independent validation iterations 41-50 20.9920s 287.305 The independent result reduced step time by 15.059% and increased throughput by 17.729% over the reproduced baseline. Loss remained finite, no iterations were skipped or NaN, and rank-0 peak allocated memory was 62.166 GiB. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills nvidia/nemo-mbridge-perf-moe-comm-overlap