Agent skill · NVIDIA
nemo-mbridge-perf-moe-comm-overlap
MoE expert-parallel communication overlap in Megatron Bridge. Covers dispatch/combine overlap, flex dispatcher backends, and expert wgrad scheduling.
What it needs
About 4k tokens when loaded.
What this skill does
MoE Communication Overlap For the higher-level overview, see: @docs/training/communication-overlap.md @skills/nemo-mbridge-perf-moe-comm-overlap/card.yaml Quick Decision Use MoE communication overlap when: EP > 1 token dispatch or combine time is visible in the profile the run is already correct and you are now tuning throughput Avoid turning it on as an early bring-up step. It is easier to validate after the dispatcher, routing mode, and recompute plan are already stable. Enablement Prerequisites expertmodelparallelsize > 1 nummoeexperts > 1 moetokendispatchertype must be "alltoall" or "flex" Precision: BF16 or FP16 If PP is used, VPP (virtualpipelinemodelparallelsize) must be set (non-None) Flex dispatcher activation Setting moeflexdispatcherbackend alone does not activate flex dispatch. You must also set moetokendispatchertype = "flex". Recompute And CUDA Graph Interaction Full recompute is not a good companion for the overlap path. delaywgradcompute adds further constraints if CUDA-graph scopes include attention or MoE-router work. In practice, selective recompute is the safer pairing when overlap is enabled. Measured Evidence HybridEP production-shape validation A 2026-07-25 controlled Qwen3 30B-A3B pretraining comparison used 16 H100 GPUs, BF16, sequence length 4096, TP=1, PP=1, CP=1, EP=16, MBS=1, GBS=1024, forced-balanced routing, HybridEP, and Transformer Engine CUDA-graph scopes moerouter and moepreprocess. The only performance change was plain EP overlap; delayed wgrad stayed disabled. Case Steady window Step time Model TFLOPS/GPU --- ---: ---: ---: EP overlap off iterations 5-20 24.7138s 244.039 EP overlap on, search run iterations 5-20 21.0725s 286.208 EP overlap on, independent validation iterations 41-50 20.9920s 287.305 The independent result reduced step time by 15.059% and increased throughput by 17.729% over the reproduced baseline. Loss remained finite, no iterations were skipped or NaN, and rank-0 peak allocated memory was 62.166 GiB. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills NVIDIA/nemo-mbridge-perf-moe-comm-overlap