Agent skill · NVIDIA
nemo-mbridge-perf-moe-dispatcher-selection
Choose the right MoE token dispatcher (`alltoall`, DeepEP, or HybridEP) for the hardware, EP degree, and optimization stage. Summarizes patterns from DSV3, Qwen3, Qwen3-Next, and VLM bring-up work.
What it needs
About 5k tokens when loaded.
What this skill does
MoE Dispatcher Selection Guide Stable docs: @docs/training/moe-optimization.md Card: @skills/nemo-mbridge-perf-moe-dispatcher-selection/card.yaml Quick Decision By hardware Hardware First choice Why --- --- --- H100 DeepEP, if the runtime package is installed Strong default for cross-node EP on Hopper B200 DeepEP, if the runtime package is installed Good first choice unless a platform-specific HybridEP path is available GB200 / GB300 NVL72 HybridEP, if the runtime package is installed Best fit for NVLink-domain-aware dispatch and lower memory pressure Unknown or first bring-up alltoall Easiest path for correctness and debugging By EP degree EP size Guidance --- --- Small EP Dispatcher choice is usually second-order; start with alltoall or DeepEP Medium EP DeepEP often becomes worthwhile Large EP HybridEP is usually the best target on NVL72 systems Model-Family Patterns Workload Common best path Notes --- --- --- DSV3 at large scale HybridEP on GB200 or GB300, DeepEP on H100 Dispatcher choice matters more as EP and PP both grow Qwen3 235B DeepEP on H100, HybridEP on GB200 HybridEP usually wins on GB200 and often uses less memory Qwen3 30B DeepEP Smaller models still benefit, but the absolute gap is smaller Qwen3-Next Close race in BF16, HybridEP stronger in FP8 or memory-tight runs Good reminder to test, not assume MoE VLMs Start simple, then test HybridEP on GB200-class systems Vision workloads are sensitive to both memory and host overhead Rounded Evidence Summary Backend availability gate Do not interpret a dispatcher timing until the container has proven that the selected backend package is available. --moeflexdispatcherbackend None selects the standard alltoall dispatcher, while deepep and hybridep select moetokendispatchertype="flex" and then require their corresponding runtime packages at model construction time. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills NVIDIA/nemo-mbridge-perf-moe-dispatcher-selection