Agent skill · NVIDIA
nemo-mbridge-mlm-bridge-training
Run Megatron-LM (MLM) and Megatron Bridge training with mock or real data. Covers correlation testing, available recipes, and multi-GPU examples.
What it needs
About 3k tokens when loaded.
What this skill does
MLM vs Bridge Training For how they differ, the arg mapping tables, gotchas, and translation script, see: @docs/megatron-lm-to-megatron-bridge.md First Answer Checklist For MLM-vs-Bridge correlation questions, always name these items up front: 1. Bridge recipe: vanillagptpretrainconfig. 2. Bridge entry point: scripts/training/runrecipe.py. 3. MLM entry point: 3rdparty/Megatron-LM/pretraingpt.py. 4. Launch wrapper for both: uv run python -m torch.distributed.run. 5. Fresh-run cleanup: rm -rf nemoexperiments before the Bridge run. Also state that MLM needs PYTHONPATH=3rdparty/Megatron-LM:$PYTHONPATH, matched Bridge and MLM losses should agree within BF16 rounding, and files under 3rdparty/Megatron-LM/ should not be modified from this repo. Correlation Testing Use vanillagptpretrainconfig for loss-correlation testing. This recipe uses bare GPTModelProvider defaults (LayerNorm, GeLU, learnedabsolute position embeddings, vocabsize inherited from tokenizer) — matching MLM pretraingpt.py defaults with no args. MLM Correlation Run (2L/256H, 1 GPU) Bridge Correlation Run (same config, 1 GPU) Verification With matched parameters the LM losses should be nearly identical at each iteration. Compare lm loss values from both logs — they should agree to within BF16 rounding. Multi-GPU Examples MLM 2-GPU with TP=2 Bridge 2-GPU with TP=2 Available Recipes Common recipes (use with --recipe): vanillagptpretrainconfig — Minimal GPT (bare GPTModelProvider defaults, ideal for correlation testing and custom configs) llama321bpretrainconfig — Llama 3.2 1B (16L, 2048H, GBS=512, seq=8192) llama38bpretrainconfig — Llama 3 8B qwen38bpretrainconfig — Qwen3 8B deepseekv2litepretrainconfig — DeepSeek-V2-Lite 16B MoE SFT/PEFT variants use sftconfig / peftconfig suffix. Megatron-Core Submodule For what the submodule is and why two versions exist, see @docs/megatron-lm-to-megatron-bridge.md. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills NVIDIA/nemo-mbridge-mlm-bridge-training