Agent skill · NVIDIA
nemo-mbridge-perf-megatron-fsdp
Operational guide for enabling Megatron FSDP in Megatron-Bridge, including config knobs, code anchors, pitfalls, and verification.
What it needs
About 2k tokens when loaded.
What this skill does
Megatron FSDP Skill For stable background and recommendation level, see: @docs/training/megatron-fsdp.md @skills/nemo-mbridge-perf-megatron-fsdp/card.yaml Enablement Minimal Megatron FSDP override in Bridge: Example recipe fixup: Performance harness note: Code Anchors Bridge config definition: Bridge validation: Runtime wrapper selection: Perf harness overrides: Pitfalls 1. Public recipes often expose usemegatronfsdp but still default to ckptformat="torchdist". If save/load is enabled, switch to fsdpdtensor. 2. usetorchfsdp2 exists, but on the validated branch Bridge still fails before training because ddpwrap passes pgcollection. 3. CPU offloading is only valid when pipelinemodelparallelsize == 1 and activation recomputation is disabled. 4. Upstream warns that FSDP and TP/CP can want different CUDADEVICEMAXCONNECTIONS settings on Hopper and earlier. 5. Megatron FSDP and FSDP2 are mutually exclusive. Verification Use the existing 2-GPU functional smoke test: Success criteria: Pytest reports 1 passed The log shows finite loss at the last iteration The run finishes without a checkpoint format assertion
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills NVIDIA/nemo-mbridge-perf-megatron-fsdp