Agent skill · software engineering · nvidia
nemo-mbridge-perf-cpu-offloading
Validate and use CPU offloading in Megatron Bridge, including layer-level activation offloading and fractional optimizer state offloading with HybridDeviceOptimizer.
Why this skill is useful
Provides specific configuration commands and domain knowledge for CPU offloading in Megatron Bridge that the AI wouldn't reliably generate on its own.
What it needs
About 4k tokens when loaded. Last updated 2026-08-07. 2,816 stars on the source repository.
What this skill does
CPU Offloading References Stable docs: @docs/training/cpu-offloading.md Structured metadata: @skills/nemo-mbridge-perf-cpu-offloading/card.yaml What It Is Two independent mechanisms to move data from GPU to CPU memory: Mechanism Config namespace What gets offloaded PP restriction --- --- --- --- Activation offloading model.cpuoffloading Activations (and optionally weights) per transformer layer PP must be 1 Optimizer offloading optimizer.optimizercpuoffload Adam optimizer states (momentum + variance) via HybridDeviceOptimizer None Quick Decision Situation Recommendation --- --- Large MoE model (30B+), needs PP > 1 Optimizer offloading — activation offloading is blocked by PP=1 Small/medium model, PP=1 fits, activation memory dominates Activation offloading Want tunable memory-speed tradeoff Optimizer offloading with fractional optimizeroffloadfraction Throughput is top priority Don't enable — offloading always adds overhead CUDA graphs are needed Only optimizer offloading — activation offloading is incompatible Memory pressure is moderate Optimizer offload at 25–50% fraction for best efficiency Enablement Optimizer CPU offloading (recommended for large models) CLI overrides: Activation CPU offloading (small/medium models only) Config Parameter Reference Optimizer offloading Parameter Default Description ----------- --------- ------------- optimizercpuoffload False Master switch optimizeroffloadfraction 0.0 Fraction of optimizer states on CPU (0.0–1.0) overlapcpuoptimizerd2hh2d False Overlap GPU↔CPU transfers with compute usetorchoptimizerforcpuoffload False Use torch.optim instead of fused optimizer for CPU portion Activation offloading Parameter Default Description ----------- --------- ------------- cpuoffloading False Master switch cpuoffloadingnumlayers 0 Number of transformer layers to offload (0 to numlayers-1) cpuoffloadingactivations True Offload activations cpuoffloadingweights False Offload weights cpuoffloadingdoublebuffering False Double-buffer across la …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills nvidia/nemo-mbridge-perf-cpu-offloading