Agent skill · NVIDIA
nemo-mbridge-perf-cuda-graphs
Validate and use CUDA graph capture in Megatron Bridge, including local full-iteration graphs and Transformer Engine scoped graphs for attention, MLP, and MoE modules.
What it needs
About 6k tokens when loaded.
What this skill does
CUDA Graphs Stable documentation: @docs/training/cuda-graphs.md Card: @skills/nemo-mbridge-perf-cuda-graphs/card.yaml <!-- NVSkills CI refresh: 2026-06-15. No instruction changes. --> What It Is CUDA graphs capture GPU operations once and replay them with minimal host-driver overhead. Bridge supports two implementations: cudagraphimpl Mechanism Scope support --- --- --- "local" MCore FullCudaGraphWrapper wrapping entire fwd+bwd fulliteration "transformerengine" TE makegraphedcallables() per layer attn, mlp, moe, moerouter, moepreprocess, mamba Quick Decision Start with TE-scoped graphs for most training workloads, then verify replay timing against eager on the same dispatcher, layout, and container: dense models: attn, then optionally mlp dropless MoE: attn moerouter moepreprocess VLMs: the same dropless-MoE scope, but only after the real-data path is stable Use local + fulliteration only when you specifically want full-iteration capture and can satisfy the tighter constraints. For recompute-heavy workloads: TE-scoped graphs pair naturally with selective recompute full recompute usually pushes you toward local full-iteration graphs or away from graphs entirely Related docs: @docs/training/cuda-graphs.md @docs/training/activation-recomputation.md Enablement Local full-iteration graph TE scoped graph (dense model) TE scoped graph (MoE model) Performance harness CLI Valid CLI values live in scripts/performance/argumentparser.py: VALIDCUDAGRAPHIMPLS: ["none", "local", "transformerengine"] VALIDCUDAGRAPHSCOPES: ["fulliteration", "attn", "mlp", "moe", "moerouter", "moepreprocess", "mamba"] The performance harness uses a comma-separated --cudagraphscope value and auto-enables model.useterngtracker plus rng.terngtracker when --cudagraphimpl is not none. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills NVIDIA/nemo-mbridge-perf-cuda-graphs