Agent skill · software engineering · nousresearch
accelerate
Run PyTorch training across GPUs with minimal changes.
Why this skill is useful
Adds executable scripts and workflows for distributed training with PyTorch that are not commonly known or documented.
What it needs
Requires accelerate, torch, transformers installed locally. About 4k tokens when loaded. Last updated 2026-08-07. 226,679 stars on the source repository.
What this skill does
HuggingFace Accelerate - Unified Distributed Training Quick start Accelerate simplifies distributed training to 4 lines of code. Installation: Convert PyTorch script (4 lines): Run (single command): Common workflows Workflow 1: From single GPU to multi-GPU Original script: With Accelerate (4 lines added): Configure (interactive): Questions: Which machine? (single/multi GPU/TPU/CPU) How many machines? (1) Mixed precision? (no/fp16/bf16/fp8) DeepSpeed? (no/yes) Launch (works on any setup): Workflow 2: Mixed precision training Enable FP16/BF16: Workflow 3: DeepSpeed ZeRO integration Enable DeepSpeed ZeRO-2 (pass a DeepSpeedPlugin, not a raw dict): Or point at a full DeepSpeed JSON config via the plugin: dsconfig.json (a raw DeepSpeed config — passed via the plugin, NOT via --configfile): Or via interactive config: Launch (--configfile expects an accelerate YAML, not a raw DeepSpeed JSON): Workflow 4: FSDP (Fully Sharded Data Parallel) Enable FSDP: Or via config: Workflow 5: Gradient accumulation Accumulate gradients: Effective batch size: batchsize numgpus gradientaccumulationsteps When to use vs alternatives Use Accelerate when: Want simplest distributed training Need single script for any hardware Use HuggingFace ecosystem Want flexibility (DDP/DeepSpeed/FSDP/Megatron) Need quick prototyping Key advantages: 4 lines: Minimal code changes Unified API: Same code for DDP, DeepSpeed, FSDP, Megatron Automatic: Device placement, mixed precision, sharding Interactive config: No manual launcher setup Single launch: Works everywhere Use alternatives instead: PyTorch Lightning: Need callbacks, high-level abstractions Ray Train: Multi-node orchestration, hyperparameter tuning DeepSpeed: Direct API control, advanced features Raw DDP: Maximum control, minimal abstraction Common issues Issue: Wrong device placement Don't manually move to device: Issue: Gradient accumulation not working Use context manager: Issue: Checkpointing in distributed Use accelerator methods: Issue: Differen …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills nousresearch/accelerate