Agent skill · software engineering · davila7
huggingface-accelerate
Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
Why this skill is useful
Adds executable scripts for distributed training with automatic device placement and mixed precision that aren't in public docs.
What it needs
Requires accelerate, torch, transformers installed locally. About 4k tokens when loaded. Last updated 2026-08-06. 30,138 stars on the source repository.
What this skill does
HuggingFace Accelerate - Unified Distributed Training Quick start Accelerate simplifies distributed training to 4 lines of code. Installation: Convert PyTorch script (4 lines): Run (single command): Common workflows Workflow 1: From single GPU to multi-GPU Original script: With Accelerate (4 lines added): Configure (interactive): Questions: Which machine? (single/multi GPU/TPU/CPU) How many machines? (1) Mixed precision? (no/fp16/bf16/fp8) DeepSpeed? (no/yes) Launch (works on any setup): Workflow 2: Mixed precision training Enable FP16/BF16: Workflow 3: DeepSpeed ZeRO integration Enable DeepSpeed ZeRO-2: Or via config: deepspeedconfig.json: Launch: Workflow 4: FSDP (Fully Sharded Data Parallel) Enable FSDP: Or via config: Workflow 5: Gradient accumulation Accumulate gradients: Effective batch size: batchsize numgpus gradientaccumulationsteps When to use vs alternatives Use Accelerate when: Want simplest distributed training Need single script for any hardware Use HuggingFace ecosystem Want flexibility (DDP/DeepSpeed/FSDP/Megatron) Need quick prototyping Key advantages: 4 lines: Minimal code changes Unified API: Same code for DDP, DeepSpeed, FSDP, Megatron Automatic: Device placement, mixed precision, sharding Interactive config: No manual launcher setup Single launch: Works everywhere Use alternatives instead: PyTorch Lightning: Need callbacks, high-level abstractions Ray Train: Multi-node orchestration, hyperparameter tuning DeepSpeed: Direct API control, advanced features Raw DDP: Maximum control, minimal abstraction Common issues Issue: Wrong device placement Don't manually move to device: Issue: Gradient accumulation not working Use context manager: Issue: Checkpointing in distributed Use accelerator methods: Issue: Different results with FSDP Ensure same random seed: Advanced topics Megatron integration: See references/megatron-integration.md for tensor parallelism, pipeline parallelism, and sequence parallelism setup. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills davila7/distributed-training-accelerate