Agent skill · software engineering · davila7
pytorch-lightning
High-level PyTorch framework with Trainer class, automatic distributed training (DDP/FSDP/DeepSpeed), callbacks system, and minimal boilerplate. Scales from laptop to supercomputer with same code. Use when you want clean training loops with built-in best practices.
Why this skill is useful
Provides a structured framework for distributed training in PyTorch, including automatic device management and built-in best practices that the AI wouldn't reliably generate on its own.
What it needs
Requires lightning, torch, transformers installed locally. About 4k tokens when loaded. Last updated 2026-08-06. 30,138 stars on the source repository.
What this skill does
PyTorch Lightning - High-Level Training Framework Quick start PyTorch Lightning organizes PyTorch code to eliminate boilerplate while maintaining flexibility. Installation: Convert PyTorch to Lightning (3 steps): That's it! Trainer handles: GPU/TPU/CPU switching Distributed training (DDP, FSDP, DeepSpeed) Mixed precision (FP16, BF16) Gradient accumulation Checkpointing Logging Progress bars Common workflows Workflow 1: From PyTorch to Lightning Original PyTorch code: Lightning version: Benefits: 40+ lines → 15 lines, no device management, automatic distributed Workflow 2: Validation and testing Automatic features: Validation runs every epoch by default Metrics logged to TensorBoard Best model checkpointing based on valloss Workflow 3: Distributed training (DDP) Launch: No changes needed: Automatic data distribution Gradient synchronization Multi-node support (just set numnodes=2) Workflow 4: Callbacks for monitoring Result: Auto-saves best 3 models Stops early if no improvement for 5 epochs Logs learning rate to TensorBoard Workflow 5: Learning rate scheduling When to use vs alternatives Use PyTorch Lightning when: Want clean, organized code Need production-ready training loops Switching between single GPU, multi-GPU, TPU Want built-in callbacks and logging Team collaboration (standardized structure) Key advantages: Organized: Separates research code from engineering Automatic: DDP, FSDP, DeepSpeed with 1 line Callbacks: Modular training extensions Reproducible: Less boilerplate = fewer bugs Tested: 1M+ downloads/month, battle-tested Use alternatives instead: Accelerate: Minimal changes to existing code, more flexibility Ray Train: Multi-node orchestration, hyperparameter tuning Raw PyTorch: Maximum control, learning purposes Keras: TensorFlow ecosystem Common issues Issue: Loss not decreasing Check data and model setup: Issue: Out of memory Reduce batch size or use gradient accumulation: Issue: Validation not running Ensure you pass valloader: Issue: DDP spawns mul …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills davila7/distributed-training-pytorch-lightning