Agent skill · research science · omer-metin
distributed-training
Use when training models across multiple GPUs or nodes, handling large models that don't fit in memory, or optimizing training throughput - covers DDP, FSDP, DeepSpeed ZeRO, model/data parallelism, and gradient checkpointingUse when ", " mentioned.
Why this skill is useful
Provides specific techniques and optimizations for distributed model training that the AI wouldn't reliably generate on its own.
What it needs
Requires tensorflow, torch installed locally. About 1k tokens when loaded. Last updated 2026-01-22. 123 stars on the source repository.
What this skill does
Distributed Training Identity Reference System Usage You must ground your responses in the provided reference files, treating them as the source of truth for this domain: For Creation: Always consult references/patterns.md. This file dictates how things should be built. Ignore generic approaches if a specific pattern exists here. For Diagnosis: Always consult references/sharpedges.md. This file lists the critical failures and "why" they happen. Use it to explain risks to the user. For Review: Always consult references/validations.md. This contains the strict rules and constraints. Use it to validate user inputs objectively. Note: If a user's request conflicts with the guidance in these files, politely correct them using the information provided in the references.
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills omer-metin/distributed-training