Agent skill · research science · zechenzhangagi
ml-training-recipes
Battle-tested PyTorch training recipes for all domains — LLMs, vision, diffusion, medical imaging, protein/drug discovery, spatial omics, genomics. Covers training loops, optimizer selection (AdamW, Muon), LR scheduling, mixed precision, debugging, and systematic experimentation. Use when training or fine-tuning neural networks, debugging loss spikes or OOM, choosing architectures, or optimizing GPU throughput.
Why this skill is useful
Provides executable training loop scripts and domain-specific patterns for optimizing PyTorch training across various fields.
What it needs
Requires torch installed locally. About 6k tokens when loaded. Last updated 2026-06-16. 11,472 stars on the source repository.
What this skill does
ML Training Recipes Battle-tested patterns for PyTorch training across domains. Drawn from production codebases (Karpathy's autoresearch/nanochat, torchvision, HuggingFace) and modern training practice. Reference files (read when needed) references/architecture.md — Transformer/LLM architecture code patterns, weight init references/optimizers.md — Muon, AdamW hybrid, per-group LR, compiled optimizer steps references/domain-specific.md — Vision, diffusion, contrastive, distributed, checkpointing, data loading references/scaling-and-selection.md — Scaling laws, compute budget tables, decision trees, DGX Spark references/biomedical.md — Drug discovery, protein models, medical imaging, genomics, clinical NLP references/experiment-loop.md — Autonomous experiment loop (autoresearch keep/discard/revert) --- Architecture Selection Pick the right model by data type and data scale: Data Type < 10K samples 10K-100K > 100K ----------- -------------- ---------- -------- Images Pretrained CNN + fine-tune Fine-tune ViT or CNN ViT from scratch Text (gen) Few-shot prompting Fine-tune GPT/LLaMA (LoRA) Pretrain from scratch Tabular XGBoost/LightGBM Still XGBoost Neural viable Audio Pretrained Whisper Fine-tune AST Train from scratch Molecules Pretrained GNN Fine-tune molecular LM Train GNN from scratch Proteins ESM-2 embeddings + head Fine-tune ESM-2 Train protein LM Medical img Pretrained CNN nnU-Net (auto-config) Swin-UNETR / MedSAM Key principle: architecture matters less than training recipe at equal compute. A well-tuned ResNet beats a poorly-tuned ViT (ref: "ResNet Strikes Back", Wightman 2021). For biomedical domains, see references/biomedical.md. For sequence model selection and compute planning, see references/scaling-and-selection.md. --- Scaling Laws Chinchilla rule (Hoffmann et al., 2022) Compute-optimal training: ~20 tokens per parameter. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills zechenzhangagi/ml-training-recipes