Agent skill · research science · davila7
torchforge-rl-training
Provides guidance for PyTorch-native agentic RL using torchforge, Meta's library separating infra from algorithms. Use when you want clean RL abstractions, easy algorithm experimentation, or scalable training with Monarch and TorchTitan.
Why this skill is useful
Provides domain-specific knowledge and workflows for reinforcement learning using Meta's torchforge library that the AI wouldn't reliably generate on its own.
What it needs
Requires monarch, torch, torchtitan, vllm installed locally. About 5k tokens when loaded. Last updated 2026-08-06. 30,138 stars on the source repository.
What this skill does
torchforge: PyTorch-Native Agentic RL Library torchforge is Meta's PyTorch-native RL library that separates infrastructure concerns from algorithm concerns. It enables rapid RL research by letting you focus on algorithms while handling distributed training, inference, and weight sync automatically. When to Use torchforge Choose torchforge when you need: Clean separation between RL algorithms and infrastructure PyTorch-native abstractions (no Ray dependency) Easy algorithm experimentation (GRPO, DAPO, SAPO in ~100 lines) Scalable training with Monarch actor system Integration with TorchTitan for model parallelism Consider alternatives when: You need production-ready stability → use miles or verl You want Megatron-native training → use slime torchforge is experimental and APIs may change Key Features Algorithm isolation: Implement RL algorithms without touching infrastructure Scalability: From single GPU to thousands via Monarch Modern stack: TorchTitan (training), vLLM (inference), TorchStore (sync) Loss functions: GRPO, DAPO, CISPO, GSPO, SAPO built-in Architecture Overview Installation ROCm Installation Quick Start SFT Training (2+ GPUs) GRPO Training (3+ GPUs) --- Workflow 1: GRPO Training for Math Reasoning Use this workflow for training reasoning models with group-relative advantages. Prerequisites Checklist [ ] 3+ GPUs (GPU0: trainer, GPU1: refmodel, GPU2: generator) [ ] Model from HuggingFace Hub [ ] Training dataset (GSM8K, MATH, etc.) Step 1: Create Configuration Step 2: Define Reward Function Step 3: Launch Training Step 4: Monitor Progress [ ] Check W&B dashboard for loss curves [ ] Verify entropy is decreasing (policy becoming more deterministic) [ ] Monitor KL divergence (should stay bounded) --- Workflow 2: Custom Loss Function Use this workflow to implement new RL algorithms. Step 1: Create Loss Class Step 2: Integrate into Application --- Workflow 3: Multi-GPU Distributed Training Use this workflow for scaling to multiple GPUs or nodes. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills davila7/post-training-torchforge