Agent skill · research science · zechenzhangagi
miles-rl-training
Provides guidance for enterprise-grade RL training using miles, a production-ready fork of slime. Use when training large MoE models with FP8/INT4, needing train-inference alignment, or requiring speculative RL for maximum throughput.
Why this skill is useful
Adds executable scripts and domain-specific workflows for enterprise-grade reinforcement learning training that are not commonly found in public documentation.
What it needs
Requires docker, ray, sglang-router, torch, transformers installed locally. About 4k tokens when loaded. Last updated 2026-06-16. 11,472 stars on the source repository.
What this skill does
miles: Enterprise-Grade RL for Large-Scale Model Training miles is a high-performance, enterprise-ready RL framework optimized for large-scale model post-training. Built as a production fork of slime, it addresses critical challenges in MoE training stability, low-precision training, and train-inference alignment. When to Use miles Choose miles when you need: Training 1TB+ MoE models (DeepSeek V3, Qwen3-MoE) FP8 or INT4 quantization-aware training Bit-wise identical train-inference alignment Speculative RL for maximum throughput Production stability with enterprise support Consider alternatives when: You want the research-grade original → use slime You need flexible backend swapping → use verl You want PyTorch-native abstractions → use torchforge Key Features Low-Precision Training Unified FP8: End-to-end FP8 for both inference and training INT4 QAT: 1TB models on single-machine VRAM (H200) Rollout Routing Replay (R3): Bit-wise expert alignment for MoE Performance Optimizations Speculative RL: 25%+ rollout speedup with online SFT draft models Zero-Copy Weight Sync: CUDA IPC zero-copy mapping Partial Rollout: Recycle half-finished trajectories Train-Inference Alignment TIS/MIS: Truncated/Masked Importance Sampling for off-policy correction Kernel-level optimization: FlashAttention-3, DeepGEMM integration Installation Quick Start miles inherits slime's configuration system. Basic training: --- Workflow 1: Large MoE Training Use this workflow for training large MoE models like DeepSeek V3 or Qwen3-MoE. Prerequisites Checklist [ ] H100/H200 GPUs with FP8 support [ ] MoE model (DeepSeek V3, Qwen3-MoE) [ ] Docker environment with miles Step 1: Environment Setup Step 2: Configure Training Verification Checklist [ ] Model loads without errors [ ] Routing decisions are consistent [ ] No NaN/Inf in loss values --- Workflow 2: Speculative RL Training Use this workflow for maximum rollout throughput with EAGLE speculative decoding. How Speculative RL Works 1. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills zechenzhangagi/miles