Agent skill · research science · davila7
verl-rl-training
Provides guidance for training LLMs with reinforcement learning using verl (Volcano Engine RL). Use when implementing RLHF, GRPO, PPO, or other RL algorithms for LLM post-training at scale with flexible infrastructure backends.
Why this skill is useful
Adds executable scripts and domain-specific workflows for training LLMs with reinforcement learning that are not commonly found in public documentation.
What it needs
Requires ray, torch, transformers, verl, vllm installed locally. About 5k tokens when loaded. Last updated 2026-08-06. 30,138 stars on the source repository.
What this skill does
verl: Volcano Engine Reinforcement Learning for LLMs verl is a flexible, efficient, and production-ready RL training library for large language models from ByteDance's Seed team. It implements the HybridFlow framework (EuroSys 2025) and powers models like Doubao-1.5-pro achieving O1-level performance on math benchmarks. When to Use verl Choose verl when you need: Production-ready RL training at scale (tested up to 671B parameters) Flexibility to swap backends (FSDP ↔ Megatron-LM ↔ vLLM ↔ SGLang) Support for multiple RL algorithms (PPO, GRPO, RLOO, REINFORCE++, DAPO) Multi-turn rollout with tool calling for agentic workflows Vision-language model RL training Consider alternatives when: You need Megatron-native training → use slime or miles You want PyTorch-native abstractions with Monarch → use torchforge You only need simple SFT/DPO → use TRL or Axolotl Key Features Training backends: FSDP, FSDP2, Megatron-LM Rollout engines: vLLM, SGLang, HuggingFace Transformers Algorithms: PPO, GRPO, DAPO, RLOO, ReMax, REINFORCE++, SPIN, SPPO Models: Qwen-3, Llama-3.1, DeepSeek, Gemma-2 (0.5B to 671B) Advanced: LoRA RL, sequence parallelism, expert parallelism, multi-turn tools Installation Quick Start: GRPO Training Core Architecture verl uses a HybridFlow programming model separating control flow from computation: --- Workflow 1: Math Reasoning with GRPO Use this workflow for training reasoning models on math tasks like GSM8K or MATH. Prerequisites Checklist [ ] GPU cluster with 8+ GPUs (H100 recommended) [ ] Dataset in parquet format with prompt and rewardmodel columns [ ] Base model from HuggingFace Hub Step 1: Prepare Dataset Step 2: Define Reward Function Step 3: Create Training Config Step 4: Launch Training Step 5: Monitor and Validate [ ] Check WandB/TensorBoard for loss curves [ ] Verify reward is increasing over steps [ ] Run evaluation on held-out test set --- Workflow 2: PPO with Critic Model Use this workflow when you need value-based advantage estimation (GAE). …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills davila7/post-training-verl