Agent skill · practicalswan
trl-training
Train and fine-tune transformer language models using TRL (Transformers Reinforcement Learning). Supports SFT, DPO, GRPO, KTO, RLOO and Reward Model training via CLI commands.
What it needs
About 5k tokens when loaded.
What this skill does
TRL Training Skill You are an expert at using the TRL (Transformers Reinforcement Learning) library to train and fine-tune large language models. Overview TRL provides CLI commands for post-training foundation models using state-of-the-art techniques: SFT (Supervised Fine-Tuning): Fine-tune models on instruction-following or conversational datasets DPO (Direct Preference Optimization): Align models using preference data GRPO (Group Relative Policy Optimization): Train models by ranking multiple sampled outputs relative to each other and optimizing based on their comparative rewards. RLOO (Reinforce Leave One Out): Online RL training with generation-based rewards Reward Model Training: Train reward models for RLHF TRL is built on top of Hugging Face Transformers and Accelerate, providing seamless integration with the Hugging Face ecosystem. Core Commands trl sft - Supervised Fine-Tuning Fine-tune language models on instruction-following or conversational datasets. Full training: Train with LoRA adapters: trl dpo - Direct Preference Optimization Align models using preference data (chosen/rejected pairs). Full training: Train with LoRA adapters: trl grpo - Group Relative Policy Optimization Train models using reward functions or LLM-as-a-judge for evaluating generations and providing rewards. Basic usage: trl rloo - Reinforce Leave One Out Online RL training where the model generates text and receives rewards based on custom criteria. Basic usage: trl reward - Reward Model Training Train a reward model to score text quality for RLHF. Full training: Train with LoRA adapters: Configuration Files TRL supports YAML configuration files for reproducible training. All CLI arguments can be specified in a config file. Example config (sftconfig.yaml): Launch with config: Override config values: Distributed Training TRL integrates with Accelerate for multi-GPU and multi-node training. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills practicalswan/trl-training