Agent skill · vasilyu1983
ai-post-training
Post-training and alignment: reward modeling, RLHF/PPO, DPO/DAAs, GRPO, RLVR, RLAIF, over-optimization. Use when adapting an SFT model with preference or verifiable-reward signals.
What it needs
About 11k tokens when loaded.
What this skill does
AI Post-Training Domain: the rung after supervised fine-tuning — turning a pretrained or SFT'd base model into an aligned, preference-tuned, or reasoning-capable model with a reward signal. This skill owns the post-training decision and pipeline: when to post-train at all, which reward signal you can produce, which algorithm family fits, and how to keep it from over-optimizing. Per-algorithm operational depth lives in ai-llm/references/post-training.md (PPO, DPO, SimPO, KTO, GRPO, GSPO, DAPO, RLVR, RULER, ORPO — catalogue + decision tree); this skill routes there. It does not cover: pretraining (ai-pretraining), the prompt→RAG→SFT promotion ladder (ai-architecture-advisor), or serving the result (ai-llm-inference). Quick Reference You have / want Method Deep ref --- --- --- Labeled demonstrations of the target behavior SFT (baseline — exhaust it first; not RL) ai-llm Pairwise preferences, want the least machinery DPO (or DAAs: KTO / ORPO / SimPO) methods A stronger teacher model, a small student On-policy distillation — try before GRPO methods Preferences + reward model + online RL GRPO / RLOO (critic-free, 2026 default); PPO is the reference algorithm, now trl.experimental methods Many samples scorable per prompt, drop the critic GRPO (group-relative advantage) methods A real task with no mechanical checker Rubrics as rewards (the fourth reward source) methods A multi-turn agent acting in an environment Agentic RL (trajectory reward, rollout infra) methods A verifiable checker (math/code/tests) as the reward RLVR (via GRPO or a GRPO-family variant — GSPO/DAPO/RLOO) — the dominant 2026 reasoning recipe methods Scale preference labels cheaply RLAIF / Constitutional AI (model-as-judge) data A quick lift with no RL loop Rejection sampling (best-of-N → SFT) methods Train/choose the reward model itself Bradley-Terry RM, ORM vs PRM, generative RM reward Stop reward hacking / over-refusal KL regularization, eval harness, over-optimization controls over-optimization Interpr …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills vasilyu1983/ai-post-training