Agent skill · research science · aradotso
openclaw-rl-training
OpenClaw-RL framework for training personalized AI agents via reinforcement learning from natural conversation feedback
Why this skill is useful
Adds executable scripts for training personalized AI agents using OpenClaw-RL, which are not available in public documentation.
What it needs
Requires git, pip installed locally. About 6k tokens when loaded. Last updated 2026-07-09. 70 stars on the source repository.
What this skill does
OpenClaw-RL Training Skill by ara.so — Daily 2026 Skills collection. OpenClaw-RL is a fully asynchronous reinforcement learning framework that converts live multi-turn conversations into training signals for personalized AI agents. It wraps a self-hosted model as an OpenAI-compatible API via OpenClaw, intercepts conversations, and continuously optimizes the policy in the background without interrupting usage. It also supports scalable RL for terminal, GUI, SWE, and tool-call agents. Architecture Overview Four independent async loops that never block each other: 1. Agent Serving — OpenClaw-compatible API serving rollouts 2. Rollout Collection — Captures multi-turn conversations as training trajectories 3. PRM/Judge Evaluation — Scores turns using next-state feedback (majority voting optional) 4. Policy Training — GRPO/OPD/Combine training via slime or Tinker Installation Project Structure Three Learning Paradigms 1. Binary RL (GRPO) A Process Reward Model scores each turn from next-state feedback. Uses GRPO advantage estimation with PPO-style clipped surrogate loss. 2. On-Policy Distillation (OPD) When next state reveals useful hindsight, a judge extracts a textual hint to augment the prompt, creating an enhanced teacher. Token-level log-probability gap becomes a directional advantage signal. 3. Combination Method (Recommended) Merges Binary RL scalar supervision with OPD token-level directional signal. Strongest and most robust optimization. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/openclaw-rl-training