Agent skill · pedronauck
autoresearch
Autonomously optimize any Claude Code skill by running it repeatedly, scoring outputs against binary evals, mutating the prompt, and keeping improvements. Based on Karpathy's autoresearch methodology. Use when: optimize this skill, improve this skill, run autoresearch on, make this skill better, self-improve skill, benchmark skill, eval my skill, run evals on. Outputs: an improved SKILL.md, a results log, and a changelog of every mutation tried.
What it needs
About 9k tokens when loaded.
What this skill does
Autoresearch for Skills Most skills work about 70% of the time. The other 30% you get garbage. The fix isn't to rewrite the skill from scratch. It's to let an agent run it dozens of times, score every output, and tighten the prompt until that 30% disappears. This skill adapts Andrej Karpathy's autoresearch methodology (autonomous experimentation loops) to Claude Code skills. Instead of optimizing ML training code, we optimize skill prompts. --- the core job Take any existing skill, define what "good output" looks like as binary yes/no checks, then run an autonomous loop that: 1. Generates outputs from the skill using test inputs 2. Scores every output against the eval criteria 3. Mutates the skill prompt to fix failures 4. Keeps mutations that improve the score, discards the rest 5. Repeats until the score ceiling is hit or the user stops it Output: An improved SKILL.md + results.tsv log + changelog.md of every mutation attempted + a live HTML dashboard you can watch in your browser. --- before starting: gather context STOP. Do not run any experiments until all fields below are confirmed with the user. Ask for any missing fields before proceeding. 1. Target skill — Which skill do you want to optimize? (need the exact path to SKILL.md) 2. Test inputs — What 3-5 different prompts/scenarios should we test the skill with? (variety matters — pick inputs that cover different use cases so we don't overfit to one scenario) 3. Eval criteria — What 3-6 binary yes/no checks define a good output? (these are your "test questions" — see eval-guide.md for how to write good evals) 4. Runs per experiment — How many times should we run the skill per mutation? Default: 5. (more runs = more reliable scores, but slower and more expensive. 5 is the sweet spot for most skills.) 5. Run interval — How often should experiments cycle? Default: every 2 minutes. (shorter = faster iteration, but costs more) 6. Budget cap — Optional. Max number of experiment cycles before stopping. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills pedronauck/autoresearch