Agent skill · research science · aradotso
awesome-autoresearch
awesome-autoresearch — an agent skill for coding agents.
Why this skill is useful
Provides a detailed implementation of an autonomous research loop with specific patterns and scripts for measuring and optimizing experiments that the AI wouldn't generate on its own.
What it needs
Requires npm installed locally. About 6k tokens when loaded. Last updated 2026-07-09. 70 stars on the source repository.
What this skill does
python Canonical keep-or-revert autoresearch loop import subprocess, shutil, json from pathlib import Path METRICCMD = ["python", "eval.py"] # returns JSON {"score": float} BUDGET = 20 # number of iterations GOAL = "maximize score" def measure() -> float: result = subprocess.run(METRICCMD, captureoutput=True, text=True) return json.loads(result.stdout)["score"] def runloop(agentproposefn): bestscore = measure() print(f"Baseline: {bestscore:.4f}") for step in range(BUDGET): # Agent proposes a diff/edit agentproposefn(goal=GOAL, step=step, best=bestscore) score = measure() if score > bestscore: bestscore = score print(f"[{step}] ✅ Improved → {score:.4f}") # Commit the change (git add -A && git commit) subprocess.run(["git", "commit", "-am", f"step {step}: {score:.4f}"]) else: print(f"[{step}] ❌ Reverted ({score:.4f} < {bestscore:.4f})") # Revert to last good state subprocess.run(["git", "checkout", "--", "."]) print(f"Final best: {bestscore:.4f}") markdown Autoresearch Loop You are running an autonomous improvement loop. Each iteration: 1. Read GOAL.md for the objective and metric command 2. Propose ONE focused change to the codebase 3. Apply the change 4. Run: python eval.py → parse {"score": float} 5. If score improves over baseline: git commit -am "step N: <score>" 6. Else: git checkout -- . 7. Log to experiments.jsonl 8. Repeat until BUDGET iterations or target score reached markdown GOAL.md Objective Minimize validation bits-per-byte on the Shakespeare dataset. Metric Command Returns: {"valbpb": float} — lower is better. Budget Max iterations: 20 Max wall time: 2 hours Constraints Single file edits only (model.py or train.py) No external API calls Must run on single A100 bash Install Codex CLI npm install -g @openai/codex Run autoresearch loop via Codex codex "Read GOAL.md. Run the autoresearch loop: propose a change, measure eval.py output, keep if improved else revert. Repeat 20 times. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/awesome-autoresearch