Agent skill · boshu2
skill-eval
Author and tier behavioral probes for a skill, including seeded-defect probes that escape ceiling saturation. Triggers: "measure this skill", "the probe came back INERT", "the control arm aces it", "harden this scenario", "is this skill actually doing anything".
What it needs
About 7k tokens when loaded.
What this skill does
/skill-eval Author one behavioral probe for one skill, at the cheapest tier that can still separate the arms, and report the verdict honestly. A probe measures behavior-change — did loading the skill change what the agent did — never quality-uplift. This skill authors and tiers probes. scripts/probe-skill.sh runs them. Insight: when a probe returns INERT because the control arm already aces the scenario, the measurement failed, not the skill. Weakening the producer is one escape and it costs realism. The cheaper escape is to plant the defect: build a scenario containing exactly one flaw the discipline catches and a skim does not, then grade whether the agent acted on it. Signal you manufacture is signal you can reproduce. The failure mode this exists to prevent: a skill catalog whose tier badges are editorial. A skill nobody measured is a skill nobody can defend, and re-running a saturated scenario at a lower effort level produces more rows in the ledger without producing more knowledge. Modes Trigger phrases Mode Entry point --- --- --- "measure this skill", "does this skill do anything" author tier 1 (quiz probe) evals/skill-probes/<id>/ "the control arm aces it", "harden this scenario" author tier 2 (seeded-defect probe) references/seeding.md "the probe came back INERT" diagnose headroom gate skill.probe-headroom "run the probes" run a tier scripts/probe-skill.sh Inputs Required: the skill slug, and one sentence naming the action the skill should cause — a tool call made, an artifact written, a question raised, a sequence followed. If the sentence names a belief instead of an action ("understands that…", "considers…"), stop: that is not probeable, and rewriting it as an action is the actual work. Optional: an existing probe id to harden. Non-goals. This skill does not score output quality, rank skills, claim a skill is good, or gate a release. It does not run claude -p. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills boshu2/skill-eval--3b3cc8