Agent skill · data analytics · borghei
agent-harness
Test and evaluation harness for AI agents — scenario suites, deterministic replay, regression diffing, cost and latency budgets. Use when agent quality is vibe-checked, before shipping a prompt or model change, or when evals drift.
Why this skill is useful
Adds executable scripts for scenario testing and regression evaluation that are not commonly found in public documentation.
What it needs
Requires python3 installed locally. About 7k tokens when loaded. Last updated 2026-07-21. 462 stars on the source repository.
What this skill does
Agent Harness Most agents ship on vibes: someone tries eight prompts, the output looks good, it goes to production, and the next prompt tweak silently breaks a refusal nobody re-tested. This skill builds the harness around an agent so its behaviour becomes measurable — scenario suites with structural assertions, deterministic replay of recorded tool calls, paired regression diffing across prompt and model changes, and per-scenario cost and latency budgets. The tools here score an agent; they never invoke one, so they run offline on every commit. When to use this skill An agent is going to production and the only quality evidence is manual spot-checking A prompt, tool schema, or model version is changing and you need to know what broke Two model or configuration options need a defensible comparison, not a demo An incident happened and you need the behaviour encoded as a permanent regression test Agent cost or latency is climbing across releases and nobody can point to when An existing eval suite reports a healthy pass rate that nobody trusts Inputs the skill expects The agent's tool inventory — names, arguments, and which tools are irreversible Recorded transcripts per scenario: tool calls, final output, turns, latency, cost, error state The behavioural rules the agent must hold (refusals, escalation triggers, policy boundaries) Known failure history — past incidents, customer complaints, internal bug reports Current cost and latency expectations per interaction The release gate that consumes the result (CI job, review checklist, launch review) Clarify First Before building the harness, confirm these inputs. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills borghei/agent-harness