Agent skill · code-yeongyu
eval-integrity
Assess integrity of an eval, benchmark, metric, A/B test, holdout, LLM-judge, or user-study claim when the question is whether measurement, comparison, or evidence is independent and trustworthy.
What it needs
About 3k tokens when loaded.
What this skill does
Evaluation Integrity Provide a read-only audit of whether an evaluation can support its stated claim. Do not alter code, data, prompts, labels, or study operations. Begin with exactly one verdict line in the form Verdict: PASS, Verdict: FAIL, or Verdict: INCONCLUSIVE. Emit no other verdict line and do not restate the verdict later in the report. A favorable score is not itself evidence of integrity. Route correctly NOT for code review: route implementation, correctness, security, or maintainability review to review-work. NOT for data processing: route cleaning, transforming, joining, modeling, or analyzing data to data-scientist. Use this skill for the independence and validity of the measurement setup, not for producing a result from it. Audit method 1. State the claim and map the evaluation Write the claim in testable form: what comparison, metric, threshold, or causal statement is being made; for which population; and under which decision rule. Map every role before judging it: Element Record --- --- Claim The conclusion and decision it is meant to justify. System under test Exact model, version, configuration, prompt, policy, or treatment. Dataset and split Sources, collection dates, inclusion rules, train/dev/test or holdout assignment. Designer Who selected the task, data, prompts, treatments, metric, and stopping rule. Evaluator or judge Who or what assigns quality judgments, including model and prompt versions. Scorer The rule or system that turns outputs or judgments into the reported metric. Subjects Humans, records, tasks, or units exposed to treatment or observation. Trace shared ownership, shared data, shared prompts, shared labels, and shared selection decisions across this map. Several models do not establish independence merely because there are several of them; identify distinct training, tuning, prompt design, labels, data, and decision authority. 2. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills code-yeongyu/eval-integrity