--- name: integration-test-verify-loop version: 1.0.0 description: "[Testing] Use when you need to drive an integration-test suite to fully green — each round runs /integration-test-verify (whole system by default, or the target named in the prompt), and on ANY failure combines /debug-investigate + /integration-test-review (report-only) to adjudicate the fault (test wrong · test not optimal · source wrong), then /fix to resolve it at the owning layer, then /changes-review on that round's fix diff, then re-runs a FRESH full verify — looping until the whole suite passes its 2-consecutive-green-runs gate with zero failures." --- > **[BLOCKING]** Execute skill steps in declared order. NEVER skip, reorder, or merge steps without explicit user approval. > **[BLOCKING]** Before each step or sub-skill call, update task tracking: set `in_progress` when step starts, set `completed` when step ends. > **[BLOCKING]** Every completed/skipped step MUST include brief evidence or explicit skip reason. > **[BLOCKING]** If Task tools are unavailable, create and maintain an equivalent step-by-step plan tracker with the same status transitions. ## Quick Summary **Goal:** Drive an integration-test suite to **fully green** by pairing `/integration-test-verify` with a combined fault-adjudication + `/fix` half in a recursive loop — each round runs a FRESH full `/integration-test-verify` over `{scope}` (**the WHOLE system by default**, or the target named in the prompt), and on ANY failure runs `/debug-investigate` **and** `/integration-test-review` (report-only) together to decide _who is at fault_, then `/fix` at the owning layer — stopping only when a fresh full verify passes its **2-consecutive-green-runs-without-DB-reset** gate with **zero failed tests**, proven by actual runner output. **Summary:** - **Each round = verify (find) + adjudicate (diagnose) + `/fix` (resolve) + `/changes-review` (prove the fix is sound).** `/integration-test-verify` reports pass/fail but must not own the fix; the adjudication pair decides _test-wrong vs test-not-optimal vs source-wrong_; `/fix` lands the change; `/changes-review` code-reviews that change before the next round. A round is incomplete until all four have run (or the verify returned zero failures). - **Steps (in order):** (0) resolve `{scope}` + Goal Contract → (0b) bind the convergence loop (protocol loop primary + optional `/goal` accelerator) → (1) round loop { snapshot → run `/integration-test-verify` INLINE over `{scope}` → **on failure:** `/debug-investigate` + `/integration-test-review` report-only → emit ONE Fault Verdict per failure → `/fix` at the owning layer → **CONDITIONAL `/changes-review` on the round's fix diff when any fix landed** → Round Integrity Check → log iteration } → (2) converge on a zero-failure fresh verify OR escalate → (3) terminal spec/doc sync + recap. - **Every fix gets code-reviewed in the round that lands it.** Whenever a round applies ANY fix — source, test, scenario, or spec — that round runs `/changes-review` (report-only, INLINE) over the round's fix diff; validated findings fold back into the SAME round's fix set. No fix applied → skipped with a recorded reason. — why: a green test cannot see a wrong-layer fix, a broken invariant elsewhere, or a security/performance regression. - **Default scope is the WHOLE SYSTEM.** With no target in the prompt, `{scope}` = **every** discovered integration-test project (`testProjectPattern` glob > `testProjects` list) — NOT the git-changed subset. The loop passes `{scope}` to `/integration-test-verify` **explicitly** so it never falls through to its change-scoped default (`integration-test-verify/SKILL.md:132,159`). - **Convergence:** stop ONLY when a **fresh full** `/integration-test-verify` over the CURRENT (post-fix) code reports **zero failed tests across 2 consecutive runs without a DB reset**, with real runner output — never a stale green predating the last fix. - **Inline invariant:** run `/integration-test-verify`, `/debug-investigate`, and `/integration-test-review` via the `Skill` tool, NEVER the `Agent` tool. `/debug-investigate` requires its `/why-review` gate **in the same session/main agent** (`debug-investigate/SKILL.md:34`), and `/integration-test-review` self-binds its own fix + re-review obligations — a sub-agent cannot own or carry either back to this loop. Their OWN internal fan-outs (verify's `integration-tester` per-project sub-agents, `integration-test-review`'s phase agents) stay sub-agents by their own design, so context stays bounded. - **No fake green — Round Integrity Check:** a round converges only if the executed test count did **not shrink** and the skipped count did **not grow** versus the prior round. Deleting, skipping, or narrowing tests is a REGRESSION, never convergence. - **Bounded:** round cap default 3; failing count not shrinking across 2 rounds, or cap hit with failures still open → **STOP & escalate** via `AskUserQuestion`. Increasing failures → STOP (fixes regressing). Environment/infrastructure fault → **BLOCKED**, escalate immediately — never loop against an unhealthy system. **Why this skill exists (READ FIRST — it is the whole justification):** the obligation to loop already exists as **prose scattered across three skills**, with no mechanism behind it. `/integration-test-verify` says _"After fixing → re-run the full 2-run verify sequence"_ (`integration-test-verify/SKILL.md:238`) and its `SYNC:integration-test-execution-discipline` §5 says _"Loop until the whole suite is green"_ (`:404`) — but there is **no round cap, no Goal Contract, no shrinking-failures gate, and no escalation path**, so an agent that fixes one test and reports success is not violating anything mechanical. Worse, the fix half is **triple-owned and undefined**: `/integration-test-review` fixes tests and re-reviews itself (P5–P8), `/fix --target=test` runs its own unbounded _"if tests fail, repeat from step 2"_ (`fix/SKILL.md:218`), and verify's own failure protocol says fix-and-re-run (`:232-238`) — three overlapping loops that can double-fix the same failure or each assume another owns it. This skill makes the loop a **bounded, evidence-gated convergence contract with one owner**: verify FINDS, the `/debug-investigate` + `/integration-test-review` pair ADJUDICATES fault, `/fix` RESOLVES, and a fresh full verify RE-PROVES — with a Round Integrity Check so the suite can never go "green" by losing tests. Without it, "tests failed, then something fixed them" ships on a single green run over a hand-picked subset. **Workflow:** resolve `{scope}` (whole system by default) + Goal Contract → bind the convergence loop → **round loop** { run `/integration-test-verify` INLINE → on failure run `/debug-investigate` + `/integration-test-review` report-only → emit Fault Verdict → `/fix` at owning layer → conditional `/changes-review` on the round's fix diff → Round Integrity Check → log iteration } → converge when a fresh full verify is 2/2 green with zero failures → terminal `/spec [mode=sync]` + `/docs-update` + recap. **Key Rules:** - **Each round pairs verify (find) + adjudicate (diagnose) + `/fix` (resolve) + `/changes-review` (review the fix).** Never skip the adjudication half and jump straight to a fix — an unadjudicated failure gets "fixed" by whatever is nearest, which is almost always the assertion. - **CONDITIONAL `/changes-review` every round that lands a fix.** Working tree changed vs the round's snapshot → run `/changes-review` INLINE, report-only, scoped to the round's fix diff; validate its findings and fold them into the SAME round's fix set; unfixable validated findings → STOP & escalate. Working tree unchanged → skip with a recorded reason. NEVER let a round's fixes reach the next round un-reviewed. - **Default `{scope}` = the WHOLE system**; a target named in the prompt narrows it. Pass `{scope}` to `/integration-test-verify` EXPLICITLY every round — never let it fall back to git auto-detect. - **MUST run INLINE via the `Skill` tool — NEVER dispatch `/integration-test-verify`, `/debug-investigate`, `/integration-test-review`, or `/changes-review` as a sub-agent** (their in-session gates are lost). Their internal fan-outs remain sub-agents by their own design. - **`/integration-test-review` runs REPORT-ONLY.** It performs its 8-gate review and STOPS before its P5 fix / P6 re-review / P7 build-and-run — the loop owns fixing and re-running. If it cannot be constrained and self-fixes anyway, treat that as this round's fix half and skip the `/fix` step for that round — never double-fix. - **One written Fault Verdict per failure BEFORE any edit** — `TEST-WRONG` · `TEST-NOT-OPTIMAL` · `SOURCE-WRONG` · `ENVIRONMENT-BLOCKED` · `AMBIGUOUS` — with `file:line` evidence and confidence. `AMBIGUOUS` → `AskUserQuestion`, never a silent pick. - **NEVER force green.** No weakened or removed assertions, no skip annotations, no widened assertion timeouts, no retries around a failing assertion, no repository-hacked domain data, no narrowed scope. Fix the SCENARIO (an ARRANGE barrier on a real observable) or the product defect. - **Convergence = a fresh full verify over the post-fix code, 2/2 green, zero failures, real runner output, Round Integrity Check passed.** All five, or it is not converged. - **Round cap (default 3)**; failures not shrinking across 2 rounds, increasing, or cap hit with failures open → **STOP & escalate** via `AskUserQuestion`. `ENVIRONMENT-BLOCKED` → escalate immediately; never loop against an unhealthy system. --- ## First Principle — Convergence, Not Motion > A round that changes the code is progress **only if** the next fresh verify has fewer failing tests. > The loop exists to reach a fixed point (a whole suite that is green twice in a row), not to keep editing until something passes. > If the failing count stops shrinking, that is a signal to **escalate**, not to spin another round. > And a suite that got greener by having fewer tests did not converge — it regressed. --- ## Step 0 — Resolve Verification Scope + Goal Contract (FIRST ACTION) 1. **Read `docs/project-config.json` → `integrationTestVerify`** before anything else (the same config `/integration-test-verify` obeys, `integration-test-verify/SKILL.md:74-92`). You need `quickRunCommand`, `testProjectPattern`, `testProjects`, `systemCheckCommand`, `startupScript`, and `referenceDocs` to resolve scope and to read the project's integration-test reference docs. 2. **Resolve `{scope}` — WHOLE SYSTEM by default:** | Prompt | `{scope}` | | ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | | **No target named** (default) | **Every** integration-test project discoverable via `testProjectPattern` glob > `testProjects` list. The WHOLE system. | | Names a suite/project/module/feature | Only the test projects covering that target, resolved from the same config; state which projects the target maps to and how you resolved it. | | Names a diff/branch/PR | The test projects covering that change set — this is the ONLY case where change-scoping is correct, and it must be explicit in the prompt. | > **NEVER let `/integration-test-verify` resolve scope by itself.** Its Step 3 priority ends in git auto-detect and _"Only run projects relevant to the current change"_ (`integration-test-verify/SKILL.md:132,159`) — correct when it is a workflow step after an edit, WRONG as this loop's default. Pass `{scope}` explicitly in the invocation every round. — why: a loop that silently verifies only the changed subset reports "all green" for a system it never ran. 3. **Record `{scope}` as a stable string** (the resolved project list). The scope is FIXED for the whole loop — it never narrows as rounds progress. If a fix legitimately ADDS a test project, widen `{scope}` and say so in the Iteration Log; narrowing it is forbidden. 4. **Resolve/create the Goal Contract** per `SYNC:goal-contract-satisfaction-loop` (`plans/goals/{YYMMDD-HHmm}-{slug}/goal.md`, template `.claude/templates/goal-contract-template.md`). Its single **required** Success Criterion: > _A fresh full `/integration-test-verify` over `{scope}` reports **zero failed tests** across **2 consecutive runs without a DB reset**, evidenced by actual test-runner output (Passed/Failed/Skipped counts), with no test deleted, skipped, or weakened to get there._ Record in **Constraints**: `{scope}` (the project list), the round cap (default 3), the baseline executed/skipped test counts once round 1 reports them, and `quickRunCommand`. ## Step 0b — Bind the Convergence Loop (protocol-first; `/goal` is an optional accelerator) The convergence loop is bound by TWO layers. The **protocol loop (Steps 1–2) is the BINDING mechanism** and MUST be self-driven by you, the running agent, on every host — with or without any command or hook. The **`/goal` command is an OPTIONAL accelerator** layered on top; it is never the primary mechanism, and its absence NEVER weakens the loop. This mirrors the project rule that hooks/trackers are accelerators only — correctness must not depend on them. **1. Protocol loop — ALWAYS binding (hook/command-independent).** You are personally responsible for not stopping until the loop converges or bounded-escalates. This binds Claude, Codex, and Copilot equally, whether or not `/goal` exists: > Repeatedly run `/integration-test-verify` INLINE over `{scope}` (passed explicitly, never re-derived). After each run, if ANY test failed, adjudicate every failure with `/debug-investigate` + `/integration-test-review` (report-only) into ONE Fault Verdict, apply the fix via `/fix` at the owning layer, then re-run a FRESH full `/integration-test-verify` over `{scope}`. Do NOT stop while the last verify still reported a failing test. Converge ONLY when a fresh full verify reports zero failures across 2 consecutive runs without a DB reset AND the Round Integrity Check passes (executed test count not shrunk, skipped count not grown). Cap at `{N=5}` rounds; if the failing count does not shrink across 2 consecutive rounds, failures increase, the cap is hit with failures still open, or any failure is ENVIRONMENT-BLOCKED → STOP and escalate via `AskUserQuestion`. Never loop open-ended, and NEVER reach green by weakening, skipping, deleting, or de-scoping a test. Treat this as a standing obligation you re-read at every Step 2 checkpoint — NOT a one-time note you can rationalize away after the first fix cycle. The Goal Contract's required Success Criterion (Step 0) is its durable, host-independent record. **2. `/goal` command — invoke as an accelerator WHEN AVAILABLE.** If a `/goal` command exists and you are permitted to run it in this environment, ALSO invoke it (a real tool/command call, NOT a paraphrase, NOT a Goal Contract file substituted for it) with the SAME condition, so a session Stop hook mechanically enforces the loop: ``` /goal integration-test green convergence loop: repeatedly run /integration-test-verify INLINE over {scope} (passed explicitly). If any test failed → adjudicate each failure with /debug-investigate + /integration-test-review (report-only) into one Fault Verdict, fix via /fix at the owning layer, and run another round. If a fresh full verify reports zero failures across 2 consecutive runs without DB reset AND executed test count has not shrunk and skipped count has not grown → CONVERGED, run the terminal /spec [mode=sync] + /docs-update and clear the gate. Do NOT stop while the last verify still reported a failing test. Cap at {N=5} rounds; if the failing count does not shrink across 2 consecutive rounds, failures increase, the cap is hit with failures open, or a failure is ENVIRONMENT-BLOCKED → STOP and escalate via AskUserQuestion. Never loop open-ended; never reach green by weakening, skipping, deleting, or de-scoping a test. ``` The `/goal` Stop hook blocks stopping until the condition holds and auto-clears when met — do not tell the user to clear it. **If `/goal` is unavailable, unregistered, or not permitted** (e.g. Codex/Copilot, or a Claude run without the command): DO NOT error, DO NOT block, and DO NOT invent a stand-in gate. Record ONE line in the Goal Contract — `/goal accelerator unavailable — loop bound by protocol (Steps 1–2) + this Goal Contract` — and proceed. The protocol loop above plus the Goal Contract are the same gate, enforced by discipline instead of a hook. > **Nested gates (by design, safe):** each inner `/debug-investigate` self-binds its own `/why-review` validation gate that clears when its root cause is validated, and `/integration-test-review` runs REPORT-ONLY so its P5 fix / P6 re-review loop is deferred to this caller — no inner fix gate is installed. THIS outer loop owns the single convergence gate. All self-clear on satisfaction — no orphaned gate. Do NOT tell the user to clear any of them. ## Step 1 — Round Loop (verify → adjudicate → fix → review → integrity-check → log) Each round couples four halves — **verify to find, adjudicate to diagnose, fix to resolve, `/changes-review` to prove the fix itself is sound.** For each round `R` (starting at 1), do ALL of: 1. **Snapshot before:** record the working-tree fingerprint — `git status --porcelain` + `git diff --stat`. This is the fixes-applied baseline for the round and the objective backstop for convergence detection (Step 2). 2. **Run `/integration-test-verify` INLINE** via the `Skill` tool (NEVER the `Agent` tool), passing `{scope}` **explicitly** so it skips its own scope derivation. It runs its full contract: system check → the named projects → the **2-consecutive-green-runs-without-DB-reset** gate → a report with real Passed/Failed/Skipped counts and failing test names. Let it fan out its own `integration-tester` sub-agents per isolated project (its design, `integration-test-verify/SKILL.md:188-206`) — that fan-out is bounded and correct. **Tell it explicitly that this run IS a round of this loop**, so it returns its counts + failing names to this loop instead of recommending `/workflow-integration-test-green` as a next step — why: that recommendation is correct for a standalone verify but circular here, and would restart the loop that is already running. 3. **Record the round's counts:** executed, passed, failed, skipped — per project and total, from **actual runner output**. These feed the Round Integrity Check (1.8) and the shrinking-failures gate (Step 2). No output = no counts = no claim. 4. **If failures = 0** and the 2-run gate was green → this round converged; go to Step 2 (no adjudication or fix half needed). 5. **If failures > 0 — ADJUDICATE (the combined half). Run BOTH, INLINE, in this order, per failure or per failure cluster:** **(a) `/debug-investigate`** — trace the failure end-to-start to the defect's owning layer, producing a `file:line` root cause with a confidence score, validated through its own `/why-review` gate. It is investigation-ONLY — it never patches (`debug-investigate/SKILL.md:22`). **(b) `/integration-test-review` — REPORT-ONLY** — scoped to the failing tests **and the production code they exercise**. Its 8 gates supply the test-side verdict: G1 assertion value (mutation probe), G2 data state, G3 repeatability, G4 domain logic (does the test assert only what the handler writes?), G5 spec traceability, G6 three-way sync, G7 change coverage, G8 scenario fidelity. **Direct it to STOP after its findings report** — no P5 fix, no P6 re-review, no P7 build-and-run; this loop owns fixing and re-running. **Combine (a) + (b) into ONE written Fault Verdict per failure, BEF