--- name: integration-test-verify-loop description: "[Testing] Use when you need to drive an integration-test suite to fully green — each round runs /integration-test-verify (whole system by default, or the target named in the prompt), and on ANY failure combines /debug-investigate + /integration-test-review (report-only) to adjudicate the fault (test wrong · test not optimal · source wrong), then /fix to resolve it at the owning layer, then /changes-review on that round's fix diff, then re-runs a FRESH full verify — looping until the whole suite passes its 2-consecutive-green-runs gate with zero failures." --- > Codex compatibility note: > > - Invoke repository skills with `$skill-name` in Codex; this mirrored copy rewrites legacy Claude `/skill-name` references. > - Task tracker mandate: BEFORE executing any workflow or skill step, create/update task tracking for all steps and keep it synchronized as progress changes. > - User-question prompts mean to ask the user directly in Codex. > - Ignore Claude-specific mode-switch instructions when they appear. > - Strict execution contract: when a user explicitly invokes a skill, execute that skill protocol as written. > - Subagent authorization: when a skill is user-invoked or AI-detected and its protocol requires subagents, that skill activation authorizes use of the required `spawn_agent` subagent(s) for that task. > - Do not skip, reorder, or merge protocol steps unless the user explicitly approves the deviation first. > - For workflow skills, execute each listed child-skill step explicitly and report step-by-step evidence. > - If a required step/tool cannot run in this environment, stop and ask the user before adapting. ## Codex Project-Reference Loading (No Hooks) Codex uses static project-reference loading instead of runtime-injected project docs. When coding, planning, debugging, testing, or reviewing, open project docs explicitly using this routing. **Always read:** - `docs/project-config.json` (project-specific paths, commands, modules, and workflow/test settings) - `docs/project-reference/docs-index-reference.md` (routes to the full `docs/project-reference/*` catalog) - `docs/project-reference/lessons.md` (always-on guardrails and anti-patterns) **Missing/stale context route:** If `docs/project-config.json`, the docs index, `lessons.md`, `CLAUDE.md`, `AGENTS.md`, or any task-required reference doc is missing or stale, auto-run `$project-init` or the narrow setup route (`$project-config`, `$docs-init`, `$scan-all`, `$scan --target=`, `$claude-md-init`) before ordinary project-specific work. If Codex mirrors or `AGENTS.md` are missing/stale, ask the user to run `$sync-codex`; do not auto-run it. **Situation-based docs:** - Project structure/architecture/tech-stack/deployment/setup (any layer — backend, frontend, or infra): `project-structure-reference.md` - Backend/CQRS/API/domain/entity changes: `backend-patterns-reference.md`, `domain-entities-reference.md` - Frontend/UI/styling/design-system: `frontend-patterns-reference.md`, `scss-styling-guide.md`, `design-system/README.md` - Spec authoring, `docs/specs/` pathing, or TC format: `feature-spec-reference.md`, `spec-system-reference.md`, `spec-principles.md` - Behavior/public-contract changes or spec-test-code sync: `workflow-spec-test-code-cycle-reference.md` plus the spec docs above - Derived spec indexes/ERDs/reimplementation guides: `spec-system-reference.md` and source Feature Specs under `docs/specs/` - Integration test implementation/review: `integration-test-reference.md` - E2E test implementation/review: `e2e-test-reference.md` - Code review/audit work: `code-review-rules.md` plus domain docs above based on changed files Do not read all docs blindly. Start from `docs-index-reference.md`, then open only relevant files for the task. > **[BLOCKING]** Execute skill steps in declared order. NEVER skip, reorder, or merge steps without explicit user approval. > **[BLOCKING]** Before each step or sub-skill call, update task tracking: set `in_progress` when step starts, set `completed` when step ends. > **[BLOCKING]** Every completed/skipped step MUST include brief evidence or explicit skip reason. > **[BLOCKING]** If Task tools are unavailable, create and maintain an equivalent step-by-step plan tracker with the same status transitions. ## Quick Summary **Goal:** Drive an integration-test suite to **fully green** by pairing `$integration-test-verify` with a combined fault-adjudication + `$fix` half in a recursive loop — each round runs a FRESH full `$integration-test-verify` over `{scope}` (**the WHOLE system by default**, or the target named in the prompt), and on ANY failure runs `$debug-investigate` **and** `$integration-test-review` (report-only) together to decide _who is at fault_, then `$fix` at the owning layer — stopping only when a fresh full verify passes its **2-consecutive-green-runs-without-DB-reset** gate with **zero failed tests**, proven by actual runner output. **Summary:** - **Each round = verify (find) + adjudicate (diagnose) + `$fix` (resolve) + `$changes-review` (prove the fix is sound).** `$integration-test-verify` reports pass/fail but must not own the fix; the adjudication pair decides _test-wrong vs test-not-optimal vs source-wrong_; `$fix` lands the change; `$changes-review` code-reviews that change before the next round. A round is incomplete until all four have run (or the verify returned zero failures). - **Steps (in order):** (0) resolve `{scope}` + Goal Contract → (0b) bind the convergence loop (protocol loop primary + optional `/goal` accelerator) → (1) round loop { snapshot → run `$integration-test-verify` INLINE over `{scope}` → **on failure:** `$debug-investigate` + `$integration-test-review` report-only → emit ONE Fault Verdict per failure → `$fix` at the owning layer → **CONDITIONAL `$changes-review` on the round's fix diff when any fix landed** → Round Integrity Check → log iteration } → (2) converge on a zero-failure fresh verify OR escalate → (3) terminal spec/doc sync + recap. - **Every fix gets code-reviewed in the round that lands it.** Whenever a round applies ANY fix — source, test, scenario, or spec — that round runs `$changes-review` (report-only, INLINE) over the round's fix diff; validated findings fold back into the SAME round's fix set. No fix applied → skipped with a recorded reason. — why: a green test cannot see a wrong-layer fix, a broken invariant elsewhere, or a security/performance regression. - **Default scope is the WHOLE SYSTEM.** With no target in the prompt, `{scope}` = **every** discovered integration-test project (`testProjectPattern` glob > `testProjects` list) — NOT the git-changed subset. The loop passes `{scope}` to `$integration-test-verify` **explicitly** so it never falls through to its change-scoped default (`integration-test-verify/SKILL.md:132,159`). - **Convergence:** stop ONLY when a **fresh full** `$integration-test-verify` over the CURRENT (post-fix) code reports **zero failed tests across 2 consecutive runs without a DB reset**, with real runner output — never a stale green predating the last fix. - **Inline invariant:** run `$integration-test-verify`, `$debug-investigate`, and `$integration-test-review` via the skill invocation, NEVER the `spawn_agent` tool. `$debug-investigate` requires its `$why-review` gate **in the same session/main agent** (`debug-investigate/SKILL.md:34`), and `$integration-test-review` self-binds its own fix + re-review obligations — a sub-agent cannot own or carry either back to this loop. Their OWN internal fan-outs (verify's `integration-tester` per-project sub-agents, `integration-test-review`'s phase agents) stay sub-agents by their own design, so context stays bounded. - **No fake green — Round Integrity Check:** a round converges only if the executed test count did **not shrink** and the skipped count did **not grow** versus the prior round. Deleting, skipping, or narrowing tests is a REGRESSION, never convergence. - **Bounded:** round cap default 3; failing count not shrinking across 2 rounds, or cap hit with failures still open → **STOP & escalate** by asking the user directly. Increasing failures → STOP (fixes regressing). Environment/infrastructure fault → **BLOCKED**, escalate immediately — never loop against an unhealthy system. **Why this skill exists (READ FIRST — it is the whole justification):** the obligation to loop already exists as **prose scattered across three skills**, with no mechanism behind it. `$integration-test-verify` says _"After fixing → re-run the full 2-run verify sequence"_ (`integration-test-verify/SKILL.md:238`) and its `SYNC:integration-test-execution-discipline` §5 says _"Loop until the whole suite is green"_ (`:404`) — but there is **no round cap, no Goal Contract, no shrinking-failures gate, and no escalation path**, so an agent that fixes one test and reports success is not violating anything mechanical. Worse, the fix half is **triple-owned and undefined**: `$integration-test-review` fixes tests and re-reviews itself (P5–P8), `$fix --target=test` runs its own unbounded _"if tests fail, repeat from step 2"_ (`fix/SKILL.md:218`), and verify's own failure protocol says fix-and-re-run (`:232-238`) — three overlapping loops that can double-fix the same failure or each assume another owns it. This skill makes the loop a **bounded, evidence-gated convergence contract with one owner**: verify FINDS, the `$debug-investigate` + `$integration-test-review` pair ADJUDICATES fault, `$fix` RESOLVES, and a fresh full verify RE-PROVES — with a Round Integrity Check so the suite can never go "green" by losing tests. Without it, "tests failed, then something fixed them" ships on a single green run over a hand-picked subset. **Workflow:** resolve `{scope}` (whole system by default) + Goal Contract → bind the convergence loop → **round loop** { run `$integration-test-verify` INLINE → on failure run `$debug-investigate` + `$integration-test-review` report-only → emit Fault Verdict → `$fix` at owning layer → conditional `$changes-review` on the round's fix diff → Round Integrity Check → log iteration } → converge when a fresh full verify is 2/2 green with zero failures → terminal `$spec [mode=sync]` + `$docs-update` + recap. **Key Rules:** - **Each round pairs verify (find) + adjudicate (diagnose) + `$fix` (resolve) + `$changes-review` (review the fix).** Never skip the adjudication half and jump straight to a fix — an unadjudicated failure gets "fixed" by whatever is nearest, which is almost always the assertion. - **CONDITIONAL `$changes-review` every round that lands a fix.** Working tree changed vs the round's snapshot → run `$changes-review` INLINE, report-only, scoped to the round's fix diff; validate its findings and fold them into the SAME round's fix set; unfixable validated findings → STOP & escalate. Working tree unchanged → skip with a recorded reason. NEVER let a round's fixes reach the next round un-reviewed. - **Default `{scope}` = the WHOLE system**; a target named in the prompt narrows it. Pass `{scope}` to `$integration-test-verify` EXPLICITLY every round — never let it fall back to git auto-detect. - **MUST run INLINE via the skill invocation — NEVER dispatch `$integration-test-verify`, `$debug-investigate`, `$integration-test-review`, or `$changes-review` as a sub-agent** (their in-session gates are lost). Their internal fan-outs remain sub-agents by their own design. - **`$integration-test-review` runs REPORT-ONLY.** It performs its 8-gate review and STOPS before its P5 fix / P6 re-review / P7 build-and-run — the loop owns fixing and re-running. If it cannot be constrained and self-fixes anyway, treat that as this round's fix half and skip the `$fix` step for that round — never double-fix. - **One written Fault Verdict per failure BEFORE any edit** — `TEST-WRONG` · `TEST-NOT-OPTIMAL` · `SOURCE-WRONG` · `ENVIRONMENT-BLOCKED` · `AMBIGUOUS` — with `file:line` evidence and confidence. `AMBIGUOUS` → ask the user directly, never a silent pick. - **NEVER force green.** No weakened or removed assertions, no skip annotations, no widened assertion timeouts, no retries around a failing assertion, no repository-hacked domain data, no narrowed scope. Fix the SCENARIO (an ARRANGE barrier on a real observable) or the product defect. - **Convergence = a fresh full verify over the post-fix code, 2/2 green, zero failures, real runner output, Round Integrity Check passed.** All five, or it is not converged. - **Round cap (default 3)**; failures not shrinking across 2 rounds, increasing, or cap hit with failures open → **STOP & escalate** by asking the user directly. `ENVIRONMENT-BLOCKED` → escalate immediately; never loop against an unhealthy system. --- ## First Principle — Convergence, Not Motion > A round that changes the code is progress **only if** the next fresh verify has fewer failing tests. > The loop exists to reach a fixed point (a whole suite that is green twice in a row), not to keep editing until something passes. > If the failing count stops shrinking, that is a signal to **escalate**, not to spin another round. > And a suite that got greener by having fewer tests did not converge — it regressed. --- ## Step 0 — Resolve Verification Scope + Goal Contract (FIRST ACTION) 1. **Read `docs/project-config.json` → `integrationTestVerify`** before anything else (the same config `$integration-test-verify` obeys, `integration-test-verify/SKILL.md:74-92`). You need `quickRunCommand`, `testProjectPattern`, `testProjects`, `systemCheckCommand`, `startupScript`, and `referenceDocs` to resolve scope and to read the project's integration-test reference docs. 2. **Resolve `{scope}` — WHOLE SYSTEM by default:** | Prompt | `{scope}` | | ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | | **No target named** (default) | **Every** integration-test project discoverable via `testProjectPattern` glob > `testProjects` list. The WHOLE system. | | Names a suite/project/module/feature | Only the test projects covering that target, resolved from the same config; state which projects the target maps to and how you resolved it. | | Names a diff/branch/PR | The test projects covering that change set — this is the ONLY case where change-scoping is correct, and it must be explicit in the prompt. | > **NEVER let `$integration-test-verify` resolve scope by itself.** Its Step 3 priority ends in git auto-detect and _"Only run projects relevant to the current change"_ (`integration-test-verify/SKILL.md:132,159`) — correct when it is a workflow step after an edit, WRONG as this loop's default. Pass `{scope}` explicitly in the invocation every round. — why: a loop that silently verifies only the changed subset reports "all green" for a system it never ran. 3. **Record `{scope}` as a stable string** (the resolved project list). The scope is FIXED for the whole loop — it never narrows as rounds progress. If a fix legitimately ADDS a test project, widen `{scope}` and say so in the Iteration Log; narrowing it is forbidden. 4. **Resolve/create the Goal Contract** per `SYNC:goal-contract-satisfaction-loop` (`plans/goals/{YYMMDD-HHmm}-{slug}/goal.md`, template `.claude/templates/goal-contract-template.md`). Its single **required** Success Criterion: > _A fresh full `$integration-test-verify` over `{scope}` reports **zero failed tests** across **2 consecutive runs without a DB reset**, evidenced by actual test-runner output (Passed/Failed/Skipped counts), with no test deleted, skipped, or weakened to get there._ Record in **Constraints**: `{scope}` (the project list), the round cap (default 3), the baseline executed/skipped test counts once round 1 reports them, and `quickRunCommand`. ## Step 0b — Bind the Convergence Loop (protocol-first; `/goal` is an optional accelerator) The convergence loop is bound by TWO layers. The **protocol loop (Steps 1–2) is the BINDING mechanism** and MUST be self-driven by you, the running agent, on every host — with or without any command or hook. The **`/goal` command is an OPTIONAL accelerator** layered on top; it is never the primary mechanism, and its absence NEVER weakens the loop. This mirrors the project rule that hooks/trackers are accelerators only — correctness must not depend on them. **1. Protocol loop — ALWAYS binding (hook/command-independent).** You are personally responsible for not stopping until the loop converges or bounded-escalates. This binds Claude, Codex, and Copilot equally, whether or not `/goal` exists: > Repeatedly run `$integration-test-verify` INLINE over `{scope}` (passed explicitly, never re-derived). After each run, if ANY test failed, adjudicate every failure with `$debug-investigate` + `$integration-test-review` (report-only) into ONE Fault Verdict, apply the fix via `$fix` at the owning layer, then re-run a FRESH full `$integration-test-verify` over `{scope}`. Do NOT stop while the last verify still reported a failing test. Converge ONLY when a fresh full verify reports zero failures across 2 consecutive runs without a DB reset AND the Round Integrity Check passes (executed test count not shrunk, skipped count not grown). Cap at `{N=5}` rounds; if the failing count does not shrink across 2 consecutive rounds, failures increase, the cap is hit with failures still open, or any failure is ENVIRONMENT-BLOCKED → STOP and escalate by asking the user directly. Never loop open-ended, and NEVER reach green by weakening, skipping, deleting, or de-scoping a test. Treat this as a standing obligation you re-read at every Step 2 checkpoint — NOT a one-time note you can rationalize away after the first fix cycle. The Goal Contract's required Success Criterion (Step 0) is its durable, host-independent record. **2. `/goal` command — invoke as an accelerator WHEN AVAILABLE.** If a `/goal` command exists and you are permitted to run it in this environment, ALSO invoke it (a real tool/command call, NOT a paraphrase, NOT a Goal Contract file substituted for it) with the SAME condition, so a session Stop hook mechanically enforces the loop: ``` /goal integration-test green convergence loop: repeatedly run $integration-test-verify INLINE over {scope} (passed explicitly). If any test failed → adjudicate each failure with $debug-investigate + $integration-test-review (report-only) into one Fault Verdict, fix via $fix at the owning layer, and run another round. If a fresh full verify reports zero failures across 2 consecutive runs without DB reset AND executed test count has not shrunk and skipped count has not grown → CONVERGED, run the terminal $spec [mode=sync] + $docs-update and clear the gate. Do NOT stop while the last verify still reported a failing test. Cap at {N=5} rounds; if the failing count does not shrink across 2 consecutive rounds, failures increase, the cap is hit with failures open, or a failure is ENVIRONMENT-BLOCKED → STOP and escalate by asking the user directly. Never loop open-ended; never reach green by weakening, skipping, deleting, or de-scoping a test. ``` The `/goal` Stop hook blocks stopping until the condition holds and auto-clears when met — do not tell the user to clear it. **If `/goal` is unavailable, unregistered, or not permitted** (e.g. Codex/Copilot, or a Claude run without the command): DO NOT error, DO NOT block, and DO NOT invent a stand-in gate. Record ONE line in the Goal Contract — `/goal accelerator unavailable — loop bound by protocol (Steps