---
name: test-coverage
description: Use after writing tests to assess coverage quality across structural, mutation, requirements, and API/integration dimensions; organized knowledge for choosing and interpreting coverage analyses.
---

# Test Coverage Analysis

A reference manual for choosing, applying, and interpreting test-coverage analyses on an existing test suite.

This skill is a **knowledge reference**, not a procedure. It does not tell you when to write tests or which test types to design — that is the job of `design-testing-strategy`. It tells you, once tests exist, **which mechanical signal best measures what those tests do (and do not) exercise**, and how to read that signal honestly.

## What Coverage Analysis Is

Test coverage analysis is the **post-hoc measurement** of how thoroughly a test suite exercises a software artifact along one or more **axes**. It answers the question *"what did my tests actually touch?"* — for some specific definition of "touch."

The word "coverage" is overloaded. It can mean any of:

- **Structural / code coverage** — which lines, statements, branches, conditions, or paths in the source code were executed (measured by instrumentation).
- **Mutation coverage** — what proportion of deliberately-injected source faults the test suite detects (measured by re-running the suite against mutated code).
- **Requirements / feature coverage** — which acceptance criteria, user stories, or specification clauses have at least one verifying test (measured by traceability).
- **API / integration coverage** — which endpoints, methods, status codes, contract interactions, and schema fields are exercised (measured by request/response inspection).
- **Specification-domain coverage** — equivalence classes, boundary values, parameter combinations, state transitions, error paths (measured by analyzing test inputs against a model).

### Category correction: coverage is not a test type

Mutation testing, MC/DC, branch coverage, RTM linkage, contract coverage, and schema coverage are **measurements about** an existing test suite. They are **not** test types in the way unit, integration, e2e, contract, or smoke tests are.

- "Should I write a unit test or a mutation test?" is a malformed question. The correct framing is: *"I already have unit/integration tests; should I additionally run mutation analysis against them?"*
- Mutation tools generate variants of the source and re-execute the **existing** suite. They produce a score, not new tests.
- MC/DC and branch coverage are reports computed from instrumented runs of the **existing** suite.
- RTM linkage is a property of test metadata (tags, IDs), not a separate execution.

If a "test strategy" places mutation testing alongside unit / integration / e2e, that strategy has confused *what to test* with *how to measure the tests*. The two questions are orthogonal.

### The asymmetry principle

**Low coverage is strong evidence of weak testing. High coverage is weak evidence of strong testing.**

Coverage is *necessary-but-not-sufficient*. A test can execute a line without asserting anything meaningful; 100% line coverage is routinely achievable with zero assertions ([thinkinglabs.io](https://thinkinglabs.io/articles/2022/03/19/the-fallacy-of-the-100-code-coverage.html), [codeintelligently.com](https://codeintelligently.com/blog/ai-generated-tests-false-confidence)). Use coverage as a **tripwire**, not a **trophy**. Once a coverage percentage becomes a target, it ceases to be a good metric (Goodhart's law applied to testing; see [Optivem Journal](https://journal.optivem.com/p/code-coverage-targets-recipe-for-disaster)).

### What coverage analysis is NOT

- **NOT a measure of test quality.** Lines can execute without assertions.
- **NOT a measure of correctness.** Coverage proves the test ran, not that it would have failed on a bug.
- **NOT a synonym for "well tested".** Mutation testing routinely refutes 100%-coverage-with-no-assertions suites.
- **NOT a substitute for risk-based test selection** per [ISO/IEC/IEEE 29119](https://en.wikipedia.org/wiki/ISO/IEC_29119).
- **NOT a target.** Treat as a floor and a trend, never as the goal itself.

---

## Per-Type Structure

Every coverage type in this skill is documented in the same six sub-fields, in this order:

1. **Definition** — what it measures.
2. **What it does NOT measure** — its limits / blind spots.
3. **Typical tools** — per ecosystem.
4. **When to use vs skip** — applicability heuristics.
5. **Targets / thresholds & pitfalls** — defensible numeric ranges (always with the risk caveat) and common gaming patterns.
6. **Cost-benefit ROI** — order-of-magnitude cost vs the signal you actually buy.

Scan any section by these headings.

---

## Structural / Code Coverage

Measured by instrumenting the compiled or interpreted program and recording which structural elements (lines, statements, branches, conditions, paths) the test suite executes.

### Line / Statement Coverage

- **Definition.** Percentage of source-code lines (or statements) executed at least once.
- **What it does NOT measure.** Whether branches were taken in both directions. Whether assertions verified the result. Whether boundary values were tested. Multiple statements on one line distort the metric ([Metridev](https://www.metridev.com/en/metrics/statement-vs-branch-coverage-understanding-the-difference/)).
- **Typical tools.**

  | Ecosystem | Tool |
  |-----------|------|
  | JS/TS | [Istanbul / nyc](https://istanbul.js.org/) (built into Jest, Vitest, Karma); `--coverage` flag |
  | Python | [Coverage.py](https://coverage.readthedocs.io/) + `pytest-cov`; supports branch mode |
  | JVM | [JaCoCo](https://www.eclemma.org/jacoco/) — bytecode instrumentation, industry standard |
  | C/C++ | `gcov` / `lcov` / `gcovr`, [llvm-cov](https://llvm.org/docs/CommandGuide/llvm-cov.html) |
  | Go | `go test -cover`, `go tool cover` ([build-cover](https://go.dev/doc/build-cover) added integration-test mode in Go 1.20) |
  | .NET | [Coverlet](https://github.com/coverlet-coverage/coverlet) (open-source default), [JetBrains dotCover](https://www.jetbrains.com/dotcover/), AltCover. **OpenCover is in maintenance mode — prefer Coverlet / dotCover / AltCover** ([NDepend guide](https://blog.ndepend.com/guide-code-coverage-tools/)) |
  | Ruby | [SimpleCov](https://github.com/simplecov-ruby/simplecov) |
  | Swift / Obj-C | Xcode built-in (llvm-cov backend) |
  | Rust | `cargo-llvm-cov`, `cargo-tarpaulin` |
  | Report formats | Cobertura XML, LCOV, Clover; aggregators: Codecov, Coveralls, SonarQube |

- **When to use vs skip.** Always-on; cost is near-zero (a CI flag). Never use as a quality goal in itself.
- **Targets / thresholds & pitfalls.** 70–85% is typical for general-purpose code ([Qt blog](https://www.qt.io/quality-assurance/blog/is-70-80-90-or-100-code-coverage-good-enough)). Apply only with the risk caveat (see Risk-Based Interpretation below). Tests with no assertions still count lines as covered. Single-line `if (x) doA(); else doB();` shows 100% statement coverage with only one branch exercised. Snapshot-only tests inflate numbers without verifying behavior.
- **Cost-benefit ROI.** Very high — cost near-zero, value is a tripwire on regression in test reach.

### Branch / Decision Coverage

- **Definition.** Percentage of decision branches (true/false outcomes of `if`, `while`, `for`, `?:`, `switch` cases) executed.
- **What it does NOT measure.** Compound-condition independence (`A && B` taken `true` might never test `A=true, B=false`). Order of evaluation. Loop iteration counts. Assertion strength.
- **Typical tools.** Same as line coverage; enable with `--branch` (coverage.py), branch mode (Istanbul is branch-aware by default), JaCoCo reports branches natively. Strictly stronger than line/statement coverage ([Graph AI](https://www.graphapp.ai/blog/statement-coverage-vs-branch-coverage-a-comprehensive-comparison)).
- **When to use vs skip.** Default for any non-trivial logic. Prefer branch over line as the primary structural metric.
- **Targets / thresholds & pitfalls.** 70–80% branch is a "respectable" target for business apps ([Lead With Skills](https://www.leadwithskills.com/blogs/test-coverage-metrics-lines-branches-conditions-paths)) — always paired with the risk caveat. Compound conditions hide gaps: `if (A || B)` achieves 100% branch coverage with only one true-evaluating sub-condition and one false branch overall.
- **Cost-benefit ROI.** High — best single structural metric for general-purpose code ([LinearB](https://linearb.io/blog/what-is-branch-coverage)).

### Condition Coverage

- **Definition.** Every Boolean **sub-condition** in every decision has taken both `true` and `false` at least once.
- **What it does NOT measure.** Whether each sub-condition independently affects the outcome (that is MC/DC). Does not require all combinations.
- **Typical tools.** Same toolchains as branch coverage; many report condition coverage as a separate column.

  | Ecosystem | Tool |
  |-----------|------|
  | JVM | JaCoCo (condition counters in branch reports) |
  | C / C++ | gcov/gcovr (`--branch-counts`), Qt Coco |
  | .NET | Coverlet (condition coverage via Cobertura output) |

- **When to use vs skip.** Informative for code with compound expressions; rarely useful as a CI gate on its own.
- **Targets / thresholds & pitfalls.** Achievable without exercising every combination. `if (A && B)` hits 100% condition coverage with `{A=T,B=F}` and `{A=F,B=T}` — neither makes the decision `true`. Treat as a diagnostic, not a gate.
- **ROI:** Medium — useful diagnostically when investigating *why* branch coverage looks high but bugs persist; not a gate.

### MC/DC — Modified Condition/Decision Coverage

- **Definition** (per [Wikipedia](https://en.wikipedia.org/wiki/Modified_condition/decision_coverage)):
  1. Every entry/exit point invoked at least once.
  2. Every decision has taken every outcome at least once.
  3. Every condition in a decision has taken every outcome at least once.
  4. Each condition has been shown to **independently** affect the decision's outcome (holding the other conditions fixed).

  For `n` conditions, MC/DC is achievable with `n+1` to `2n` tests via independence pairs — vastly cheaper than the `2^n` of exhaustive multiple-condition coverage ([LDRA](https://ldra.com/capabilities/mc-dc/)).

- **What it does NOT measure.** Loop iteration counts, data values, integration paths, assertion strength.
- **Typical tools.** LDRA TBvision, Rapita RapiCover, VectorCAST, Razorcat TESSY, [Qt Coco](https://www.qt.io/quality-assurance/coco/feature-modified-condition-decision-coverage-mcdc), Parasoft C/C++test. Mostly commercial — open-source MC/DC is rare.
- **When to use vs skip.** When mandated by a standard (DO-178C DAL A, ISO 26262 ASIL D, IEC 62304 Class C high-risk modules, EN 50128 SIL 4, IEC 61508 SIL 4). Outside regulated domains, branch coverage + mutation testing covers the same intent at lower cost.
- **Targets / thresholds & pitfalls.** 100% by definition in regulated domains. Short-circuit evaluation in C-like languages can make some independence pairs unreachable; compiler optimizations can collapse conditions, so coverage builds must disable optimization — meaning the coverage-build binary is not the release-build binary, an acknowledged regulatory risk ([Verifysoft](https://www.verifysoft.com/en_ISO_26262_Road_Vehicles_Functional_Safety.html)).
- **Cost-benefit ROI.** Very high cost (specialist toolchain + labor + documentation overhead); high value only where required by law/standard.

### Function / Method Coverage

- **Definition.** Percentage of declared functions/methods invoked at least once.
- **What it does NOT measure.** Anything about the bodies of those functions.
- **Typical tools.** Reported by most structural-coverage tools as a side column.

  | Ecosystem | Tool |
  |-----------|------|
  | JVM | JaCoCo (method counter) |
  | .NET | Coverlet (methods column) |
  | Python | coverage.py (`report -m` granularity), pytest-cov |
  | JS/TS | Istanbul (functions metric in lcov / json-summary) |

- **When to use vs skip.** As a quick "did I forget a module?" check; never as a primary metric.
- **Targets / thresholds & pitfalls.** Often deceptively high — many functions are entered by happy-path tests with no error-path coverage inside.
- **ROI:** Low — informational only; useful as a "module forgotten?" tripwire, not a gate.

### Path Coverage

- **Definition.** Percentage of unique linearly-independent paths through a function. Bounded by cyclomatic complexity `V(G) = decisions + 1` ([Cyclomatic complexity](https://en.wikipedia.org/wiki/Cyclomatic_complexity)).
- **What it does NOT measure.** Anything practical for non-trivial functions — `N` decisions yields `2^N` paths, unbounded for loops.
- **Typical tools.** Some commercial safety-critical tools report basis-path counts; rarely a CI artifact.

  | Ecosystem | Tool |
  |-----------|------|
  | Safety-critical C/C++ | LDRA TBvision, VectorCAST (basis-path metrics) |
  | Any / complexity proxy | lizard, radon, SonarQube (cyclomatic complexity as a *bound*, not a path metric) |

- **When to use vs skip.** Rarely as a coverage target. Cyclomatic complexity is more useful as a **complexity signal** that *bounds the minimum* number of tests needed to exercise distinct flows.
- **Targets / thresholds & pitfalls.** Combinatorial explosion. Most production code is uncovered at path-coverage level and that is acceptable.
- **ROI:** Low for production code; meaningful only inside very small, very high-criticality functions — outside that, use complexity as a *signal* and stop.

---

## Mutation Testing as Coverage Analysis

> **Reminder:** Mutation testing is a coverage analysis of an existing test suite. It is **not** a test type. It produces a score and a list of survived mutants; it does not produce new tests. You apply it *to* your unit / integration suite, not *instead of* it.

### Definition

Mutation testing introduces small, syntactic modifications ("mutants") to the source and re-runs the existing test suite against each mutant. If at least one test **fails** for a given mutant, the mutant is **killed** (the suite detected the fault). If all tests **pass**, the mutant **survived** (the suite is blind to that change). It measures *test-suite fault-detection power*, not source-code reach ([Stryker docs](https://stryker-mutator.io/docs/)).

Typical mutation operators:

- **Arithmetic** — `+` → `-`, `*` → `/`, `++` → `--`.
- **Conditional / relational** — `<` → `<=`, `==` → `!=`, `&&` → `||`.
- **Boolean / negation** — `true` → `false`, remove `!`.
- **Statement removal / block deletion.**
- **Return value** — `return x` → `return null` / `return ""`.
- **Increment / decrement of literal constants.**
- **Conditional boundary** — `>` → `>=`.

### Mutant states ([Stryker docs](https://stryker-mutator.io/docs/mutation-testing-elements/mutant-states-and-metrics/))

| State | Meaning |
|-------|---------|
| **Killed** | At least one test failed on the mutant. Suite detected the fault. |
| **Survived** | All tests passed on the mutant. Suite is blind. |
| **No coverage** | No test executed the mutated code (orthogonal gap — code itself is untested). |
| **Timeout** | Tests hung; usually counted as a kill (the suite *did* observe abnormal behavior). |
| **Compile error / runtime error** | Mutant is syntactically/semantically invalid; usually filtered. |
| **Ignored** | Filtered by config (generated code, glue, etc.). |

Score: `mutation_score = killed_mutants / (total_mutants - equivalent_mutants - errors)`. Some tools also report a "killed%" relative to *covered* mutants only.

### What it does NOT measure

- **Dead-code regions** — appear as `no coverage`, identical to "line not covered."
- **Semantic correctness** of assertions — a wrong-but-strict assertion still kills mutants.
- **Boundary data values** — operator mutants approximate this but do not replace BVA.
- **Equivalent mutants** — variants that produce identical observable behavior. Detection is undecidable in general; manual review is the only certain method. Modern tools (Stryker TypeScript Checker, PIT with Major) reduce these heuristically. **Do not chase 100% mutation score** — equivalents make it asymptotically unattainable ([Stryker docs](https://stryker-mutator.io/docs/mutation-testing-elements/equivalent-mutants/)).

### Typical tools

| Ecosystem | Tool |
|-----------|------|
| JS / TS | [Stryker (StrykerJS)](https://stryker-mutator.io/) — TypeScript checker plugin filters compile-error mutants |
| .NET (C#) | [Stryker .NET](https://stryker-mutator.io/docs/stryker-net/introduction/); documented in [Microsoft Learn](https://learn.microsoft.com/en-us/dotnet/core/testing/mutation-testing) |
| Java / JVM | [PIT (Pitest)](https://pitest.org/) — reference standard for JVM; Major Mutator for research |
| Python | [mutmut](https://mutmut.readthedocs.io/), [Cosmic Ray](https://cosmic-ray.readthedocs.io/), [MutPy](https://github.com/mutpy/mutpy) |
| Go | [go-mutesting](https://github.com/avito-tech/go-mutesting), [ooze](https://github.com/gtramontina/ooze) |
| PHP | [Infection](https://infection.github.io/) |
| Ruby | [mutant](https://github.com/mbj/mutant) |
| Rust | [cargo-mutants](https://mutants.rs/) |
| C / C++ | [Mull](https://github.com/mull-project/mull) — LLVM-based |
| Scala | [Stryker4s](https://stryker-mutator.io/docs/stryker4s/introduction/) |

### When to use vs skip

**Apply when:**

- Suite is already structurally mature (typically >80% branch coverage). On a sparse suite, mutation results are dominated by `no coverage` and you learn nothing new beyond what structural coverage already shows.
- Artifact is **pure-logic core** — financial calculations, security-critical validation, parsers, encryption, authorization decisions.
- Criticality is high enough that suite blind spots represent material risk.

**Skip when:**

- Glue code, controllers, framework wiring — operators generate noise on declarative constructs.
- UI rendering — equivalents dominate.
- Configuration, DTOs, declarative serialization.
- Brand-new suite still being built up.
- Tight CI feedback loop where N×suite runtime is prohibitive (mitigate with incremental analysis, not by giving up coverage).

### Targets / thresholds & pitfalls

Stryker defaults ([config](https://stryker-mutator.io/docs/stryker-js/configuration/)): `high: 80`, `low: 60`, `break: null`. Set `break` to fail the build below a floor. Apply with the risk caveat: 60–80% on a mature unit suite over pure-logic core is a reasonable starting point; never on glue code. Common pitfalls: chasing equivalents (asymptote), running on UI/config (noise), running on shallow suites (re-reports what coverage already shows).

### Cost-benefit ROI

- **Cost.** CPU-quadratic-ish. A 60-second suite generating 1,000 mutants is up to 1,000 × 60s without optimization. Modern tools mitigate via incremental analysis, per-mutant test selection, and parallel runners.
- **Benefit.** Catches *missing assertions* and *over-mocked* tests that structural coverage cannot detect. It is the **only practical coverage technique that scores assertion strength** — the chief failure mode of "100% coverage with no assertions" ([codeintelligently.com](https://codeintelligently.com/blog/ai-generated-tests-false-confidence)).
- **CI pattern.** Incremental mutation on PR diff; nightly full run on critical modules ([oneuptime](https://oneuptime.com/blog/post/2026-01-24-mutation-testing/view); see also research roundup at [greg4cr.github.io](https://greg4cr.github.io/pdf/23mutationci.pdf)).

### Relationship to structural coverage

Mutation testing **subsumes and supplements** structural coverage:

- A mutant in unreachable code is `no coverage` — identical signal to "line not covered."
- A mutant in covered code that survives — "covered but not meaningfully verified" — is invisible to structural coverage.

--