---
name: foundations-reliability-theory
description: Reliability-theory primitives for MTBF/MTTR, availability, hazards, FMEA, redundancy, error budgets, Weibull analysis, and SLOs. Use when modeling failure.
compatibility: Portable core only.
version: "1.2"
last_validated: 2026-08-14
---

# Reliability Theory Foundations


11 reliability theory primitives covering the mathematics of failure, availability, and repair. Each primitive is domain-agnostic: the same MTBF/MTTR arithmetic that governs hardware maintenance governs SLO budget calculation; the same fault tree that maps a safety system maps a payment pipeline's SPOF paths.

## Contents

- [Quick Reference](#quick-reference)
- [Primitive Index](#primitive-index)
- [Formal Supporting Theory](#formal-supporting-theory)
- [Misuse Boundaries](#misuse-boundaries)
- [Anti-Patterns](#anti-patterns)
- [Decision Checklist](#decision-checklist)
- [Composition Recipes](#composition-recipes)
- [Expert Judgment: When the Math Lies](#expert-judgment-when-the-math-lies)
- [Workflow](#workflow)
- [ASCII Flow](#ascii-flow)
- [Navigation](#navigation)
- [Related Skills](#related-skills)
- [Fact-Checking](#fact-checking)

---

## Quick Reference

| # | Primitive | When to Reach for It |
|---|-----------|---------------------|
| 1 | [MTBF / MTTR](assets/templates/reliability-theory/01-mtbf-mttr.md) | Measuring how often a system fails and how long it takes to recover |
| 2 | [Availability Formulas](assets/templates/reliability-theory/02-availability-formulas.md) | Converting MTBF/MTTR into a percentage; composing series and parallel components |
| 3 | [Hazard Functions](assets/templates/reliability-theory/03-hazard-functions.md) | Classifying current failure rate shape (constant / increasing / decreasing) |
| 4 | [Bathtub Curve](assets/templates/reliability-theory/04-bathtub-curve.md) | Identifying the lifecycle phase (infant mortality / useful life / wear-out) |
| 5 | [Fault Tree Analysis](assets/templates/reliability-theory/05-fault-tree-analysis.md) | Tracing a top event backwards to root causes; finding single points of failure |
| 6 | [FMEA](assets/templates/reliability-theory/06-fmea.md) | Enumerating failure modes bottom-up and ranking by RPN before launch |
| 7 | [Redundancy Math](assets/templates/reliability-theory/07-redundancy-math.md) | Sizing active/standby/k-of-n redundancy; checking coverage sensitivity |
| 8 | [Error Budgets](assets/templates/reliability-theory/08-error-budgets.md) | Deriving SRE error budget from SLO; computing multi-window burn rate |
| 9 | [Weibull Analysis](assets/templates/reliability-theory/09-weibull-analysis.md) | Fitting lifetime data to a distribution; estimating B10 life and phase |
| 10 | [System Reliability](assets/templates/reliability-theory/10-system-reliability.md) | Combining component reliabilities through series/parallel/mixed topologies |
| 11 | [Reliability Allocation](assets/templates/reliability-theory/11-reliability-allocation.md) | Apportioning a system reliability target to subsystems |

---

## When to Apply

**Apply reliability-theory when:**
- SLO design or error-budget math (availability targets, allowed downtime, burn rates)
- Redundancy decisions — single instance vs active-active vs N+1 vs geographic
- FMEA / fault-tree analysis on a system before launch or post-incident
- Hazard-rate questions — "is this an infant-mortality bug, random failure, or wear-out?"
- Composition math — when independent components form a chain or parallel system

**Skip and use simpler alternatives when:**
- Question is about latency/throughput, not availability — use foundations-queueing-theory
- Question is about consistency under partition — use foundations-distributed-systems
- Question is about feedback/anti-windup tuning — use foundations-control-theory
- System has no SLO and no business impact from downtime — over-engineering risk
- One-shot script, dev tooling, or non-production code — reliability math is overhead
- Failure modes are correlated (shared DB, single AZ) — independence assumption breaks composition math; flag the correlation first
- Software-control-loop or autonomous system where unsafe interactions (not just component failures) dominate — augment FTA with STPA (Leveson 2011) rather than extending the fault tree
- Stochastic-per-run AI agent where run-to-run variance is the primary concern — adapt primitives as described in [Domain Applicability Notes](#domain-applicability-notes) rather than using raw MTBF

---

## Primitive Index

Each primitive has a full playbook (definition, when to use, inputs, outputs, failure modes, worked example, sources).

| # | Primitive | Failure Mode It Addresses |
|---|-----------|--------------------------|
| 1 | [MTBF / MTTR](assets/templates/reliability-theory/01-mtbf-mttr.md) | No quantified failure rate or repair time; availability is guesswork |
| 2 | [Availability Formulas](assets/templates/reliability-theory/02-availability-formulas.md) | Availability computed incorrectly (additive instead of multiplicative in series) |
| 3 | [Hazard Functions](assets/templates/reliability-theory/03-hazard-functions.md) | Wrong distribution assumed (CFR when system is IFR or DFR) |
| 4 | [Bathtub Curve](assets/templates/reliability-theory/04-bathtub-curve.md) | Lifecycle phase invisible; burn-in skipped; wear-out surprises operations |
| 5 | [Fault Tree Analysis](assets/templates/reliability-theory/05-fault-tree-analysis.md) | Single points of failure and correlated causes not identified before launch |
| 6 | [FMEA](assets/templates/reliability-theory/06-fmea.md) | Failure modes not enumerated; highest-risk paths not mitigated before launch |
| 7 | [Redundancy Math](assets/templates/reliability-theory/07-redundancy-math.md) | Redundancy added without verifying it actually improves reliability |
| 8 | [Error Budgets](assets/templates/reliability-theory/08-error-budgets.md) | No principled mechanism to balance deployment velocity against stability |
| 9 | [Weibull Analysis](assets/templates/reliability-theory/09-weibull-analysis.md) | MTBF computed without fitting the actual distribution; maintenance timed wrong |
| 10 | [System Reliability](assets/templates/reliability-theory/10-system-reliability.md) | System availability computed from topology without common-cause correction |
| 11 | [Reliability Allocation](assets/templates/reliability-theory/11-reliability-allocation.md) | System target not distributed; teams build to arbitrary individual specs |

---

## Formal Supporting Theory

Load [`references/formal-theory-map.md`](references/formal-theory-map.md) when the work depends on reliability mathematics: survival and hazard functions, repairable vs. non-repairable systems, series/parallel composition, common-cause failure, FTA Boolean gates, FMEA scoring limits, Weibull shape interpretation, availability/SLO arithmetic, or allocation constraints.

### Key Identities Across Primitives

| Identity | Formula | Pitfall |
|----------|---------|---------|
| Availability from MTBF/MTTR | A = MTBF / (MTBF + MTTR) | Use 90th-percentile MTTR from incident records, not runbook estimate |
| Series availability | A_s = ∏ Aᵢ | Never average or sum; must multiply |
| Active-active (2 identical, independent) | A = 1 − (1−A₁)² | Fails when components share a dependency (common-cause) |
| k-of-n reliability | R = Σ C(n,j) Rʲ (1−R)ⁿ⁻ʲ, j=k..n | Assumes independence; check coverage factor c |
| Error budget (monthly) | budget_minutes = (1 − SLO) × 43,800 min | SLO definition drift invalidates budget comparisons |
| Weibull MTTF | MTTF = η · Γ(1 + 1/β) | Requires ≥10 complete failures for defensible β |
| AI agent chain reliability | R = ∏ rᵢ (per-step) | Per-step rᵢ must be measured across ≥10 runs per step |

These identities are expanded with derivations in [`references/formal-theory-map.md`](references/formal-theory-map.md).

## Misuse Boundaries

Load [`references/patterns-scenarios-traps.md`](references/patterns-scenarios-traps.md) before claiming availability, using MTBF as a promise, adding redundancy, accepting an FMEA RPN ranking, fitting Weibull with sparse data, or converting SLOs into release policy. It contains scenarios, anti-patterns, and calculation traps.

---

## Anti-Patterns

| Anti-Pattern | Reliability Diagnosis | Fix |
|-------------|----------------------|-----|
| Arithmetic average of subsystem MTBFs used as system MTBF | Series availability is multiplicative; averaging overstates reliability | Compute A_system = ∏ Aᵢ using primitive 02; never sum or average MTBFs across parallel systems |
| MTTR estimated from happy-path runbook execution time | Tail incidents run longer than rehearsed recovery; actual MTTR is higher | Sample MTTR from real incident records; use 90th percentile not mean; include detection-to-restore, not just restore duration |
| Error budget burn measured weekly when traffic is bursty | A 4-hour burst failure exhausts the hourly budget invisibly inside a weekly window | Implement multi-window burn rate (1-hour and 6-hour) alongside the 30-day window (primitive 08) |
| Weibull fit applied to fewer than 6 complete failure observations | β and η confidence intervals span orders of magnitude; shape classification is noise | Report confidence intervals explicitly; do not act on β classification until ≥10 failures are observed |
| RPN score used as the sole prioritisation signal in FMEA | A Severity=10, Occurrence=1, Detection=10 item scores RPN=100 — low — but is catastrophic if it occurs | Always review all S≥9 items independently of RPN; never allow a low RPN to deprioritise a catastrophic failure mode |
| Redundancy added without modelling switchover reliability | Active/standby failover mechanism fails; redundancy provides no benefit or reduces reliability | Model coverage probability c in the imperfect-coverage formula; measure switchover reliability before sizing more units (primitive 07) |
| Correlated failures in parallel components treated as independent | Common power rail, same AZ, or shared codebase invalidates the independence assumption; parallel formula drastically overstates reliability | Apply beta-factor common-cause correction in primitive 10; audit shared dependencies before claiming availability improvement |
| Phase II (CFR) assumed without testing | Early or late phases have non-constant hazard rates; exponential MTBF formula produces wrong predictions | Plot empirical h(t) from observation data (primitive 03) before choosing a distribution |
| pass@1 used as the sole agent reliability metric | Single-run success conceals consistency variance: perturbations reduced success from 96.9% to 88.1% in one benchmark; capability and reliability rankings diverge at long horizons | Use pass^k across ≥10 runs as the consistency floor; pair with Markov chain step-reliability for sequential tool chains (primitive 10 extension). See [references/ai-agent-reliability.md](references/ai-agent-reliability.md) |
| MTBF applied directly to stochastic-per-run AI agents | Classic MTBF assumes a stationary failure rate; LLM agents degrade non-linearly with task duration (Reliability Decay Curve); single-run availability is not meaningful | Adapt primitive 01 by measuring pass^k and RDC across task-duration buckets; flag duration-dependent degradation explicitly |
| Multi-agent topology treated as a plain series chain | Series math (R = ∏ rᵢ) assumes a step's failure is contained. In uncoordinated multi-agent systems errors propagate *and amplify* across handoffs; MAST attributes ~41.8% of failures to specification/system-design and ~36.9% to inter-agent misalignment — neither is a per-step reliability drop | Use ∏ rᵢ as a best case, not an estimate. Enumerate handoff failure modes (context loss, format mismatch, missing termination) in agent FMEA (primitive 06); a centralised validation bottleneck contains amplification far better than peer-to-peer topology. See [references/ai-agent-reliability.md](references/ai-agent-reliability.md) |
| Agent monitoring assumed to detect agent failure | LLM systems fail "plausible": the model narrates a failed step into fluent prose, so the error never surfaces as an error. ~70% of silent failures in one production runtime were found by human observation despite 4,286 unit tests and 827 governance checks | Detection scores (the D in RPN) must be measured against *silent* failure, not crash failure. Add per-step output validation gates and end-to-end assertions on ground truth, not on the agent's own report of success |
| Safety-I only: treating reliability as absence of failures | FMEA/FTA enumerate deviations from a nominal; they cannot surface emergent failures that arise from normal work coupling in sociotechnical systems | Complement FTA/FMEA with Safety-II perspective (Hollnagel 2014): understand adaptive capacity, not only failure modes. See Conceptual Complements below |

---

## Decision Checklist

- [ ] **No failure rate data yet**: start by computing MTBF and MTTR from incident records → primitive 01.
- [ ] **Need an availability percentage**: translate MTBF/MTTR → primitive 02.
- [ ] **System has multiple components in series or parallel**: compose availability through topology → primitive 10.
- [ ] **SLO exists or is being set**: derive error budget and burn-rate alert thresholds → primitive 08.
- [ ] **System target must be distributed to teams or suppliers**: allocate per-subsystem reliability targets → primitive 11.
- [ ] **Pre-launch reliability review required**: enumerate failure modes with RPN ranking → primitive 06.
- [ ] **High-severity failure modes identified in FMEA**: build fault tree for those top events → primitive 05.
- [ ] **Adding redundancy**: verify coverage is sufficient; check whether redundancy helps or hurts → primitive 07. Then **validate coverage probability with fault injection experiments** before treating the redundancy as live (see primitive 07 Validation section).
- [ ] **Failure time data available**: fit Weibull for B10 life and maintenance scheduling → primitive 09.
- [ ] **Unclear which lifecycle phase the system is in**: classify hazard rate shape → primitive 03.
- [ ] **New deployment or hardware received**: plan burn-in; watch for infant-mortality phase → primitive 04.
- [ ] **AI/LLM agent system**: do not use MTBF directly — measure pass^k (k ≥ 10) for consistency; use Markov chain step-reliability for sequential tool chains (primitive 10 extension); measure RDC across task-duration buckets. See [`references/ai-agent-reliability.md`](references/ai-agent-reliability.md).

---

## Composition Recipes

Full composition guide and domain-scenario stacks live in [`assets/templates/reliability-theory/README.md`](assets/templates/reliability-theory/README.md).

Quick stacks:

**Service availability target** — establish and validate an SLO against real architecture:
Primitive 01 (measure MTBF/MTTR) → Primitive 02 (compute A per component) → Primitive 10 (compose through topology) → Primitive 11 (allocate target to lagging subsystems) → Primitive 08 (set error budget and burn-rate alerts).

**Worked example:** SLO 99.9% monthly = 43.8 min downtime budget. Single instance: MTBF=720 h, MTTR=2 h → A = 720/(720+2) = 99.72% → 121 min/month, blows budget. Add active-active pair (independent failures): A_pair ≈ 1 − (1−0.9972)² = 99.9992% → 0.35 min/month. But shared DB caps at A_DB = 99.95% → real A = min(99.9992%, 99.95%) = 99.95% (common-cause correction, primitive 10). Bathtub note: failure spike in first 30 days post-deploy is infant mortality (DFR phase), not random CFR — use canary or burn-in, not autoscale.

**FMEA before launch** — find and rank failure risks before shipping:
Primitive 06 (FMEA worksheet, RPN ranking) → Primitive 05 (fault tree for top S≥9 items, find SPOFs) → Primitive 07 (redundancy math for identified SPOFs) → Primitive 06 again (re-score residual RPN after mitigations).

**Post-incident reliability update** — update models and improve after an incident:
Primitive 01 (update MTBF/MTTR from incident) → Primitive 03 (re-classify hazard phase) → Primitive 09 (re-fit Weibull if ≥10 failures available) → Primitive 06 (add failure mode to FMEA) → Primitive 11 (re-allocate targets to subsystems that fell below spec).

**AI agent system reliability baseline** — establish a defensible reliability figure for an LLM agent pipeline:
Step 1: Run ≥10 independent episodes per task bucket (short / medium / long) — compute pass^k per bucket; do not use pass@1 alone.
Step 2: Map the agent's tool calls to a series chain → apply Markov step-reliability (primitive 10 extension): R_system = ∏ rᵢ.
Step 3: Run FMEA (primitive 06) with agent-specific failure modes (context overflow, tool hallucination, schema drift, rate-limit cascade) — score all S≥9 items independently of RPN.
Step 4: For highest-severity items (S≥9), build fault trees (primitive 05) and augment with STPA where control-loop hazards are present.
Step 5: Set error budget (primitive 08) extending to a correctness budget — fraction of responses meeting a quality bar — alongside availability.

**Worked example (agent pipeline):** A 5-step research agent with per-step reliabilities [0.98, 0.95, 0.97, 0.92, 0.99] → R_system = 0.98 × 0.95 × 0.97 × 0.92 × 0.99 ≈ 0.823. Single-run pass@1 on a short task was 0.94 — the chain composition reveals a ≈17.7% expected failure rate on full-length runs, far worse than the short-task figure suggests. Bottleneck is step 4 (r=0.92); improving it to 0.97 raises R_system to ≈0.867. (Corrected 2026-07-11: prior figures of 0.824/0.862 were rounding errors; exact products are 0.8225 and 0.8672.)

---

## Domain Applicability Notes

Different system types call for different subsets of the 11 primitives. The core arithmetic is domain-agnostic; the calibration data and vocabulary shift.

**Hardware / IEC context** (electronic equipment, safety instrumented systems, aerospace): All 11 primitives apply directly. MIL-HDBK-217, IEC 61508, ISO 26262, and DO-178C provide failure-rate data. Bathtub curve and Weibull analysis (primitives 04, 09) are first-class tools. FMEA governed by IEC 60812. FTA by IEC 61025.

**Software / SRE context** (cloud services, microservices, distributed systems): Primitives 01, 02, 08, 10 are the workhorses. The bathtub curve maps to the deploy lifecycle: infant-mortality phase corresponds to the first 24–72 hours post-deploy; random-failure phase is steady-state operation; wear-out corresponds to technical debt accumulation and dependency rot. SRGM critique applies (Xie 1991 assumes monotone DFR; breaks when code changes during the observation window).

**AI / LLM agent systems**: Classic MTBF is not directly applicable to stochastic-per-run agents. Three adaptations are required:
- Replace single-run availability with **pass^k** (all k runs succeed; k ≥ 10 minimum) as the consistency baseline.
- Use **Reliability Decay Curve (RDC)** and **Variance Amplification Factor (VAF)** to measure duration-dependent degradation rather than a time-stationary failure rate.
- Model sequential tool-call chains with **Markov chain step-reliability** (primitive 10 extension): R_system = ∏ rᵢ where rᵢ is the step-level reliability of each tool call.
- Treat R = ∏ rᵢ as an **upper bound** for multi-agent topologies, not an estimate — inter-agent handoffs add failure modes that no per-step reliability captures (see Anti-Patterns).
Full vocabulary and worked examples: [`references/ai-agent-reliability.md`](references/ai-agent-reliability.md).

**Where the reliability actually comes from.** A 2026 cross-benchmark decomposition of a production enterprise agent (Dastidar 2026, arXiv:2607.17044) found the uplift over the frontier base model came mostly from scaffolding, routing, and specialist-model selection — the verification loop contributed only ≈+1.5 points in isolation, with an instrumented catch rate of ≈0.20 and fix rate of 0.75. Two consequences for reliability allocation (primitive 11) on agent system