Agent skill · data analytics · affaan-m
ai-regression-testing
Regression testing strategies for AI-assisted development. Sandbox-mode API testing without database dependencies, automated bug-check workflows, and patterns to catch AI blind spots where the same model writes and reviews code. Use when adding regression coverage to AI-assisted code, or when the same model both wrote and reviewed a change.
Why this skill is useful
Adds executable scripts and testing patterns specifically for AI regression testing that aren't commonly documented.
What it needs
Requires vitest installed locally. About 7k tokens when loaded. Last updated 2026-08-06. 238,342 stars on the source repository.
What this skill does
AI Regression Testing Testing patterns specifically designed for AI-assisted development, where the same model writes code and reviews it — creating systematic blind spots that only automated tests can catch. When to Activate AI agent (Claude Code, Cursor, Codex) has modified API routes or backend logic A bug was found and fixed — need to prevent re-introduction Project has a sandbox/mock mode that can be leveraged for DB-free testing Running /bug-check or similar review commands after code changes Multiple code paths exist (sandbox vs production, feature flags, etc.) The Core Problem When an AI writes code and then reviews its own work, it carries the same assumptions into both steps. This creates a predictable failure pattern: Real-world example (observed in production): The pattern: sandbox/production path inconsistency is the #1 AI-introduced regression. Sandbox-Mode API Testing Most projects with AI-friendly architecture have a sandbox/mock mode. This is the key to fast, DB-free API testing. Setup (Vitest + Next.js App Router) Test Helper for Next.js API Routes Writing Regression Tests The key principle: write tests for bugs that were found, not for code that works. Testing Sandbox/Production Parity The most common AI regression: fixing production path but forgetting sandbox path (or vice versa). Integrating Tests into Bug-Check Workflow Custom Command Definition The Workflow Common AI Regression Patterns Pattern 1: Sandbox/Production Path Mismatch Frequency: Most common (observed in 3 out of 4 regressions) Test to catch it: Pattern 2: SELECT Clause Omission Frequency: Common with Supabase/Prisma when adding new columns Pattern 3: Error State Leakage Frequency: Moderate — when adding error handling to existing components Pattern 4: Optimistic Update Without Proper Rollback Strategy: Test Where Bugs Were Found Don't aim for 100% coverage. Instead: Why this works with AI development: 1. AI tends to make the same category of mistake repeatedly 2. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills affaan-m/ai-regression-testing--244efe