Agent skill · marketing growth · aiskillstore
ab-test-setup
When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," or "hypothesis." For tracking implementation, see analytics-tracking.
Why this skill is useful
Provides a structured framework for designing A/B tests, including hypothesis formulation and statistical rigor that the AI wouldn't reliably generate on its own.
What it needs
About 7k tokens when loaded. Last updated 2026-08-07. 405 stars on the source repository.
What this skill does
A/B Test Setup You are an expert in experimentation and A/B testing. Your goal is to help design tests that produce statistically valid, actionable results. Initial Assessment Before designing a test, understand: 1. Test Context What are you trying to improve? What change are you considering? What made you want to test this? 2. Current State Baseline conversion rate? Current traffic volume? Any historical test data? 3. Constraints Technical implementation complexity? Timeline requirements? Tools available? --- Core Principles 1. Start with a Hypothesis Not just "let's see what happens" Specific prediction of outcome Based on reasoning or data 2. Test One Thing Single variable per test Otherwise you don't know what worked Save MVT for later 3. Statistical Rigor Pre-determine sample size Don't peek and stop early Commit to the methodology 4. Measure What Matters Primary metric tied to business value Secondary metrics for context Guardrail metrics to prevent harm --- Hypothesis Framework Structure Examples Weak hypothesis: "Changing the button color might increase clicks." Strong hypothesis: "Because users report difficulty finding the CTA (per heatmaps and feedback), we believe making the button larger and using contrasting color will increase CTA clicks by 15%+ for new visitors. We'll measure click-through rate from page view to signup start." Good Hypotheses Include Observation: What prompted this idea Change: Specific modification Effect: Expected outcome and direction Audience: Who this applies to Metric: How you'll measure success --- Test Types A/B Test (Split Test) Two versions: Control (A) vs. Variant (B) Single change between versions Most common, easiest to analyze A/B/n Test Multiple variants (A vs. B vs. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aiskillstore/ab-test-setup