Agent skill · research science · aj-geddes
Statistical Hypothesis Testing
Conduct statistical tests including t-tests, chi-square, ANOVA, and p-value analysis for statistical significance, hypothesis validation, and A/B testing
Why this skill is useful
Provides specific implementations of statistical tests that the AI wouldn't reliably generate on its own.
What it needs
About 4k tokens when loaded. Last updated 2026-03-04. 307 stars on the source repository.
What this skill does
Statistical Hypothesis Testing Overview Hypothesis testing provides a framework for making data-driven decisions by testing whether observed differences are statistically significant or due to chance. Testing Framework Null Hypothesis (H0): No effect or difference exists Alternative Hypothesis (H1): Effect or difference exists Significance Level (α): Threshold for rejecting H0 (typically 0.05) P-value: Probability of observing data if H0 is true Common Tests T-test: Compare means between two groups ANOVA: Compare means across multiple groups Chi-square: Test independence of categorical variables Mann-Whitney U: Non-parametric alternative to t-test Kruskal-Wallis: Non-parametric alternative to ANOVA Implementation with Python Interpretation Guidelines p < 0.05: Statistically significant (reject H0) p ≥ 0.05: Not statistically significant (fail to reject H0) Effect size: Magnitude of the difference (small/medium/large) Confidence intervals: Range of plausible parameter values Assumptions Checklist Independence of observations Normality of distributions (parametric tests) Homogeneity of variance Appropriate sample size Random sampling Common Pitfalls Misinterpreting p-values Multiple testing without correction Ignoring effect sizes Violating test assumptions Confusing correlation with causation Deliverables Test results with p-values and test statistics Effect size calculations Visualization of distributions Confidence intervals Interpretation and business implications
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aj-geddes/statistical-hypothesis-testing