Agent skill · data analytics · davila7
forecast-accuracy-review
Evaluate demand-forecast quality honestly - WMAPE, bias and Forecast Value Added against a naive benchmark over a rolling-origin backtest. Use when the user mentions forecast accuracy, MAPE, demand planning performance, tahmin doğruluğu, talep tahmini, or asks whether a forecasting process or tool is worth it. Differentiator - judges the process (value added vs doing nothing), not just the model.
Why this skill is useful
Provides detailed domain-specific metrics and methodologies for evaluating forecast accuracy that the AI wouldn't reliably generate on its own.
What it needs
About 2k tokens when loaded. Last updated 2026-08-06. 30,138 stars on the source repository.
What this skill does
Forecast Accuracy Review A forecast is only worth what it adds over the free alternative: shipping last period's number. Every review must answer "how many points does this process add over naive?" before any model discussion. Required data Per-SKU demand history at the planning bucket (usually monthly): sku, period, qty. If evaluating an existing forecast, also the forecast values with their creation dates (to avoid hindsight leakage). 18+ periods per SKU for a meaningful backtest; flag SKUs with less. Workflow 1. Profile the demand first. Per SKU compute mean, CV and zero-period share; classify smooth / erratic / intermittent / lumpy (defaults: CV 0.5 and 1.0 boundaries, intermittency at >25% zero periods - state them, adjust to natural breaks). Accuracy expectations differ by class; never report one blended number alone. 2. Set the benchmarks. Naive (last period) always; seasonal naive when 2+ full seasons exist. These are non-negotiable controls. 3. Backtest rolling-origin. One-step-ahead forecasts for each of the last 6+ periods, expanding window, using only data before each origin. A single train/test split is one lucky draw - do not accept it. 4. Score with honest metrics: WMAPE = sum( error ) / sum(actual) - the volume-weighted headline Bias = sum(error) / sum(actual) - direction; a fine WMAPE with persistent bias is quietly building excess stock or stockouts MAPE only as a footnote, and always disclose how many zero-actual periods it dropped 5. Deliver the FVA verdict. FVA = WMAPE(naive) - WMAPE(candidate), per segment and overall. Negative FVA means the process destroys value - say it plainly. 6. Validate. Recompute WMAPE for one model directly from the raw backtest rows and confirm it matches the table before presenting. Pitfalls to check explicitly MAPE with zeros: undefined on zero-actual periods; silently dropping them fakes precision on intermittent SKUs. MAPE asymmetry rewards under-forecasting (errors capped at 100% below, unbounded above). …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills davila7/forecast-accuracy-review