Agent skill · data analytics · wshobson
evaluation-methodology
PluginEval quality methodology — dimensions, rubrics, statistical methods, and scoring formulas. Use this skill when understanding how plugin quality is measured, when interpreting a low score on a specific dimension, when deciding how to improve a skill's triggering accuracy or orchestration fitness, when calibrating scoring thresholds for your marketplace, or when explaining quality badges to external partners like Neon.
Why this skill is useful
Provides detailed statistical methods and scoring formulas for evaluating plugin quality that the AI wouldn't reliably generate on its own.
What it needs
About 12k tokens when loaded. Last updated 2026-08-05. 38,570 stars on the source repository.
What this skill does
Evaluation Methodology This document is the authoritative reference for how PluginEval measures plugin and skill quality. It covers the three evaluation layers, all ten scoring dimensions, the composite formula, badge thresholds, anti-pattern flags, Elo ranking, and actionable improvement tips. Related: Full rubric anchors --- The Three Evaluation Layers PluginEval stacks three complementary layers. Each layer produces a score between 0.0 and 1.0 for each applicable dimension, and later layers override or blend with earlier ones according to per-dimension blend weights. Layer 1 — Static Analysis Speed: < 2 seconds. No LLM calls. Deterministic. The static analyzer (layers/static.py) runs six sub-checks directly against the parsed SKILL.md: Sub-check What it measures --- --- frontmatterquality Name presence, description length, trigger-phrase quality orchestrationwiring Output/input documentation, code block count, orchestrator anti-pattern progressivedisclosure Line count vs. sweet-spot (200–600 lines), references/ and assets/ bonuses structuralcompleteness Heading density, code blocks, examples section, troubleshooting section tokenefficiency MUST/NEVER/ALWAYS density, duplicate-line repetition ratio ecosystemcoherence Cross-references to other skills/agents, "related"/"see also" mentions These six sub-checks feed directly into six of the ten final dimensions (via STATICTODIMENSION mapping). The remaining four dimensions — outputquality, scopecalibration, robustness, and part of triggeringaccuracy — receive no static contribution and rely entirely on Layer 2 and/or Layer 3. Anti-pattern penalty is applied multiplicatively to the Layer 1 score: Each additional detected anti-pattern reduces the score by 5%, flooring at 50%. Layer 2 — LLM Judge Speed: 30–90 seconds. One or more LLM calls (Sonnet by default). Non-deterministic. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills wshobson/evaluation-methodology