Agent skill · google
google-agents-cli-eval
This skill should be used when the user wants to "run an evaluation", "evaluate my agent", "evaluate my ADK agent", "write an eval dataset", "analyze eval failures", "compare eval results", "optimize agent", or needs guidance on the Agent Platform eval methodology and the Quality Flywheel. Covers eval metrics, dataset schema, LLM-as-judge scoring, and common failure causes. Applies to any agents-cli project, whatever framework the agent is written in. Do NOT use for agent API code patterns (ADK: use google-agents-cli-adk-code), deployment (use google-agents-cli-deploy), or project scaffolding (use google-agents-cli-scaffold).
What it needs
About 11k tokens when loaded.
What this skill does
Agent Evaluation Guide Requires: agents-cli (uv tool install google-agents-cli) — install uv first if needed. Scaffolded project? If you used /google-agents-cli-scaffold, you already have agents-cli eval run (chains generate + grade), tests/eval/datasets/, and tests/eval/evalconfig.yaml. Start with executing eval run and iterate from there. Reference Files File Contents ------ ---------- references/datasetschema.md Canonical EvaluationDataset schema — all field types, JSON examples for single-turn / multi-turn / multi-agent, common mistakes references/metrics-guide.md Complete metrics reference — all built-in metrics, match types, custom metrics, judge model config references/user-simulation.md Dynamic conversation testing — eval dataset synthesize flags, what scenarios are, compatible metrics references/builtin-tools-eval.md googlesearch and model-internal tools — trajectory behavior, metric compatibility references/advanced-commands.md Opt-in commands: eval analyze, eval optimize, eval submit / eval results references/multimodal-eval.md Multimodal inputs — eval dataset schema, built-in metric limitations, custom evaluator pattern --- The Quality Flywheel Improving agent quality is iterative. The 4 stages below describe the loop. Each stage has a Default path (you, the coding agent, do the work directly) and an Opt-in CLI command that delegates to the Agent Platform Eval Service for better quality and scale. 1. Prepare Data Default: Use or edit the scaffolded tests/eval/datasets/basic-dataset.json to define single-turn eval inputs. Start with 1–2 cases. Opt-in (ADK projects): agents-cli eval dataset synthesize: user-simulate multi-turn datasets when you lack data; its output already includes traces, so Stage 2 collapses to agents-cli eval grade alone. See Eval Commands and references/user-simulation.md. 2. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills google/google-agents-cli-eval--9e5d0a