Agent skill · creative production · github
agentic-eval
Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-judge evaluation systems - Adding iterative improvement to agent outputs (code, reports, analysis) - Measuring and improving agent response quality
Why this skill is useful
Adds multiple executable scripts for iterative evaluation and refinement of AI outputs, enhancing self-improvement capabilities.
What it needs
About 3k tokens when loaded. Last updated 2026-08-07. 37,534 stars on the source repository.
What this skill does
Agentic Evaluation Patterns Patterns for self-improvement through iterative evaluation and refinement. Overview Evaluation patterns enable agents to assess and improve their own outputs, moving beyond single-shot generation to iterative refinement loops. When to Use Quality-critical generation: Code, reports, analysis requiring high accuracy Tasks with clear evaluation criteria: Defined success metrics exist Content requiring specific standards: Style guides, compliance, formatting --- Pattern 1: Basic Reflection Agent evaluates and improves its own output through self-critique. Key insight: Use structured JSON output for reliable parsing of critique results. --- Pattern 2: Evaluator-Optimizer Separate generation and evaluation into distinct components for clearer responsibilities. --- Pattern 3: Code-Specific Reflection Test-driven refinement loop for code generation. --- Evaluation Strategies Outcome-Based Evaluate whether output achieves the expected result. LLM-as-Judge Use LLM to compare and rank outputs. Rubric-Based Score outputs against weighted dimensions. --- Best Practices Practice Rationale ---------- ----------- Clear criteria Define specific, measurable evaluation criteria upfront Iteration limits Set max iterations (3-5) to prevent infinite loops Convergence check Stop if output score isn't improving between iterations Log history Keep full trajectory for debugging and analysis Structured output Use JSON for reliable parsing of evaluation results --- Quick Start Checklist
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills github/agentic-eval