Agent skill · operations · yonatangross
testing-llm
LLM testing patterns: mocked responses, DeepEval and RAGAS evaluation, structured output checks, agentic tests. Use when testing AI features.
Why this skill is useful
Provides domain-specific testing patterns and tools for evaluating LLM outputs that the AI wouldn't reliably generate on its own.
What it needs
Requires deepeval, ragas installed locally. About 5k tokens when loaded. Last updated 2026-08-06. 215 stars on the source repository.
What this skill does
LLM & AI Testing Patterns Patterns and tools for testing LLM integrations, evaluating AI output quality, mocking responses for deterministic CI, and applying agentic test workflows (planner, generator, healer). Of that trio only the healer keeps a local reference here; the planner and generator stages belong to the testing-e2e skill. Quick Reference Area File Purpose ------ ------ --------- Rules rules/llm-evaluation.md DeepEval quality metrics, Pydantic schema validation, timeout testing Rules rules/llm-mocking.md Mock LLM responses, VCR.py recording, custom request matchers Reference references/ork-delta.md House rules the vendor docs do not carry: GEval and RAGAS API corrections, threshold direction, cassette path, golden-dataset and latency budgets Reference references/healer-agent.md Auto-fixes failing tests (selectors, waits, dynamic content) Reference references/langfuse-v4.md Langfuse Python SDK v4 tracing and dataset runs, plus the non-zero throughput assertion Checklist checklists/llm-test-checklist.md Complete LLM testing checklist (setup, coverage, CI/CD) Upstream coverage (do not restate) DeepEval, RAGAS, VCR.py and Playwright document themselves. This skill carries only the OrchestKit delta (references/ork-delta.md) plus the house subsets in rules/ and checklists/. Fetch the source below instead of expecting the material here. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills yonatangross/testing-llm