Agent skill · writing content · alirezarezvani
senior-prompt-engineer
Improve prompts and measure LLM quality: fix inconsistent prompts, build eval sets, score RAG retrieval, check agent tool configs. Use whenever the user's prompt, chatbot or RAG answers are unreliable or they ask how to test them.
Why this skill is useful
Adds three executable Python scripts for prompt optimization, RAG evaluation, and agent orchestration that provide capabilities not found in standard practices.
What it needs
Requires python3 installed locally. About 5k tokens when loaded. Last updated 2026-08-05. 23,939 stars on the source repository.
What this skill does
Senior Prompt Engineer Eval-driven prompt engineering, RAG quality measurement, and agent workflow validation. Everything here is model-agnostic by design: techniques are framed by what they do, not by which model generation they were observed on, and the tools never hardcode model IDs or pricing — you supply your provider's current rates when you want dollar figures. Operating Rules 1. Never change a prompt without a baseline. Capture metrics first (--analyze --output baseline.json), then compare every iteration against it. 2. Eval set before optimization. 10–20 representative cases with expected outputs minimum. If the user has no eval set, build one with them before touching the prompt — optimizing against vibes is the #1 failure mode. 3. Prefer platform features over prompt hacks. If the provider offers native structured outputs / JSON schema enforcement, tool-use APIs, or prompt caching, use those instead of "respond ONLY with JSON" incantations. Prompt-level format enforcement is the fallback, not the default. 4. Current-generation models need less scaffolding. Don't add chain-of-thought boilerplate, role framing, or few-shot examples reflexively — frontier models often do worse with redundant scaffolding. Add each element only when the eval set shows it helps. 5. Cost numbers are always user-supplied. Look up the provider's current per-Mtok pricing and pass it via --price-per-mtok (never trust a cached price table — including any you remember). Tools (exact CLIs, all stdlib) 1. Prompt Optimizer — scripts/promptoptimizer.py Static analysis: token estimate, clarity/structure scores (0–100), ambiguity + redundancy detection, few-shot example extraction. --model accepts any string; only the tokenizer family is inferred (names containing "claude" → 3.5 chars/token, otherwise 4.0). Exit 0 on success, 1 on missing file. 2. RAG Evaluator — scripts/ragevaluator.py Measures retrieval and grounding quality from two JSON files (formats printed in --help). …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills alirezarezvani/senior-prompt-engineer--186852