Agent skill · software engineering · nvidia
rag-perf
Performance benchmarking for a deployed NVIDIA RAG Blueprint server: profiling pass + aiperf load test driven by a single YAML config. Not for accuracy / RAGAS scoring (use rag-eval) or for deploying / repairing services (use rag-blueprint).
Why this skill is useful
Adds specific commands and configurations for performance benchmarking of NVIDIA RAG servers that the AI wouldn't reliably generate on its own.
What it needs
Requires python, uv installed locally. About 8k tokens when loaded. Last updated 2026-08-07. 2,816 stars on the source repository.
What this skill does
RAG-Perf — config-driven perf benchmark CLI Purpose Drive a deployed NVIDIA RAG Blueprint server with a YAML config, run a server-side profiling pass (per-stage timing, citation quality, bottleneck inference) and an optional aiperf load test (TTFT / E2E / token & request throughput / error rate), and write a unified report. The CLI is intentionally minimal: rag-perf -c <config> plus --help / --version. Behaviour is fully config-driven; field variations belong in YAML. Scope Accuracy / RAGAS scoring of answer quality → use the rag-eval skill. Deploying, repairing, or configuring services (compose, helm, NIM env vars) → use the rag-blueprint skill. Production monitoring / alerting — rag-perf is a one-shot benchmark tool. Runtime requirement: a deployed RAG server reachable on the network. Prerequisites Repo cloned; run commands from the repo root (config paths in the presets are repo-root-relative). Python 3.11+ and uv on PATH. Install rag-perf into its own uv-managed venv: uv sync --project scripts/rag-perf. For unit tests: install dev extras as well — uv sync --project scripts/rag-perf --extra dev (otherwise pytest-asyncio is missing and async tests error out at collection time). A reachable RAG server (default http://localhost:8081). For the aiperf phase, the bundled nvidiarag endpoint plugin must be installed — pip install -e ./scripts/rag-perf registers it via the aiperf.plugins entry point. For synthetic queries: an OpenAI-compatible chat-completions endpoint reachable at synthetic.llmurl (default http://localhost:8999/v1/chat/completions). rag-perf itself runs without NVIDIAAPIKEY (unlike rag-eval). The synthetic LLM endpoint may require its own auth — that's the deployment's concern. Instructions 1. Pick a preset. The three under scripts/rag-perf/configs/ are: quickprofile.yaml — profile-only, ~30 s. Skips load test. For fast iteration on retrieval / reranker tuning. singlerun.yaml — one concurrency level, profiling + aiperf, ~2 min. Regression checks. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills nvidia/rag-perf