Agent skill · NVIDIA
rag-perf
Performance benchmarking for a deployed NVIDIA RAG Blueprint server: profiling pass + aiperf load test driven by a single YAML config. Not for accuracy / RAGAS scoring (use rag-eval) or for deploying / repairing services (use rag-blueprint).
What it needs
About 8k tokens when loaded.
What this skill does
RAG-Perf — config-driven perf benchmark CLI Purpose Drive a deployed NVIDIA RAG Blueprint server with a YAML config, run a server-side profiling pass (per-stage timing, citation quality, bottleneck inference) and an optional aiperf load test (TTFT / E2E / token & request throughput / error rate), and write a unified report. The CLI is intentionally minimal: rag-perf -c <config> plus --help / --version. Behaviour is fully config-driven; field variations belong in YAML. Scope Accuracy / RAGAS scoring of answer quality → use the rag-eval skill. Deploying, repairing, or configuring services (compose, helm, NIM env vars) → use the rag-blueprint skill. Production monitoring / alerting — rag-perf is a one-shot benchmark tool. Runtime requirement: a deployed RAG server reachable on the network. Prerequisites Repo cloned; run commands from the repo root (config paths in the presets are repo-root-relative). Python 3.11+ and uv on PATH. Install rag-perf into its own uv-managed venv: uv sync --project scripts/rag-perf. For unit tests: install dev extras as well — uv sync --project scripts/rag-perf --extra dev (otherwise pytest-asyncio is missing and async tests error out at collection time). A reachable RAG server (default http://localhost:8081). For the aiperf phase, the bundled nvidiarag endpoint plugin must be installed — pip install -e ./scripts/rag-perf registers it via the aiperf.plugins entry point. For synthetic queries: an OpenAI-compatible chat-completions endpoint reachable at synthetic.llmurl (default http://localhost:8999/v1/chat/completions). rag-perf itself runs without NVIDIAAPIKEY (unlike rag-eval). The synthetic LLM endpoint may require its own auth — that's the deployment's concern. Instructions 1. Pick a preset. The three under scripts/rag-perf/configs/ are: quickprofile.yaml — profile-only, ~30 s. Skips load test. For fast iteration on retrieval / reranker tuning. singlerun.yaml — one concurrency level, profiling + aiperf, ~2 min. Regression checks. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills NVIDIA/rag-perf