Agent skill · NVIDIA
jetson-llm-benchmark
Benchmark Jetson LLM/VLM serving performance across vLLM, llama.cpp, and Ollama with structured JSON output.
What it needs
About 6k tokens when loaded.
What this skill does
Jetson LLM Benchmark Reproducible Jetson benchmarks with structured JSON output so an agent can compare runs. Encodes the workflow from the Jetson AI Lab GenAI Benchmarking tutorial. Purpose Measure deployed LLM latency and throughput on a Jetson target using the correct runtime-specific benchmark wrapper. Use the JSON output to compare models, runtime flags, power modes, and before/after tuning changes. Prerequisites Run on the Jetson device that hosts the model runtime. For vLLM, start the OpenAI-compatible vLLM server first and know the served model ID. For Ollama, ensure the Ollama daemon is reachable at --endpoint and the named model is already pulled. For llama.cpp/GGUF, provide a readable .gguf model path on the host. Put the device in the intended power mode before measuring. MAXN is preferred for comparable performance numbers. Available Scripts Script Purpose Arguments -------- --------- ----------- scripts/benchvllm.sh Runs vllm bench serve against a running OpenAI-compatible vLLM server. --model, --endpoint, --concurrency, --input-len, --output-len, --num-prompts, --no-warmup, --container, --native. scripts/benchllamacpp.sh Runs llama-bench for a local GGUF model through the Jetson-appropriate NVIDIA-AI-IOT llama.cpp container. --model, --n-prompt, --n-gen, --n-gpu-layers, --threads, --container. scripts/benchollama.sh Benchmarks a local or containerized Ollama daemon through the /api/generate REST API. --model, --endpoint, --num-prompts, --input-len, --output-len, --no-warmup. If your agent runtime supports runscript, invoke the selected wrapper directly with the user-provided model identifier or local model path, then summarize the returned JSON. Otherwise run the wrapper with bash {baseDir}/scripts/<wrapper-name> .... …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills NVIDIA/jetson-llm-benchmark