Agent skill · magnus919
vllm
Operate, configure, benchmark, and troubleshoot vLLM inference servers: Docker and Kubernetes deployment, quantization-aware model configuration (tensor parallelism, KV cache), OpenAI-compatible API serving, throughput and latency benchmarking, continuous batching tuning, GPU operation, and upgrade/rollback. Use when deploying or running a vLLM server (vllm serve, vllm/vllm-openai), sizing a model and its KV cache for GPUs, selecting quantization and parallelism, serving via /v1 endpoints, measuring serving throughput or latency, tuning batching, or diagnosing GPU, OOM, or startup failures in a vLLM deployment. Do not use for model training, fine-tuning, evaluation-set design, or engine-selection methodology (that is ml-engineering), or for operating the llama.cpp stack with GGUF models (that is llama-cpp); other inference engines (TGI, Ollama, Triton) are out of scope.
What it needs
About 9k tokens when loaded.
What this skill does
vLLM Inference Serving Use this skill to operate vLLM as a production inference server: deploy it with Docker or Kubernetes, configure the model and engine (quantization, tensor parallelism, KV cache, context length), serve the OpenAI-compatible API surface, benchmark throughput and latency with comparable evidence, tune continuous batching, operate the GPUs underneath, and upgrade or roll back safely. This is a tool skill for one named engine. Serving methodology — engine selection, quantization trade-offs, deployment plans, regression triage — belongs to ml-engineering; local single-node GGUF serving with the llama.cpp stack belongs to llama-cpp. This skill owns the day-to-day operation of vLLM itself. Operating contract 1. Record the deployment before tuning it. Capture the vLLM version or image digest, model and revision, quantization, parallelism, max-model-len, KV cache settings, batching limits, GPU inventory, and workload. The serving config template exists for exactly this. 2. Confirm the target, scope, and rollback path before acting. Read-only discovery (health probes, /metrics, nvidia-smi) may proceed without confirmation. Mutations — restarting a server, changing serving args, scaling replicas, upgrading the image — require an explicit human directive naming the deployment. 3. A server that responds is not a server that serves. /health returning 200 proves liveness, not that the model loaded or that inference works. Verify at the delivery boundary: /v1/models reports the served model and a representative request returns generated tokens. 4. Benchmark before and after every change. vLLM flags, defaults, and behavior change between releases; an unmeasured tuning change is a guess. Compare only matched conditions (version, model, GPU, context, batch, workload) and record the evidence in the benchmark run record. 5. Keep evidence bounded. Summarize logs, configs, and metrics; never dump full server logs, .env files, or HF tokens into chat. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills magnus919/vllm