Agent skill · NVIDIA
tao-run-inference-service
Start, query, and stop a network-specific TAO inference microservice ({network_arch}-inference-microservice) by delegating container execution to the appropriate platform skill. Handles container image resolution, job-payload JSON construction, and the service registry. Use when the user wants to run inference on a TAO model checkpoint using a microservice container, deploy a TAO inference endpoint, or stop a running inference container.
What it needs
About 10k tokens when loaded.
What this skill does
TAO Inference Microservice Standalone install? If this session was not initialized by the TAO skill bank plugin, run the tao-setup skill first (host preflight, credentials, cross-skill discovery). Instructions To start an inference service: 1. Collect required inputs (Section 1) and resolve the container image (Section 2). 2. Build the job payload and inner command (Sections 3–4.1); use references/code-templates.yaml → jobpayloadbuilder. 3. Read skills/platform/<platform>/SKILL.md and start the container (Section 4.2). 4. Write the service registry and poll readiness (Section 4.3); use references/code-templates.yaml → registrywrite.<platform> and readinesscheck. To send an inference request: 1. Resolve which service receives the request per Section 6.0 (by jobid, by networkarch, or by explicit user choice when multiple services run — never silently default to "latest" when more than one service exists), then read the endpoint from references/code-templates.yaml → request.registryread with the resolved jobid. 2. Before building the request body, prompt the user for the vLLM-style sampling parameters (Section 6.1). Present maxtokens, topp, temperature (and any per-arch extras) with their defaults; let the user override or skip each one to accept the default. Never silently use defaults. 3. Build and send the body per Section 6.2; handle the response per Section 6.3. To stop a service: Read references/code-templates.yaml → stop.registryread to resolve the jobid, read skills/platform/<platform>/SKILL.md, then follow Section 5. Reference data (schemas, mappings, valid values — no instructions): references/service.yaml — image mappings, valid networkarch names, job payload schema, env var names, secrets classification. references/request.yaml — endpoint definition, request field schema, response shapes, code examples. references/code-templates.yaml — Python templates for payload building, registry writes, readiness checks, and stop/request flows. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills NVIDIA/tao-run-inference-service