Agent skill · NVIDIA
deepstream-sop
Use this skill when building, deploying, evaluating, debugging, or measuring latency for the DeepStream SOP Inference Microservice — a GPU-accelerated FastAPI service that detects whether operators perform assembly-line steps in order via event boundary detection (GEBD) plus VLM classification. Trigger even if the user does not name it: verify operator step sequence, detect missing or out-of-order SOP steps, score factory/work-cell video for procedure compliance, run VLM-based SOP checking on industrial cameras, or call /v1/chat/completions with a file, RTSP, or Basler camera. Also trigger for its internals: SOPVideoProcessor, DeepStream GEBD model (e.g. DDM) via Triton CAPI, nvds_custom_postprocess, Cosmos Reason 1/2 vLLM, SSE streaming, Kafka NvProto/JSON output, Basler/Pylon camera + emulation, Docker compose, chunk-level latency. Do NOT trigger for generic DeepStream pipelines, object detection/tracking, NIM imports, or video summarization.
What it needs
About 8k tokens when loaded.
What this skill does
DeepStream SOP Inference Microservice Skill This skill guides AI coding assistants in building, extending, and debugging the NVIDIA DeepStream SOP (Standard Operating Procedure) Inference Microservice — a GPU-accelerated pipeline for temporal action detection and VLM-based SOP compliance monitoring on industrial video feeds. Reference repository: https://github.com/NVIDIA/sop-monitoring-blueprints/tree/main/microservices/sop-inference-bp Local reference code: sop-inference-bp/ directory (from a local clone of the repository) --- Models Model-agnostic at both inference stages — swap via env var (and Triton dir for GEBD). Stage Role Model class Default Swap via ------ ------ ------------- --------- ---------- Stage 1 (CV) Per-frame boundary scoring → chunk segmentation Generic Event Boundary Detection (GEBD) DDM (MCG-NJU/DDM) via Triton Python backend Replace tritonmodelrepo/<model>/ + DDMMODELPATH (§ 5) Stage 3 (VLM) Per-chunk action classification Vision-language model via vLLM Cosmos Reason 1 7B (Reason 2 also supported) Set VLLMMODELPATH to a different HF ID or local path "GEBD" = swappable Stage-1 slot; "DDM" = the default architecture (terms used interchangeably). Chunking is selectable per request (§ 2): default ddm-net uses GEBD; uniform produces fixed-length chunks and bypasses Stage-1 GEBD (§ 3, § 6). DDM temporal window is configurable via FRAMESPERSIDE / SEQUENCEBATCH (§ 4, § 5), with optional TensorRT (§ 5). --- Architecture Overview Runs in a Docker container (nvds-action-sop) alongside a Kafka container. Full diagram: references/soparchitecture.svg. Data flow through the 4-stage SOPVideoProcessor pipeline (per-request): --- Section Index Each section is a standalone file in references/ — load only what your task needs. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills NVIDIA/deepstream-sop