Agent skill · NVIDIA
tao-generate-video-reasoning-annotations
Multi-step video annotation pipeline that turns raw videos into Chain-of-Thought training data — multi-level captions, structured descriptions, and QA pairs (MCQ, binary, open-ended) with reasoning traces, via VLM/LLM distillation. Use when the user wants to "create video training data", "generate video QA datasets", "build CoT reasoning traces from videos", "auto-label videos", or run the video_reasoning_annotation pipeline. Triggers include "video annotation", "video CoT", "video QA", "chain-of-thought", "video captioning pipeline", "video distillation".
What it needs
About 6k tokens when loaded.
What this skill does
Video Reasoning Annotation Pipeline Standalone install? If this session was not initialized by the TAO skill bank plugin, run the tao-setup skill first (host preflight, credentials, cross-skill discovery). Generate Chain-of-Thought training datasets from videos by producing multi-level captions, structured descriptions, and QA pairs (MCQ, binary, open-ended) with step-by-step reasoning traces. Domain-agnostic by default — customize prompts for any video domain. Purpose Transform raw videos into CoT Q&A training data for video understanding models. VLMs (e.g., Gemini, Qwen) act as "teacher" annotators: Steps 0–1 require the model to see the video (VLM calls); Steps 2–3 are text-to-text (cheaper LLM calls). Pipeline architecture Steps are individually selectable via workflow.steps. The pipeline has built-in resume — each step skips already-processed videos, so re-running after a prompt tweak is safe. Initial consultation When the user invokes this skill, walk through these questions in order. Don't skip — getting domain and VLM access right up front prevents wasted runs. 1. Videos Path to the video directory and/or a JSONL with {"videopath": "..."} per line. Confirm format (.mp4 preferred; .avi, .mov, .mkv also walked). 2. Domain — drives prompt selection Ask the user: "What domain are these videos from?" Choose one of the following branches: Domain What to do --- --- general Use the default prompts. Set promptsmodule: "" (or omit). The built-in nvidiataods.autolabel.videoreasoningannotation.prompts covers domain-agnostic content. traffic (CCTV intersections, highways; dashcam excluded) Use the reference module. Set promptsmodule: "nvidiataods.autolabel.videoreasoningannotation.promptstraffic", or copy references/promptstraffic.py into the user's project and tune for their specific camera angles, then point promptsmodule at the copy. warehouse (industrial site CCTV — safety, operations, security) Same pattern. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills NVIDIA/tao-generate-video-reasoning-annotations