Agent skill · data analytics · nvidia
tao-analyze-gaps-vlm-bcq
Extract false-positive and false-negative gaps from VLM binary-classification-question (BCQ, yes/no) predictions.
Why this skill is useful
Provides specific commands and structured output formats for analyzing VLM binary classification gaps that the AI wouldn't reliably generate on its own.
What it needs
Requires docker, nvidia-container-toolkit installed locally. About 3k tokens when loaded. Last updated 2026-08-07. 2,816 stars on the source repository.
What this skill does
VLM Binary Classification Gap Analysis Standalone install? If this session was not initialized by the TAO skill bank plugin, run the tao-setup skill first (host preflight, credentials, cross-skill discovery). Reads a VLM predictions JSON, compares each model response against ground truth, and writes FP/FN failure cases to a JSONL file with a summary report. Run it with a TAO Data Services spec file; the data-services entrypoint requires -e <spec>. Purpose After running a VLM on a binary yes/no evaluation task, the predictions need to be compared against ground truth to identify failure cases. This skill produces a structured list of FP (false positive) and FN (false negative) samples that downstream RCCA stages (e.g., cosmos generation, root cause analysis) consume to drive a DEFT iteration. Usage Generate a vlmbcqspec.yaml with the bundled helper: Omit --videos-dir when prediction videoid values are already absolute. The generated spec has this shape: Set videosdir when videoid values in the predictions are relative paths: Invoke the vlmbcq action inside the TAO Toolkit data services container with -e <spec>: Request exactly one GPU from the selected platform (computeshape.gpus: 1, computeshape.nodes: 1). VLM BCQ gap analysis does not perform GPU compute, but the Data Services image always calls nvidia-smi and fails when no GPU is visible. One is a GPU count, not a device ID; the platform selects the device. After the run, surface the FP/FN counts from kpigapsreport.txt and point downstream stages at kpigaps.jsonl. Inputs config spec: YAML file passed with -e. Template: assets/defaultvlmbcq.yaml. predictionsjson: Path to predictions JSON file. Must be a JSON array where each item has videoid, response, and gt fields. response and gt are parsed with word-boundary matching — 'yes' or 'no' anywhere in the string is recognized. Samples where both or neither are present are skipped with a warning. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills nvidia/tao-analyze-gaps-vlm-bcq