Agent skill · NVIDIA
tao-analyze-gaps-vlm-bcq
Extract false-positive and false-negative gaps from VLM binary-classification-question (BCQ, yes/no) predictions.
What it needs
About 2k tokens when loaded.
What this skill does
VLM Binary Classification Gap Analysis Standalone install? If this session was not initialized by the TAO skill bank plugin, run the tao-setup skill first (host preflight, credentials, cross-skill discovery). Reads a VLM predictions JSON, compares each model response against ground truth, and writes FP/FN failure cases to a JSONL file with a summary report. Purpose After running a VLM on a binary yes/no evaluation task, the predictions need to be compared against ground truth to identify failure cases. This skill produces a structured list of FP (false positive) and FN (false negative) samples that downstream RCCA stages (e.g., cosmos generation, root cause analysis) consume to drive a DEFT iteration. Usage Invoke the vlmbcq action inside the TAO Toolkit data services container with Hydra-style key=value overrides: Include videosdir when videoid values in the predictions are relative paths: After the run, surface the FP/FN counts from kpigapsreport.txt and point downstream stages at kpigaps.jsonl. Inputs predictionsjson: Path to predictions JSON file. Must be a JSON array where each item has videoid, response, and gt fields. response and gt are parsed with word-boundary matching — 'yes' or 'no' anywhere in the string is recognized. Samples where both or neither are present are skipped with a warning. videosdir (optional): Base directory for resolving relative videoid paths. If omitted, videoid values are used as absolute paths. Predictions JSON format: Outputs kpigaps.jsonl: One JSON object per line for each FP/FN case. Fields: videoid (absolute path), errortype (FP or FN), question, groundtruth, response. kpigapsreport.txt: Human-readable table with total FP/FN counts. If no gaps are found, no files are written and a message is logged. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills NVIDIA/tao-analyze-gaps-vlm-bcq