Agent skill · software engineering · nvidia
tao-finetune-clip
CLIP vision-language model for image-text retrieval, zero-shot classification, embedding extraction, ONNX
Why this skill is useful
Adds executable scripts for training, evaluating, and deploying the CLIP model with specific configurations that are not commonly known or available in public documentation.
What it needs
Requires docker, nvidia-container-toolkit installed locally. About 9k tokens when loaded. Last updated 2026-08-07. 2,816 stars on the source repository.
What this skill does
CLIP Standalone install? Read CLIP standalone setup and verify the bank checkout before running tao-setup or launching a CLIP action. Contrastive Language-Image Pre-training model for zero-shot and fine-tuned image classification, image-text retrieval, and embedding extraction. Fine-tuning adapts CLIP's shared image-text embedding space to domain-specific image-caption data. No default NGC pretrained checkpoint is required for spec construction, but unset checkpoint behavior is action-specific. In the validation-fixes PyTorch image, export.checkpoint: null exports the selected CLIP architecture and may initialize weights when pretrained weights are unavailable. Do not assume inference.checkpoint: null loads pretrained weights: clip inference currently calls the checkpoint loader with None and fails before embedding extraction. For PyTorch inference, checkpoint-backed evaluation/export, resume, and retrain flows, resolve and pass an exact checkpoint from the parent train output. For trusted TAO checkpoints produced by the current run or a known parent job, set TORCHFORCENOWEIGHTSONLYLOAD=1 on checkpoint-dependent PyTorch actions so PyTorch 2.6 can load the Lightning checkpoint metadata; do not set this for untrusted checkpoints. Supported actions: train, evaluate, inference, export, gentrtengine. Train Action Policy This model is AutoML-enabled at the model layer. Before handling any train-stage request, read references/skillinfo.yaml and resolve the run override from either an explicit automlpolicy value or the user's workflow request. Use automlpolicy: on by default and only expose on / off in new launch prompts. Treat phrases like "turn off AutoML", "disable AutoML", "no HPO", or "plain training" as automlpolicy: off for this run only. When automlpolicy: on, automlenabled: true, and both schemas/train.schema.json and references/spectemplatetrain.yaml are packaged, route the train action through tao-skill-bank:tao-run-automl by default with this model's skilldir. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills nvidia/tao-finetune-clip