Agent skill · research science · erichowens
clip-aware-embeddings
Semantic image-text matching with CLIP and alternatives. Use for image search, zero-shot classification, similarity matching. NOT for counting objects, fine-grained classification (celebrities,
Why this skill is useful
Provides specific commands and patterns for using CLIP effectively in semantic image-text matching, which the AI may not generate reliably on its own.
What it needs
Requires pillow, sentence-transformers, torch, transformers installed locally. About 5k tokens when loaded. Last updated 2026-07-14. 180 stars on the source repository.
What this skill does
CLIP-Aware Image Embeddings Smart image-text matching that knows when CLIP works and when to use alternatives. MCP Integrations MCP Purpose ----- --------- Firecrawl Research latest CLIP alternatives and benchmarks Hugging Face (if configured) Access model cards and documentation Quick Decision Tree When to Use This Skill ✅ Use for: Semantic image search Broad category classification Image similarity matching Zero-shot tasks on new categories ❌ Do NOT use for: Counting objects in images Fine-grained classification Spatial understanding Attribute binding Negation handling Installation Validation: Run python scripts/validatesetup.py Basic Usage Image Search Common Anti-Patterns Anti-Pattern 1: "CLIP for Everything" ❌ Wrong: Why wrong: CLIP's architecture collapses spatial information into a single vector. It literally cannot count. ✓ Right: How to detect: If query contains "how many", "count", or numeric questions → Use object detection --- Anti-Pattern 2: Fine-Grained Classification ❌ Wrong: Why wrong: CLIP trained on coarse categories. Fine-grained faces, car models, flower species require specialized models. ✓ Right: How to detect: If query asks to distinguish between similar items in same category → Use specialized model --- Anti-Pattern 3: Spatial Understanding ❌ Wrong: Why wrong: CLIP embeddings lose spatial topology. "Left" and "right" are treated as bag-of-words. ✓ Right: How to detect: If query contains directional words (left, right, above, under, next to) → Use spatial model --- Anti-Pattern 4: Attribute Binding ❌ Wrong: Why wrong: CLIP cannot bind attributes to objects. It sees "red, blue, car, truck" as a bag of concepts. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills erichowens/clip-aware-embeddings