Agent skill · research science · davila7
llava
Large Language and Vision Assistant. Enables visual instruction tuning and image-based conversations. Combines CLIP vision encoder with Vicuna/LLaMA language models. Supports multi-turn image chat, visual question answering, and instruction following. Use for vision-language chatbots or image understanding tasks. Best for conversational image analysis.
Why this skill is useful
Provides executable scripts for multi-turn image conversations and visual question answering that extend the AI's capabilities in image understanding.
What it needs
Requires pillow, torch, transformers installed locally. About 4k tokens when loaded. Last updated 2026-08-06. 30,138 stars on the source repository.
What this skill does
LLaVA - Large Language and Vision Assistant Open-source vision-language model for conversational image understanding. When to use LLaVA Use when: Building vision-language chatbots Visual question answering (VQA) Image description and captioning Multi-turn image conversations Visual instruction following Document understanding with images Metrics: 23,000+ GitHub stars GPT-4V level capabilities (targeted) Apache 2.0 License Multiple model sizes (7B-34B params) Use alternatives instead: GPT-4V: Highest quality, API-based CLIP: Simple zero-shot classification BLIP-2: Better for captioning only Flamingo: Research, not open-source Quick start Installation Basic usage Available models Model Parameters VRAM Quality ------- ------------ ------ --------- LLaVA-v1.5-7B 7B ~14 GB Good LLaVA-v1.5-13B 13B ~28 GB Better LLaVA-v1.6-34B 34B ~70 GB Best CLI usage Web UI (Gradio) Multi-turn conversations Common tasks Image captioning Visual question answering Object detection (textual) Scene understanding Document understanding Training custom model Quantization (reduce VRAM) Best practices 1. Start with 7B model - Good quality, manageable VRAM 2. Use 4-bit quantization - Reduces VRAM significantly 3. GPU required - CPU inference extremely slow 4. Clear prompts - Specific questions get better answers 5. Multi-turn conversations - Maintain conversation context 6. Temperature 0.2-0.7 - Balance creativity/consistency 7. maxnewtokens 512-1024 - For detailed responses 8. Batch processing - Process multiple images sequentially Performance Model VRAM (FP16) VRAM (4-bit) Speed (tokens/s) ------- ------------- -------------- ------------------ 7B ~14 GB ~4 GB ~20 13B ~28 GB ~8 GB ~12 34B ~70 GB ~18 GB ~5 On A100 GPU Benchmarks LLaVA achieves competitive scores on: VQAv2: 78.5% GQA: 62.0% MM-Vet: 35.4% MMBench: 64.3% Limitations 1. Hallucinations - May describe things not in image 2. Spatial reasoning - Struggles with precise locations 3. Small text - Difficulty reading fine print 4. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills davila7/multimodal-llava