Agent skill · writing content · yonatangross
multimodal-llm
Vision, audio, video generation, and multimodal LLM integration patterns. Use when processing images, transcribing audio, generating speech, generating AI video (Kling v3, Sora 2, Veo 3.1 std/lite/fast, Runway Gen-4.5 via `gen4_turbo`), or building multimodal AI pipelines.
Why this skill is useful
Adds integration patterns and executable workflows for multimodal AI capabilities that the AI cannot generate on its own.
What it needs
About 5k tokens when loaded. Last updated 2026-08-06. 215 stars on the source repository.
What this skill does
Multimodal LLM Patterns Integrate vision, audio, and video generation capabilities from leading multimodal models. Covers image analysis, document understanding, real-time voice agents, speech-to-text, text-to-speech, and AI video generation (Kling v3, Sora 2, Veo 3.1 std/lite/fast tiers, Runway Gen-4.5 via gen4turbo). Canonical model IDs (Google IDs checked against ai.google.dev/gemini-api/docs/models on 2026-09-23): Provider Model IDs ---------- ----------- Anthropic claude-opus-5-5 (recommended, the default Opus since CC 2.1.280, same 2,576 px budget as Opus 5), claude-opus-5, claude-opus-4-8, claude-opus-4-7, claude-opus-4-6, claude-sonnet-5-5, claude-sonnet-4-6, claude-haiku-4-5-20251001. claude-fable-5 is Anthropic's frontier tier above Opus (GA 2026-07). Premium cost, never auto-pin it; the fable-spend-consent gate requires explicit user consent before any Fable spend OpenAI gpt-5.5 (current flagship) Google gemini-3.1-pro-preview (flagship), gemini-3.1-flash-lite (cost) Veo veo-3.1-generate-preview / veo-3.1-lite-generate-preview / veo-3.1-fast-generate-preview Kling kling-v3 (modelname field in Kling API) Runway gen4turbo (product label: Gen-4.5) Quick Reference Category Rules Impact When to Use ---------- ------- -------- ------------- Vision: Image Analysis 1 HIGH Image captioning, VQA, multi-image comparison, object detection Vision: Document Understanding 1 HIGH OCR, chart/diagram analysis, PDF processing, table extraction Vision: Model Selection 1 MEDIUM Choosing provider, cost optimization, image size limits Audio: Speech-to-Text 1 HIGH Transcription, speaker diarization, long-form audio Audio: Text-to-Speech 1 MEDIUM Voice synthesis, expressive TTS, multi-speaker dialogue Audio: Model Selection 1 MEDIUM Real-time voice agents, provider comparison, pricing Video: Model Selection 1 HIGH Choosing video gen provider (Kling, Sora, Veo, Runway) Video: API Patterns 1 HIGH Async task polling, SDK integration, webhook callbacks Video: Multi-Shot 1 HIGH Storyb …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills yonatangross/multimodal-llm