Agent skill · research science · aradotso
gemma-tuner-multimodal
Fine-tune Gemma 4 and 3n models with audio, images, and text on Apple Silicon using PyTorch and Metal Performance Shaders.
Why this skill is useful
Adds executable scripts for fine-tuning Gemma models with multimodal data on Apple Silicon, which are not available in public documentation.
What it needs
Requires python, torch, torchaudio installed locally. Requires huggingface account access. About 5k tokens when loaded. Last updated 2026-07-09. 70 stars on the source repository.
What this skill does
Gemma Multimodal Fine-Tuner Skill by ara.so — Daily 2026 Skills collection. Fine-tune Gemma 4 and Gemma 3n models on text, images, and audio data entirely on Apple Silicon (MPS), with support for streaming large datasets from GCS/BigQuery without filling local storage. --- What It Does Text LoRA: instruction-tuning or completion fine-tuning from local CSV Image + Text LoRA: captioning and VQA from local CSV Audio + Text LoRA: the only Apple-Silicon-native path for this modality Cloud streaming: train on terabytes from GCS/BigQuery without local copy MPS-native: no NVIDIA GPU required — runs on MacBook Pro/Air/Mac Studio --- Installation Prerequisites macOS 12.3+ with Apple Silicon (arm64) Python 3.10+ (native arm64, not Rosetta) Hugging Face account with Gemma access Authenticate with Hugging Face --- CLI Commands --- Configuration (config/config.ini) The config is hierarchical INI: defaults → groups → models → datasets → profiles. Use GEMMATUNERCONFIG env var to point to config outside repo root: --- Modality Configuration Text-Only Fine-Tuning Instruction tuning (user/assistant pairs): Completion tuning (full sequence trained): CSV format for instruction tuning (data/datasets/my-text-dataset/train.csv): Image Fine-Tuning CSV format (data/datasets/my-image-dataset/train.csv): Audio Fine-Tuning CSV format (data/datasets/my-audio-dataset/train.csv): --- Supported Models Model Key Hugging Face ID Notes --- --- --- gemma-3n-e2b-it google/gemma-3n-E2B-it Default, ~2B instruct gemma-3n-e4b-it google/gemma-3n-E4B-it ~4B instruct gemma-4-e2b-it google/gemma-4-E2B-it Needs requirements-gemma4.txt gemma-4-e4b-it google/gemma-4-E4B-it Needs requirements-gemma4.txt gemma-4-e2b google/gemma-4-E2B Base, needs Gemma 4 stack gemma-4-e4b google/gemma-4-E4B Base, needs Gemma 4 stack Add custom models with a [model:your-name] section using group = gemma. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/gemma-tuner-multimodal