Agent skill · software engineering · aradotso
parlor-on-device-ai
On-device, real-time multimodal AI voice and vision assistant powered by Gemma 4 E2B and Kokoro TTS, running entirely locally via FastAPI WebSocket server.
Why this skill is useful
Adds a complete executable FastAPI WebSocket server setup for a real-time multimodal AI assistant, including specific model configurations and platform-aware TTS integration.
What it needs
Requires python, uv installed locally. About 5k tokens when loaded. Last updated 2026-07-09. 70 stars on the source repository.
What this skill does
Parlor On-Device AI Skill by ara.so — Daily 2026 Skills collection. Parlor is a real-time, on-device multimodal AI assistant. It combines Gemma 4 E2B (via LiteRT-LM) for speech and vision understanding with Kokoro TTS for voice output. Everything runs locally — no API keys, no cloud calls, no cost per request. Architecture Key features: Silero VAD in browser — hands-free, no push-to-talk Barge-in — interrupt AI mid-sentence by speaking Sentence-level TTS streaming — audio starts before full response is ready Platform-aware TTS — MLX backend on Apple Silicon, ONNX on Linux Requirements Python 3.12+ macOS with Apple Silicon or Linux with a supported GPU ~3 GB free RAM uv package manager Installation Open http://localhost:8000, grant camera and microphone permissions, and start talking. Models download automatically on first run (~2.6 GB for Gemma 4 E2B, plus TTS models). Configuration Set environment variables before running: Variable Default Description -------------- ------------------------------- ------------------------------------------------ MODELPATH auto-download from HuggingFace Path to local .litertlm model file PORT 8000 Server port Project Structure Key Components server.py — FastAPI WebSocket Server The server handles two WebSocket connections: one for receiving audio/video from the browser, one for streaming audio back. tts.py — Platform-Aware TTS Kokoro TTS selects backend based on platform: Gemma 4 E2B Inference via LiteRT-LM Running Benchmarks Performance Reference (Apple M3 Pro) Stage Time ---------------------------------- --------------- Speech + vision understanding ~1.8–2.2s Response generation (~25 tokens) ~0.3s Text-to-speech (1–3 sentences) ~0.3–0.7s Total end-to-end ~2.5–3.0s Decode speed: ~83 tokens/sec on GPU. Common Patterns Extending the System Prompt Modify the prompt in server.py to change the AI's persona or task: Adding a New Language for TTS Kokoro supports multiple language codes. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/parlor-on-device-ai