Agent skill · software engineering · aradotso
dflash-mlx-speculative-decoding
Lossless DFlash speculative decoding for MLX on Apple Silicon — 1.7–4x faster LLM inference using block diffusion drafting with target model verification.
Why this skill is useful
Adds executable scripts for lossless speculative decoding and benchmarking that enhance LLM inference capabilities on Apple Silicon.
What it needs
Requires pip, pipx installed locally. About 4k tokens when loaded. Last updated 2026-07-09. 70 stars on the source repository.
What this skill does
dflash-mlx Speculative Decoding Skill by ara.so — Daily 2026 Skills collection. DFlash implements lossless speculative decoding for MLX on Apple Silicon. A small draft model (~1B params) generates 16 tokens in parallel using block diffusion; the target model verifies all 16 in a single forward pass. Tokens are only emitted after target verification — output is lossless (every token is the target model's greedy argmax). Typical speedups: 1.7x–4.1x over baseline mlxlm depending on model size and context length. Acceptance rates hover around 87–90% for Qwen3.5 models. Installation Requires Python 3.10+, MLX 0.31.1+, Apple Silicon Mac. Key CLI Commands Generate text OpenAI-compatible server Benchmark Outputs per-run JSON reports with tok/s, acceptance rate, and speedup vs baseline. Supported Model Pairs Target Model Draft Model --- --- Qwen/Qwen3.5-4B z-lab/Qwen3.5-4B-DFlash Qwen/Qwen3.5-9B z-lab/Qwen3.5-9B-DFlash mlx-community/Qwen3.5-27B-4bit z-lab/Qwen3.5-27B-DFlash mlx-community/Qwen3.5-35B-A3B-4bit z-lab/Qwen3.5-35B-A3B-DFlash Draft models are auto-resolved from a registry — no --draft flag needed for listed pairs. Models without a matching draft are rejected at startup. Python API Usage Streaming generation Full generation with stats Custom draft block size and context OpenAI client against dflash-serve Tool calling (via dflash-serve) Common Patterns Side-by-side demo (baseline vs DFlash) Integrating with Open WebUI 1. Start dflash-serve --model Qwen/Qwen3.5-9B --port 8000 2. In Open WebUI settings → Connections → add OpenAI API with URL http://localhost:8000/v1 3. Select model Qwen/Qwen3.5-9B in the chat UI Works the same for Continue, aider, OpenCode, and any OpenAI-compatible client. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/dflash-mlx-speculative-decoding