Agent skill · software engineering · wshobson
quantized-export
Export a promoted fine-tuned model in the right deployment format — merged safetensors, LoRA-only, GGUF with imatrix, or FP8. Use after a checkpoint passes promotion, when choosing a quantization format for a target device, or when an exported model fails its smoke test.
Why this skill is useful
Provides specific quantization formats and deployment strategies for fine-tuned models that the AI wouldn't reliably generate on its own.
What it needs
About 5k tokens when loaded. Last updated 2026-08-05. 38,570 stars on the source repository.
What this skill does
Quantized Export The last stop after checkpoint-promotion hands off a PROMOTE verdict: a checkpoint that cleared the four-stage gate still isn't deployed until it's exported in the right format for its target runtime and proven to still work post-export. A REJECT verdict never reaches this skill — export starts only from a promoted checkpoint. Input: a promoted checkpoint (or LoRA adapter) plus the target deployment surface — GPU class, serving stack, and whether long-context/code/math workloads are in scope. Output format: an exported artifact in the chosen format plus a smoke-test diff report comparing 3–5 golden outputs pre-export and post-export. Format Map Pick format by hardware and deployment shape, not by habit — the wrong pick either wastes throughput headroom or breaks silently on specific workloads (see Workload Overrides). FP8 is the default on Hopper-class GPUs and newer. It preserves near-bf16 quality at roughly half the memory, and it's the safe first choice whenever the target GPU supports it and no edge-device constraint applies. AWQ INT4 targets older GPUs that predate FP8 hardware support. GPTQ is superseded for new deployments — don't reach for it on a fresh export; AWQ has better accuracy retention at the same bit width and wider current tooling support. GGUF with Q4KM quantization, built from an imatrix, is the edge/llama.cpp format. Use it for local or CPU-adjacent deployment, not for GPU-serving throughput — it optimizes for footprint, not tokens/sec on a datacenter GPU. NVFP4 is for Blackwell-at-scale deployments only — and explicitly NOT on GB10. NVFP4 on SM121 (GB10) runs ~32% slower than FP8 because the hardware lacks a native cvt.e2m1x2 path unless the kernel is compiled sm121a. Choosing NVFP4 on a GB10 target is a regression, not an upgrade — pick FP8 there instead. Merged vs. LoRA-only is a separate axis from quant format. A merged export folds the adapter into the base weights: larger artifact, no base-model dependency at serve time. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills wshobson/quantized-export