Agent skill · software engineering · davila7
hqq-quantization
Half-Quadratic Quantization for LLMs without calibration data. Use when quantizing models to 4/3/2-bit precision without needing calibration datasets, for fast quantization workflows, or when deploying with vLLM or HuggingFace Transformers.
Why this skill is useful
Adds a custom quantization library with executable scripts for model compression that significantly enhances AI capabilities in deploying quantized models.
What it needs
Requires hqq, torch installed locally. About 5k tokens when loaded. Last updated 2026-08-06. 30,138 stars on the source repository.
What this skill does
HQQ - Half-Quadratic Quantization Fast, calibration-free weight quantization supporting 8/4/3/2/1-bit precision with multiple optimized backends. When to use HQQ Use HQQ when: Quantizing models without calibration data (no dataset needed) Need fast quantization (minutes vs hours for GPTQ/AWQ) Deploying with vLLM or HuggingFace Transformers Fine-tuning quantized models with LoRA/PEFT Experimenting with extreme quantization (2-bit, 1-bit) Key advantages: No calibration: Quantize any model instantly without sample data Multiple backends: PyTorch, ATEN, TorchAO, Marlin, BitBlas for optimized inference Flexible precision: 8/4/3/2/1-bit with configurable group sizes Framework integration: Native HuggingFace and vLLM support PEFT compatible: Fine-tune quantized models with LoRA Use alternatives instead: AWQ: Need calibration-based accuracy, production serving GPTQ: Maximum accuracy with calibration data available bitsandbytes: Simple 8-bit/4-bit without custom backends llama.cpp/GGUF: CPU inference, Apple Silicon deployment Quick start Installation Basic quantization Quantize full model with HuggingFace Core concepts Quantization configuration HQQ uses BaseQuantizeConfig to define quantization parameters: HQQLinear layer The core quantized layer that replaces nn.Linear: Backends HQQ supports multiple inference backends for different hardware: Backend selection guide: Backend Best For Requirements --------- ---------- -------------- pytorch Compatibility Any GPU pytorchcompile Moderate speedup torch>=2.0 aten Good balance CUDA GPU torchaoint4 4-bit inference torchao installed marlin Maximum 4-bit speed Ampere+ GPU bitblas Flexible bit-widths bitblas installed HuggingFace integration Load pre-quantized models Quantize and save Mixed precision quantization vLLM integration Serve HQQ models with vLLM vLLM with custom HQQ config PEFT/LoRA fine-tuning Fine-tune quantized models QLoRA-style training Quantization workflows Workflow 1: Quick model compression Workflow 2: Optimize f …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills davila7/optimization-hqq