Agent skill · software engineering · nousresearch
peft
Fine-tune large LLMs with LoRA on limited GPU memory.
Why this skill is useful
Adds executable scripts for fine-tuning large LLMs with LoRA and QLoRA, which are not commonly found in public documentation.
What it needs
Requires bitsandbytes, peft, torch, transformers installed locally. About 6k tokens when loaded. Last updated 2026-08-07. 226,679 stars on the source repository.
What this skill does
PEFT (Parameter-Efficient Fine-Tuning) Fine-tune LLMs by training <1% of parameters using LoRA, QLoRA, and 25+ adapter methods. When to use PEFT Use PEFT/LoRA when: Fine-tuning 7B-70B models on consumer GPUs (RTX 4090, A100) Need to train <1% parameters (6MB adapters vs 14GB full model) Want fast iteration with multiple task-specific adapters Deploying multiple fine-tuned variants from one base model Use QLoRA (PEFT + quantization) when: Fine-tuning 70B models on single 24GB GPU Memory is the primary constraint Can accept ~5% quality trade-off vs full fine-tuning Use full fine-tuning instead when: Training small models (<1B parameters) Need maximum quality and have compute budget Significant domain shift requires updating all weights Quick start Installation LoRA fine-tuning (standard) QLoRA fine-tuning (memory-efficient) LoRA parameter selection Rank (r) - capacity vs efficiency Rank Trainable Params Memory Quality Use Case ------ ----------------- -------- --------- ---------- 4 ~3M Minimal Lower Simple tasks, prototyping 8 ~7M Low Good Recommended starting point 16 ~14M Medium Better General fine-tuning 32 ~27M Higher High Complex tasks 64 ~54M High Highest Domain adaptation, 70B models Alpha (loraalpha) - scaling factor Target modules by architecture Loading and merging adapters Load trained adapter Merge adapter into base model Multi-adapter serving PEFT methods comparison Method Trainable % Memory Speed Best For -------- ------------ -------- ------- ---------- LoRA 0.1-1% Low Fast General fine-tuning QLoRA 0.1-1% Very Low Medium Memory-constrained AdaLoRA 0.1-1% Low Medium Automatic rank selection IA3 0.01% Minimal Fastest Few-shot adaptation Prefix Tuning 0.1% Low Medium Generation control Prompt Tuning 0.001% Minimal Fast Simple task adaptation P-Tuning v2 0.1% Low Medium NLU tasks IA3 (minimal parameters) Prefix Tuning Integration patterns With TRL (SFTTrainer) With Axolotl (YAML config) With vLLM (inference) Performance benchmarks Memory usage (Llama 3.1 …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills nousresearch/peft