Agent skill · software engineering · zechenzhangagi
implementing-llms-litgpt
Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.
Why this skill is useful
Adds executable scripts for fine-tuning LLMs with LoRA and provides clean implementations of multiple pretrained models that aren't widely available.
What it needs
Requires litgpt, torch, transformers installed locally. About 6k tokens when loaded. Last updated 2026-06-16. 11,472 stars on the source repository.
What this skill does
LitGPT - Clean LLM Implementations Quick start LitGPT provides 20+ pretrained LLM implementations with clean, readable code and production-ready training workflows. Installation: Load and use any model: List available models: Common workflows Workflow 1: Fine-tune on custom dataset Copy this checklist: Step 1: Download pretrained model Models are saved to checkpoints/ directory. Step 2: Prepare dataset LitGPT supports multiple formats: Alpaca format (instruction-response): Save as data/mydataset.json. Step 3: Configure training Step 4: Run fine-tuning Training saves checkpoints to out/finetune/ automatically. Monitor training: Workflow 2: LoRA fine-tuning on single GPU Most memory-efficient option. Step 1: Choose base model For limited GPU memory (12-16GB): Phi-2 (2.7B) - Best quality/size tradeoff Llama 3 1B - Smallest, fastest Gemma 2B - Good reasoning Step 2: Configure LoRA parameters LoRA rank guide: r=8: Lightweight, 2-4MB adapters r=16: Standard, good quality r=32: High capacity, use for complex tasks r=64: Maximum quality, 4× larger adapters Step 3: Train with LoRA Step 4: Merge LoRA weights (optional) Merge LoRA adapters into base model for deployment: Now use merged model: Workflow 3: Pretrain from scratch Train new model on your domain data. Step 1: Prepare pretraining dataset LitGPT expects tokenized data. Use preparedataset.py: Step 2: Configure model architecture Edit config file or use existing: Step 3: Set up multi-GPU training Step 4: Launch pretraining For large-scale pretraining on cluster: Workflow 4: Convert and deploy model Export LitGPT models for production. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills zechenzhangagi/litgpt