Agent skill · software engineering · davila7
implementing-llms-litgpt
Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.
Why this skill is useful
Adds executable scripts for fine-tuning LLMs with LoRA and provides domain-specific knowledge on model architectures that the AI wouldn't generate on its own.
What it needs
Requires litgpt, torch, transformers installed locally. About 6k tokens when loaded. Last updated 2026-08-06. 30,138 stars on the source repository.
What this skill does
LitGPT - Clean LLM Implementations Quick start LitGPT provides 20+ pretrained LLM implementations with clean, readable code and production-ready training workflows. Installation: Load and use any model: List available models: Common workflows Workflow 1: Fine-tune on custom dataset Copy this checklist: Step 1: Download pretrained model Models are saved to checkpoints/ directory. Step 2: Prepare dataset LitGPT supports multiple formats: Alpaca format (instruction-response): Save as data/mydataset.json. Step 3: Configure training Step 4: Run fine-tuning Training saves checkpoints to out/finetune/ automatically. Monitor training: Workflow 2: LoRA fine-tuning on single GPU Most memory-efficient option. Step 1: Choose base model For limited GPU memory (12-16GB): Phi-2 (2.7B) - Best quality/size tradeoff Llama 3 1B - Smallest, fastest Gemma 2B - Good reasoning Step 2: Configure LoRA parameters LoRA rank guide: r=8: Lightweight, 2-4MB adapters r=16: Standard, good quality r=32: High capacity, use for complex tasks r=64: Maximum quality, 4× larger adapters Step 3: Train with LoRA Step 4: Merge LoRA weights (optional) Merge LoRA adapters into base model for deployment: Now use merged model: Workflow 3: Pretrain from scratch Train new model on your domain data. Step 1: Prepare pretraining dataset LitGPT expects tokenized data. Use preparedataset.py: Step 2: Configure model architecture Edit config file or use existing: Step 3: Set up multi-GPU training Step 4: Launch pretraining For large-scale pretraining on cluster: Workflow 4: Convert and deploy model Export LitGPT models for production. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills davila7/model-architecture-litgpt