Agent skill · research science · davila7
model-merging
Merge multiple fine-tuned models using mergekit to combine capabilities without retraining. Use when creating specialized models by blending domain-specific expertise (math + coding + chat), improving performance beyond single models, or experimenting rapidly with model variants. Covers SLERP, TIES-Merging, DARE, Task Arithmetic, linear merging, and production deployment strategies.
Why this skill is useful
Adds executable scripts and detailed methods for merging models without retraining, which are not commonly found in public documentation.
What it needs
Requires mergekit, torch, transformers installed locally. About 6k tokens when loaded. Last updated 2026-08-06. 30,138 stars on the source repository.
What this skill does
Model Merging: Combining Pre-trained Models When to Use This Skill Use Model Merging when you need to: Combine capabilities from multiple fine-tuned models without retraining Create specialized models by blending domain-specific expertise (math + coding + chat) Improve performance beyond single models (often +5-10% on benchmarks) Reduce training costs - no GPUs needed, merges run on CPU Experiment rapidly - create new model variants in minutes, not days Preserve multiple skills - merge without catastrophic forgetting Success Stories: Marcoro14-7B-slerp (best on Open LLM Leaderboard 02/2024), many top HuggingFace models use merging Tools: mergekit (Arcee AI), LazyMergekit, Model Soup Installation Quick Start Simple Linear Merge SLERP Merge (Best for 2 Models) Core Concepts 1. Merge Methods Linear (Model Soup) Simple weighted average of parameters Fast, works well for similar models Can merge 2+ models SLERP (Spherical Linear Interpolation) Interpolates along sphere in weight space Preserves magnitude of weight vectors Best for merging 2 models Smoother than linear Task Arithmetic Extract "task vectors" (fine-tuned - base) Combine task vectors, add to base Good for merging multiple specialized models TIES-Merging Task arithmetic + sparsification Resolves sign conflicts in parameters Best for merging many task-specific models DARE (Drop And REscale) Randomly drops fine-tuned parameters Rescales remaining parameters Reduces redundancy, maintains performance 2. Configuration Structure Merge Methods Guide Linear Merge Best for: Simple model combinations, equal weighting SLERP Merge Best for: Two models, smooth interpolation Layer-specific SLERP: Task Arithmetic Best for: Combining specialized skills TIES-Merging Best for: Many models, resolving conflicts DARE Merge Best for: Reducing redundancy Advanced Patterns Layer-wise Merging MoE from Merged Models Tokenizer Merging Best Practices 1. Model Compatibility 2. Weight Selection 3. Method Selection 4. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills davila7/emerging-techniques-model-merging