Agent skill · research science · davila7
transformer-lens-interpretability
Provides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation caching. Use when reverse-engineering model algorithms, studying attention patterns, or performing activation patching experiments.
Why this skill is useful
Provides domain-specific knowledge on mechanistic interpretability techniques and activation caching that the AI wouldn't reliably generate on its own.
What it needs
Requires torch, transformer-lens installed locally. About 6k tokens when loaded. Last updated 2026-08-06. 30,138 stars on the source repository.
What this skill does
TransformerLens: Mechanistic Interpretability for Transformers TransformerLens is the de facto standard library for mechanistic interpretability research on GPT-style language models. Created by Neel Nanda and maintained by Bryce Meyer, it provides clean interfaces to inspect and manipulate model internals via HookPoints on every activation. GitHub: TransformerLensOrg/TransformerLens (2,900+ stars) When to Use TransformerLens Use TransformerLens when you need to: Reverse-engineer algorithms learned during training Perform activation patching / causal tracing experiments Study attention patterns and information flow Analyze circuits (e.g., induction heads, IOI circuit) Cache and inspect intermediate activations Apply direct logit attribution Consider alternatives when: You need to work with non-transformer architectures → Use nnsight or pyvene You want to train/analyze Sparse Autoencoders → Use SAELens You need remote execution on massive models → Use nnsight with NDIF You want higher-level causal intervention abstractions → Use pyvene Installation For development version: Core Concepts HookedTransformer The main class that wraps transformer models with HookPoints on every activation: Supported Models (50+) Family Models -------- -------- GPT-2 gpt2, gpt2-medium, gpt2-large, gpt2-xl LLaMA llama-7b, llama-13b, llama-2-7b, llama-2-13b EleutherAI pythia-70m to pythia-12b, gpt-neo, gpt-j-6b Mistral mistral-7b, mixtral-8x7b Others phi, qwen, opt, gemma Activation Caching Run the model and cache all intermediate activations: ActivationCache Keys Key Pattern Shape Description ------------- ------- ------------- residpre, layer [batch, pos, dmodel] Residual before attention residmid, layer [batch, pos, dmodel] Residual after attention residpost, layer [batch, pos, dmodel] Residual after MLP attnout, layer [batch, pos, dmodel] Attention output mlpout, layer [batch, pos, dmodel] MLP output pattern, layer [batch, head, qpos, kpos] Attention pattern (post-softmax) q, layer [batc …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills davila7/mechanistic-interpretability-transformer-lens