Agent skill · software engineering · davila7
optimizing-attention-flash
Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Use when training/running transformers with long sequences (>512 tokens), encountering GPU memory issues with attention, or need faster inference. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.
Why this skill is useful
Adds executable scripts and specific integration patterns for optimizing transformer attention using Flash Attention, which the AI wouldn't reliably generate on its own.
What it needs
Requires flash-attn, torch, transformers installed locally. About 5k tokens when loaded. Last updated 2026-08-06. 30,138 stars on the source repository.
What this skill does
Flash Attention - Fast Memory-Efficient Attention Quick start Flash Attention provides 2-4x speedup and 10-20x memory reduction for transformer attention through IO-aware tiling and recomputation. PyTorch native (easiest, PyTorch 2.2+): flash-attn library (more features): Common workflows Workflow 1: Enable in existing PyTorch model Copy this checklist: Step 1: Check PyTorch version If <2.2, upgrade: Step 2: Enable Flash Attention backend Replace standard attention: Force Flash Attention backend: Step 3: Verify speedup with profiling Expected: 2-4x speedup for sequences >512 tokens. Step 4: Test accuracy matches baseline Workflow 2: Use flash-attn library for advanced features For multi-query attention, sliding window, or H100 FP8. Copy this checklist: Step 1: Install flash-attn library Step 2: Modify attention code Step 3: Enable advanced features Multi-query attention (shared K/V across heads): Sliding window attention (local attention): Step 4: Benchmark performance Workflow 3: H100 FP8 optimization (FlashAttention-3) For maximum performance on H100 GPUs. Step 1: Verify H100 GPU Step 2: Install flash-attn with FP8 support Step 3: Convert inputs to FP8 Step 4: Run with FP8 attention When to use vs alternatives Use Flash Attention when: Training transformers with sequences >512 tokens Running inference with long context (>2K tokens) GPU memory constrained (OOM with standard attention) Need 2-4x speedup without accuracy loss Using PyTorch 2.2+ or can install flash-attn Use alternatives instead: Standard attention: Sequences <256 tokens (overhead not worth it) xFormers: Need more attention variants (not just speed) Memory-efficient attention: CPU inference (Flash Attention needs GPU) Common issues Issue: ImportError: cannot import flashattn Install with no-build-isolation flag: Or install CUDA toolkit first: Issue: Slower than expected (no speedup) Flash Attention benefits increase with sequence length: <512 tokens: Minimal speedup (10-20%) 512-2K tokens: 2-3x speedu …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills davila7/optimization-flash-attention