Agent skill · software engineering · nousresearch
flash-attention
Speed up long-sequence transformer training and inference.
Why this skill is useful
Adds executable scripts and detailed workflows for integrating Flash Attention into PyTorch models, significantly enhancing performance for long-sequence transformers.
What it needs
Requires flash-attn, torch, transformers installed locally. About 5k tokens when loaded. Last updated 2026-08-07. 226,679 stars on the source repository.
What this skill does
Flash Attention - Fast Memory-Efficient Attention Quick start Flash Attention provides 2-4x speedup and 10-20x memory reduction for transformer attention through IO-aware tiling and recomputation. PyTorch native (easiest, PyTorch 2.2+): flash-attn library (more features): Common workflows Workflow 1: Enable in existing PyTorch model Copy this checklist: Step 1: Check PyTorch version If <2.2, upgrade: Step 2: Enable Flash Attention backend Replace standard attention: Force Flash Attention backend (torch.backends.cuda.sdpkernel is deprecated; use torch.nn.attention.sdpakernel with SDPBackend): Step 3: Verify speedup with profiling Expected: 2-4x speedup for sequences >512 tokens. Step 4: Test accuracy matches baseline Workflow 2: Use flash-attn library for advanced features For multi-query attention, sliding window, or H100 FP8. Copy this checklist: Step 1: Install flash-attn library Step 2: Modify attention code Step 3: Enable advanced features Multi-query attention (shared K/V across heads): Sliding window attention (local attention): Step 4: Benchmark performance Workflow 3: H100 FP8 optimization (FlashAttention-3) For maximum performance on Hopper GPUs (H100). Important: The pip package flash-attn (2.8.x) ships FlashAttention-2 only — it does not contain FA3 or FP8 H100 kernels, and flashattnfunc does not auto-use FP8. FlashAttention-3 is a separate beta build compiled from source from the repo's hopper/ directory, exposed via the flashattninterface module. FA3 supports FP16/BF16 forward+backward and FP8 forward only. Step 1: Verify H100 GPU Step 2: Build & install FlashAttention-3 from source FA3 is NOT included in pip install flash-attn. Build it from the hopper/ subdirectory: Step 3: Use the FA3 interface (FP8 forward) FA3 exposes its own module flashattninterface (distinct from the FA2 flashattn). …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills nousresearch/flash-attention