Agent skill · software engineering · aradotso
flashkda-delta-attention
High-performance Kimi Delta Attention CUDA kernels built on CUTLASS for efficient recurrent state inference
Why this skill is useful
Adds high-performance CUDA kernels for Kimi Delta Attention that significantly enhance inference capabilities beyond standard implementations.
What it needs
Requires cuda, pytorch installed locally. About 5k tokens when loaded. Last updated 2026-07-09. 70 stars on the source repository.
What this skill does
FlashKDA Delta Attention Skill Skill by ara.so — Daily 2026 Skills collection. FlashKDA provides high-performance CUDA kernels for Kimi Delta Attention (KDA) built on CUTLASS. It targets SM90+ GPUs (H100/H20 class) and integrates as a drop-in backend for flash-linear-attention's chunkkda operation. Requirements GPU: SM90+ (H100, H20, or newer) CUDA 12.9+ PyTorch 2.4+ Python 3.8+ Installation Install the FLA integration (optional but recommended): Core Kernel API flashkda.fwd The primary low-level kernel call: Tensor shapes and dtypes: Parameter Dtype Shape Notes --------------- -------------- -------------------- -------------------------------------------- q bf16 [B, T, H, K] Query; K must be 128 k bf16 [B, T, H, K] Key; K must be 128 v bf16 [B, T, H, V] Value; V must be 128 g bf16 [B, T, H, K] Gate logits (sigmoid/activation applied internally) beta bf16 [B, T, H] Beta logits (sigmoid applied internally) scale float scalar Attention scale factor out bf16 [B, T, H, V] Pre-allocated output tensor Alog fp32 [H] Per-head log-gate parameter dtbias fp32 [H, K] Per-head gate bias lowerbound float scalar Gate lower bound, range [-5.0, 0] initialstate bf16/fp32/None [B, H, V, K] or [N, H, V, K] Optional initial recurrent state finalstate bf16/fp32/None [B, H, V, K] or [N, H, V, K] Optional output final state cuseqlens int64 [N+1] Optional cumulative seq lengths for varlen Constraints: K == V == 128 required When cuseqlens is provided, B must be 1 and T is total tokens across all sequences initialstate and finalstate dtypes must match when both provided Usage via flash-linear-attention Backend (Recommended) FlashKDA auto-dispatches from FLA's chunkkda when installed: Direct Low-Level Kernel Usage Variable-Length (Packed) Batching Use cuseqlens for variable-length sequences packed into a single batch dimension: Stateful Inference (Multi-turn / Streaming) Pass initialstate from a previous call to maintain recurrent state across chunks: Configuration & Environment Variables Va …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/flashkda-delta-attention