Agent skill · software engineering · pytorch
cuda-index-width
Choose 32-bit vs 64-bit index math in PyTorch CUDA kernels. Use when fixing large-tensor indexing overflows, deciding whether to use int64_t, canUse32BitIndexMath, CUDA_KERNEL_LOOP_TYPE, or AT_DISPATCH_INDEX_TYPES, and when considering binary-size or performance impact of index-type templating.
Why this skill is useful
Provides specific patterns and utilities for managing index types in CUDA kernels, which are not commonly known or documented.
What it needs
About 3k tokens when loaded. Last updated 2026-08-07. 102,253 stars on the source repository.
What this skill does
CUDA Index Width in PyTorch Use this skill when a CUDA kernel overflows int indexing, fails near 2^31 elements, or needs a review of int vs int64t index math. Start with the overflow site Do not blindly convert all index variables to int64t. Find the expression that can exceed 32 bits and classify where it runs: One-time setup per CTA or per output element group: prefer one local 64-bit cast at the first multiply. Hot per-element linear indexing: template the kernel on indext and dispatch int vs int64t. Unsupported algorithm with 32-bit-only assumptions: add a clear TORCHCHECK(canUse32BitIndexMath(...)) instead of silently overflowing. Grid dimension overflow: changing arithmetic type is not enough; add tiling/striding over that dimension. Canonical utilities Include/use the existing PyTorch utilities instead of ad hoc checks: at::native::canUse32BitIndexMath(tensor, INTMAX) checks both numel and maximum storage offset. For CUDA files that already include cuda detail wrappers, at::cuda::detail::canUse32BitIndexMath may be available as an alias. ATDISPATCHINDEXTYPES(cond ? ScalarType::Int : ScalarType::Long, "name", [&] { ... }); provides indext. CUDAKERNELLOOPTYPE(index, nthreads, indext) keeps grid-stride loops correct for either width. Check every tensor whose offsets are computed with the selected index type, not just the output tensor. Fix patterns 1. Localized base-offset overflow If only a base pointer offset can overflow and it is computed outside the hot loop, keep the kernel otherwise unchanged: This avoids doubling kernel instantiations and keeps inner-loop arithmetic 32-bit. Use this when dimensions inside the tile still fit in int. 2. Grid-stride linear kernels If the loop index, modulo/division decomposition, or final data[index] access can exceed 32 bits, template the kernel: Prefer this over unconditionally changing the loop index to int64t, because 64-bit division/modulo in a hot CUDA loop can be measurable. 3. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills pytorch/cuda-index-width