Claude Skills by tensormux

Guide the agent through identifying, classifying, and restructuring warp divergence in CUDA kernels — distinguishing avoidable from unavoidable divergence, applying correct restructuring strategies, and assessing the real performance impact before spending engineering effort.

Guide the agent through selecting the correct and efficient thread block dimensions and grid dimensions for a CUDA kernel, covering occupancy analysis, register and shared memory constraints, tail effects, persistent kernels, and when to use `cudaOccupancyMaxActiveBlocksPerMultiprocessor` as a decision tool.

Guide the agent through a systematic process of isolating, reproducing, and diagnosing correctness errors in CUDA kernels — covering indexing bugs, layout mismatches, synchronization races, reduction errors, numerical drift, and out-of-bounds memory access.

Guide the agent through diagnosing and restructuring CUDA global memory access patterns to maximize effective memory bandwidth, covering coalescing requirements, vectorized loads, AoS vs SoA layouts, shared memory staging for non-coalesced patterns, and L2 cache behavior.

Guide the agent through designing and tuning shared memory tiling strategies for CUDA kernels, covering bank conflict analysis and elimination, tile shape selection, double buffering with async copy, occupancy tradeoffs from shared memory allocation, and the decision of when smem tiling is worth the complexity.

Guide the agent through designing and implementing a correct, performance-aware CUDA GEMM kernel (C = alpha * A * B + beta * C) for a specific problem configuration, including decisions about tiling strategy, memory hierarchy usage, tensor core eligibility, and when to defer to cuBLAS or CUTLASS instead.

Guide the agent through designing and implementing a correct, efficient CUDA LayerNorm (and RMSNorm) kernel, covering mean/variance computation strategies, Welford online accumulation, epsilon placement, affine transform application, backward pass structure, and decomposition for non-power-of-two hidden dimensions.

Guide the agent through designing and implementing a correct, efficient CUDA reduction kernel for a given operator (sum, max, min, or custom binary associative op), covering warp-level primitives, block-level reduction, multi-block strategies, and when to use CUB instead.

Guide the agent through designing and implementing a correct, numerically stable CUDA softmax kernel, covering online (single-pass) computation, row-parallel decomposition, warp-level reductions, fp16/bf16 precision pitfalls, masked softmax variants, and when to fuse with attention versus implementing standalone.

Guide the agent through choosing and tuning kernels for the prefill phase versus the decode phase of LLM inference. The two phases have fundamentally different arithmetic intensity, occupy different sides of the roofline, and respond to different optimizations. Continuous batching and speculative decoding shift the balance and must be reasoned about explicitly.

Guide the agent through planning how a custom CUDA kernel will be wrapped as a TensorRT plugin so it can be invoked from inside a TensorRT engine — covering API choice (IPluginV3 vs IPluginV2DynamicExt), the plugin lifecycle, dynamic shape handling, serialization, mixed precision (FP16/INT8/FP8), workspace management, CUDA graph compatibility, and the C++/Python binding strategy. The output is an integration plan with explicit decisions, not the plugin source code itself.

Guide the agent through implementing a Triton kernel that unpacks and dequantizes a quantized weight tensor (int4 or int8) into fp16 or bf16. This is the standalone building block underneath W4A16 / W8A16 schemes (AWQ, GPTQ, SqueezeLLM, bitsandbytes NF4, int8 per-channel). Covers bit-unpacking, per-group scale/zero arithmetic, NF4 codebook lookup, and — critically — when *not* to write a standalone dequant kernel because the operation should be fused into the matmul instead.

Guide the agent through implementing a single Triton kernel that computes `y = rmsnorm(x + residual)` while also writing back `x + residual` for the next transformer block's residual stream. This fusion is the dominant pattern in LLaMA, Mistral, Qwen, and similar decoder blocks: every attention sub-block and every MLP sub-block ends with `residual_add -> rmsnorm`. Done correctly, the kernel saves one full read+write pass over the activation tensor compared to a naive `add` kernel followed by an

Guide the agent through implementing a Triton kernel that writes newly computed K and V tensors into a pre-allocated KV cache during LLM inference. This covers two cache layouts (contiguous and paged / vLLM-style PagedAttention), unified prefill and decode handling via a `slot_mapping` tensor, GQA/MQA where the cache stores fewer heads than Q, optional fp8/int8 quantized KV cache with scaling, coalesced versus scattered write patterns, and the boundary checks required to avoid corrupting other r

Guide the agent through implementing a correct, numerically stable RMSNorm kernel in Triton: `y = x * rsqrt(mean(x², axis=-1) + eps) * weight`. RMSNorm is the dominant normalization in modern decoder-only LLMs (LLaMA, Mistral, Qwen, Gemma, DeepSeek). This skill covers one-pass sum-of-squares with fp32 accumulation, the persistent kernel pattern when the hidden dim fits in a single tile, masking for non-divisible tails, the affine weight broadcast (no bias), and the backward pass. RMSNorm is stru

Guide the agent through implementing a correct Triton kernel that applies Rotary Position Embeddings (RoPE) to query and key tensors before attention. This covers the two incompatible layout conventions (GPT-NeoX/HuggingFace-LLaMA vs GPT-J/original-paper), pre-computed cos/sin table consumption, per-token position handling for continuous batching, partial-RoPE masking, and the precision discipline required to keep cos/sin in fp32 while applying to fp16/bf16 activations. RoPE is the dominant posi

Guide the agent through implementing a Triton kernel for LLM decode-time token sampling: take a `[batch, vocab]` logits tensor, apply per-request temperature, top-k, and top-p (nucleus) filtering, renormalize, and draw one token per request. This is the last hot kernel on every decode step — it runs once per generated token, so latency directly translates into tokens/second.

Guide the agent through implementing a correct, numerically stable Triton kernel that computes `y = silu(a) * b`, the elementwise activation step inside SwiGLU MLPs used by LLaMA, Mistral, Qwen, Gemma, and similar modern LLMs. The full MLP is `down_proj( silu(gate_proj(x)) * up_proj(x) )`; this skill covers the fused activation that sits between the two GEMMs. It also generalizes to GeGLU (`gelu(a) * b`) and ReGLU (`relu(a) * b`), which share the same kernel structure with a different activation

Guide the agent through planning the integration of a custom CUDA or Triton kernel into the vLLM inference engine before any integration code is written — covering where the op plugs into the engine, paged KV cache and continuous batching compatibility, CUDA graph capture constraints, tensor parallelism implications, and the testing and benchmarking strategy. This skill produces an integration plan, not a kernel implementation.

Guide the agent through selecting tile sizes and work partitioning strategies for a CUDA or Triton kernel, based on shared memory budget, register pressure, occupancy targets, problem shape, and access pattern.

Guide the agent through deciding whether to fuse multiple elementwise operations into a single kernel pass, and if so, how to implement the fusion correctly and efficiently in CUDA or Triton.

Guide the agent through correctly handling partial tiles — cases where a problem dimension does not evenly divide the tile size — in CUDA and Triton kernels, without introducing out-of-bounds accesses, incorrect output values, or silent data corruption.

Guide the agent through constructing a systematic, coverage-complete test plan for a compute kernel, covering correctness, numerical precision, boundary conditions, layout variations, and performance regression.

Guide the agent through identifying numerical instability risks in a kernel's computation path and applying the correct stabilization strategy for each risk class.

Guide the agent through translating a CUDA kernel to AMD HIP for ROCm-compatible hardware (MI250, MI300, RDNA), preserving correctness and performance intent while adapting to the HIP execution model, memory model, and AMD-specific toolchain.

Guide the agent through systematically porting an existing CUDA kernel to Triton, mapping the CUDA execution model to Triton's tile-based program model, preserving numerical correctness, and identifying the patterns that do not translate directly.

Guide the agent through planning a compute kernel that must run correctly and performantly on multiple hardware backends (NVIDIA, AMD, CPU fallback, or future backends) before any backend-specific implementation is written — covering abstraction strategy, feature compatibility mapping, and the tradeoffs between portability and performance.

Guide the agent through a systematic process for diagnosing and isolating accuracy degradation in a quantized (INT8, FP8, or low-bit) kernel, from measuring the error to identifying the specific computational step responsible.

Guide the agent through designing and implementing FP8 compute kernels for inference and training on NVIDIA Hopper (sm_90) and Ada Lovelace (sm_89) hardware, covering FP8 format selection, scaling strategy, tensor core usage via WGMMA or cuBLAS, and dequantization epilogue design.

Guide the agent through designing and implementing an INT8 quantized matrix multiplication or linear layer kernel for inference, covering quantization scheme selection, scale computation, int32 accumulation, dequantization epilogue, and the decision between custom code and library solutions.

Guide the agent through the systematic process of choosing and tuning block size parameters in Triton kernels — BLOCK_M, BLOCK_N, BLOCK_K for GEMM-style kernels; BLOCK_SIZE for reduction and pointwise kernels; and the associated num_warps and num_stages values that control parallelism and pipeline depth. This is an optimization skill, not a write-kernel skill. It assumes a correct kernel exists and asks: what configuration makes it fast?

Guide the agent through implementing a Flash Attention 2-style fused attention kernel in Triton. This covers the outer loop over KV sequence blocks, online softmax with running max and log-sum-exp tracking, numerically stable incremental output accumulation, causal masking, score scaling, output rescaling at the end, and correct stride arithmetic for batch and head dimensions. This is not a tutorial on attention mechanics — it is a decision framework for a correct Triton implementation.

Guide the agent through implementing a correct, performant blocked matrix multiplication kernel in Triton. This covers tile assignment via `program_id`, pointer arithmetic for A/B/C tiles, accumulation with `tl.dot`, boundary masking for non-divisible shapes, swizzled tile ordering for L2 reuse, and autotuning for BLOCK_M/BLOCK_N/BLOCK_K/num_stages/num_warps.

Guide the agent through implementing a correct, numerically stable row-wise layer normalization kernel in Triton. This covers mean and variance computation with fp32 accumulation, epsilon handling, affine transform with gamma/beta, the RMSNorm variant, masking for hidden dimensions not divisible by BLOCK_SIZE, and pointer arithmetic for 1D affine parameters applied to 2D or higher-rank inputs.

Guide the agent through implementing a numerically stable, performant row-wise softmax kernel in Triton. This covers single-program-per-row assignment, online max+sum reduction with `tl.max`/`tl.sum`, masking for rows wider than BLOCK_SIZE, fp32 accumulation to avoid overflow and precision loss, and the masked softmax variant for attention.