Spaces:

---

kernels-community

/

README

Like 14

Running

Hi! I’d like to publish kernels under the crit-labs org.

Built: ten kernel-builder projects build all their variants (kernel-builder eb9a88f). Each project has 18 CUDA variants for torch 2.10-2.15 and CUDA 12.6-13.4, except pinned-gemm, which is CUDA 13 only, and three are torch-noarch. Each project’s own tests pass on an RTX 5090, 512 in total.
Source: https://github.com/crit-labs/crit-kernels (decode-gemv, fixed-tile-gemm, fused-norms, grouped-experts, mma-rows, paged-kv-attention, pinned-gemm, recurrent-scans, topk-routing, workbench-starter).
Purpose: crit’s plugin library loads these kernels through the Kernel Hub (https://github.com/crit-labs/crit). crit is an open-source toolkit for building and evaluating LLM inference engines. The kernels cover quantized decode GEMV and prompt GEMMs (NVFP4/FP8/MXFP8), paged-KV attention, recurrent scans for hybrid models, routed-expert GEMMs and top-k routing, and fused norms. workbench-starter is the template crit’s kernel workbench starts from.
Benchmarks: an engine built from these kernels decodes a 27B NVFP4 model at 220 tok/s sampled on an RTX 5090.