Instructions to use kernels-community/finegrained-fp8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Kernels
How to use kernels-community/finegrained-fp8 with Kernels:
# !pip install kernels from kernels import get_kernel # a version (or an explicit revision) is required; see the "Files and versions" tab for the available ones kernel = get_kernel("kernels-community/finegrained-fp8", version=1) - Notebooks
- Google Colab
- Kaggle
Commit History
-
Fix divide-by-zero NaN in _fp8_act_quant_kernel
Verified signature# Fix divide-by-zero NaN in `_fp8_act_quant_kernel` `s = tl.max(tl.abs(x)) / 448.0` is zero when a 128-block is all-zero. Then `y = x/s = NaN` propagates through every downstream FP8 matmul. ```diff - s = tl.max(tl.abs(x)) / 448.0 # float8_e4m3fn max - y = (x / s).to(y_ptr.dtype.element_ty) + amax = tl.max(tl.abs(x)) + s = tl.maximum(amax / 448.0, 1e-12) # eps floor; float8_e4m3fn max = 448 + y = (x / s).to(y_ptr.dtype.element_ty) ``` Same hunk in `build/torch-cuda`, `build/torch-xpu`, `build/torch-rocm`. Matches `_per_token_group_quant_fp8` (vllm) and `_per_token_group_quant_8bit` (sglang). ## Trigger Attention masks zero out hidden states at padded positions before the first FP8 quant. Fires on `Qwen/Qwen3-8B-FP8`, `Qwen/Qwen3.5-9B-FP8`, `Qwen/Qwen3.5-27B-FP8`, `RedHatAI/Qwen3-8B-FP8-block` when a batch contains any padding. ## Validation H100 80GB, `Qwen/Qwen3.5-27B-FP8` via transformers git main, Triton path forced. | condition | before | after | |---|---|-…ec0cc2d
-
Uploaded using `kernel-builder`.
Verified signatureb0b199b -
Uploaded using `kernel-builder`.
Verified signature6502310 -
Uploaded using `kernel-builder`.
Verified signature1fb71e7 -
Uploaded using `kernel-builder`.
Verified signature13a5abf -
Uploaded using `kernel-builder`.
Verified signature5dd05b4 -
Uploaded using `kernel-builder`.
Verified signaturec3024c3
-
Uploaded using `kernel-builder`.
Verified signature88096a4 -
Uploaded using `kernel-builder`.
Verified signature2e50bc1 -
Uploaded using `kernel-builder`.
Verified signature0effc0b -
Uploaded using `kernel-builder`.
Verified signature5e2fdc6 -
Uploaded using `kernel-builder`.
Verified signature9bc6cb0 -
Uploaded using `kernel-builder`.
Verified signature43fa394 -
Uploaded using `kernel-builder`.
Verified signature0b53a54 -
Uploaded using `kernel-builder`.
Verified signatured5c22fd
-
Uploaded using `kernel-builder`.
Verified signature827f2f4 -
Uploaded using `kernel-builder`.
Verified signaturedc8d6b6 -
Uploaded using `kernel-builder`.
Verified signature9d9c1da -
Uploaded using `kernel-builder`.
Verified signaturef2dccf0
-
Uploaded using `kernel-builder`.
Verified signature9bc4bb7 -
Uploaded using `kernel-builder`.
Verified signature8720b03 -
Uploaded using `kernel-builder`.
Verified signature611ba70 -
Uploaded using `kernel-builder`.
Verified signatureb89d9ac
-
Uploaded using `kernel-builder`.
Verified signature7d6cced -
Uploaded using `kernel-builder`.
Verified signature138a392
-
Build uploaded using `kernels`.
Verified signature3ed240d
-
Build uploaded using `kernels`.
Verified signatureaa8cc0c -
Build uploaded using `kernels`.
Verified signature7d2ee95
-
Build uploaded using `kernels`.
Verified signatureb1e6bf1 -
Build uploaded using `kernels`.
Verified signaturee6a53d5
-
Build uploaded using `kernels`.
Verified signature0afce10 -
Build uploaded using `kernels`.
Verified signature08e5623
-
Build uploaded using `kernels`.
Verified signature0c86a40 -
Build uploaded using `kernels`.
Verified signature73d4076
-
Build uploaded using `kernels`.
Verified signaturee9d1f46 -
initial commit
Verified signature53848dc