Instructions to use replicate/flash-attn4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Kernels
How to use replicate/flash-attn4 with Kernels:
# !pip install kernels from kernels import get_kernel # a version (or an explicit revision) is required; see the "Files and versions" tab for the available ones kernel = get_kernel("replicate/flash-attn4", version=1) - Notebooks
- Google Colab
- Kaggle
Download build/torch-cuda/fast_math.py from replicate/flash-attn4: direct link, hf CLI and curl.
- Browser
- Download file 492 Bytes
-
https://huggingface.co/replicate/flash-attn4/resolve/main/build/torch-cuda/fast_math.py
- Command line
-
hf download hf://replicate/flash-attn4/build/torch-cuda/fast_math.py
-
curl -L -o fast_math.py https://huggingface.co/replicate/flash-attn4/resolve/main/build/torch-cuda/fast_math.py
492 Bytes
| # Copyright (c) 2025, Tri Dao. | |
| import cutlass | |
| import cutlass.cute as cute | |
| from cutlass import Int32 | |
| def clz(x: Int32) -> Int32: | |
| # for i in cutlass.range_constexpr(32): | |
| # if (1 << (31 - i)) & x: | |
| # return Int32(i) | |
| # return Int32(32) | |
| # Early exit is not supported yet | |
| res = Int32(32) | |
| done = False | |
| for i in cutlass.range(32): | |
| if ((1 << (31 - i)) & x) and not done: | |
| res = Int32(i) | |
| done = True | |
| return res | |