Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -18,15 +18,18 @@ Prebuilt Python 3.11 wheels.
|
|
| 18 |
|---|---|---|---|---|---|
|
| 19 |
| `ace_step-1.6.0-py3-none-any.whl` | any | β | β | built by us | Pure-Python, cross-platform |
|
| 20 |
| `qwen_tts-0.1.1+deapi3-py3-none-any.whl` | any | β | β | built by us | Pure-Python, patched fork (`+deapi3`) |
|
| 21 |
-
| `image_server_kernels-0.4.
|
| 22 |
-
| `image_server_kernels-0.3.
|
| 23 |
-
| `image_server_kernels-0.
|
|
|
|
|
|
|
|
|
|
| 24 |
| `block_sparse_attn-0.0.2-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | Used by video pipeline |
|
| 25 |
| `block_sparse_attn-0.0.2-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.8 | 2.8 | built by us | Used by video pipeline |
|
| 26 |
| `q8_kernels-0.0.5-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | Used by LTX video |
|
| 27 |
| `q8_kernels-0.0.5-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.8 | 2.8 | built by us | Used by LTX video |
|
| 28 |
| `sageattention-2.2.0+cu128torch2.8.0-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | [woct0rdho/SageAttention](https://github.com/woct0rdho/SageAttention/releases/tag/v2.2.0-windows) | Mirror of upstream release |
|
| 29 |
-
| `sageattention-2.2.0+cu129torch2.8-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.9 | 2.8 |
|
| 30 |
| `flash_attn-2.8.2+cu128torch2.8-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | [mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/tag/v0.4.10) | Mirror of upstream release |
|
| 31 |
| `flash_attn-2.8.3+cu130torch2.10-cp311-cp311-win_amd64.whl` | Windows x64 | 13.0 | 2.10 | [mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels) | Mirror of upstream release |
|
| 32 |
| `flash_attn-2.8.3+cu128torch2.8-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.8 | 2.8 | [mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/tag/v0.7.16) | Mirror of upstream release |
|
|
@@ -37,7 +40,7 @@ Prebuilt Python 3.11 wheels.
|
|
| 37 |
BASE=https://huggingface.co/deAPI-ai/image-server-wheels/resolve/main
|
| 38 |
|
| 39 |
# Windows
|
| 40 |
-
pip install $BASE/image_server_kernels-0.4.
|
| 41 |
pip install $BASE/q8_kernels-0.0.5-cp311-cp311-win_amd64.whl
|
| 42 |
pip install $BASE/block_sparse_attn-0.0.2-cp311-cp311-win_amd64.whl
|
| 43 |
pip install $BASE/flash_attn-2.8.2+cu128torch2.8-cp311-cp311-win_amd64.whl
|
|
@@ -46,39 +49,16 @@ pip install --no-deps $BASE/ace_step-1.6.0-py3-none-any.whl
|
|
| 46 |
pip install --no-deps $BASE/qwen_tts-0.1.1+deapi3-py3-none-any.whl
|
| 47 |
|
| 48 |
# Linux
|
| 49 |
-
pip install $BASE/image_server_kernels-0.
|
| 50 |
pip install $BASE/q8_kernels-0.0.5-cp311-cp311-linux_x86_64.whl
|
| 51 |
pip install $BASE/block_sparse_attn-0.0.2-cp311-cp311-linux_x86_64.whl
|
| 52 |
pip install $BASE/sageattention-2.2.0+cu129torch2.8-cp311-cp311-linux_x86_64.whl
|
| 53 |
pip install --no-deps $BASE/flash_attn-2.8.3+cu128torch2.8-cp311-cp311-linux_x86_64.whl
|
| 54 |
```
|
| 55 |
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
> `image_server_kernels` is at **0.4.2 on Windows but only 0.3.0 on Linux** β the
|
| 61 |
-
> Linux build of 0.4.2 has not been made yet. Sol-Attn, W4A8 and the bundled GGUF
|
| 62 |
-
> kernels are therefore Windows-only for now.
|
| 63 |
-
|
| 64 |
-
## What is verified in `image_server_kernels-0.4.2`
|
| 65 |
-
|
| 66 |
-
Built and tested on Windows 11 / RTX 5090 Laptop (SM120), CUDA 12.8, torch 2.8.0+cu128.
|
| 67 |
-
|
| 68 |
-
- `scripts/smoke_test_rebuild.py` β ALL PASS: `fp8_dense_gemm` (per-tensor cos 0.99965,
|
| 69 |
-
per-channel+bias 0.99972), `dynamic_per_token_scaled_fp8_quant` 0.99965,
|
| 70 |
-
`fp8_dense_gemm_pre_quantized` 1.00000, `fp8_dense_gemm_sm120` 1.00000 via native
|
| 71 |
-
CUTLASS 3.x, `rmsnorm_forward` 0.999999.
|
| 72 |
-
- `scripts/validate_gguf_kernels.py` against real GGUF tensors β ALL PASS for
|
| 73 |
-
Q4_K, Q5_K, Q6_K, Q5_0, Q5_1, Q8_0 (cos 0.99984β1.00000), which covers the bf16
|
| 74 |
-
K-quant dequant path the in-tree patch exists for.
|
| 75 |
-
- `sol_attn` on real MiniMax-H3 activations (S=15,479, 5 capture points):
|
| 76 |
-
**2.22x vs SageAttention2 at cos 0.9805** at `tau=1.0`. Both speed and accuracy
|
| 77 |
-
rise with sequence length (4k β 1.68x/0.965, 15.5k β 2.90x/0.980 in one run),
|
| 78 |
-
so longer clips do better than this figure.
|
| 79 |
-
|
| 80 |
-
Not exercised on this box: `sol_attn` on Ada SM89 (compiles and links, but no Ada
|
| 81 |
-
GPU here), `w4a8_linear`, `nvfp4_*`, `fp8_grouped_gemm`, `w8a16_dense_gemm`.
|
| 82 |
|
| 83 |
## Credits
|
| 84 |
|
|
@@ -86,26 +66,17 @@ GPU here), `w4a8_linear`, `nvfp4_*`, `fp8_grouped_gemm`, `w8a16_dense_gemm`.
|
|
| 86 |
[mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels)
|
| 87 |
β all credit for those builds goes to the upstream author.
|
| 88 |
|
| 89 |
-
`sageattention` is **a mirror, not our build**. The library is
|
| 90 |
-
[thu-ml/SageAttention](https://github.com/thu-ml/SageAttention) (Apache-2.0); the
|
| 91 |
-
|
| 92 |
[woct0rdho](https://github.com/woct0rdho/SageAttention/releases), and we found it
|
| 93 |
through [wildminder/AI-windows-whl](https://github.com/wildminder/AI-windows-whl),
|
| 94 |
which indexes Windows CUDA wheels. All credit for the library and the build goes to
|
| 95 |
-
them. The
|
| 96 |
(`sha256:4379951403809dfcd5b1e10d35e287abf42afafb7b27615c09d6062cfbdf230a`).
|
| 97 |
|
| 98 |
-
> The **Linux** `sageattention-2.2.0+cu129torch2.8` wheel's provenance is not
|
| 99 |
-
> recorded β we have not confirmed whether it is our own build or a mirror. Fill
|
| 100 |
-
> this in before relying on the attribution.
|
| 101 |
-
|
| 102 |
We mirror third-party wheels so the install scripts have a single source of truth and
|
| 103 |
do not break if upstream release URLs change.
|
| 104 |
|
| 105 |
-
`ace_step`, `qwen_tts`, `image_server_kernels`,
|
| 106 |
-
were built in-house.
|
| 107 |
-
|
| 108 |
-
Sol-Attn in `image_server_kernels` wraps CUDA kernels from comfy-kitchen
|
| 109 |
-
(Apache-2.0); see `third_party/comfy_kitchen_sol/PROVENANCE.md` in the
|
| 110 |
-
`image-server-kernels` repo. The bundled GGUF kernels are llama.cpp's `ggml-cuda`
|
| 111 |
-
(MIT), vendored per `third_party/llamacpp_gguf/PROVENANCE.md`.
|
|
|
|
| 18 |
|---|---|---|---|---|---|
|
| 19 |
| `ace_step-1.6.0-py3-none-any.whl` | any | β | β | built by us | Pure-Python, cross-platform |
|
| 20 |
| `qwen_tts-0.1.1+deapi3-py3-none-any.whl` | any | β | β | built by us | Pure-Python, patched fork (`+deapi3`) |
|
| 21 |
+
| `image_server_kernels-0.4.3+cu128torch2.8-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | **Current.** FP8 dense + grouped GEMM, NVFP4 W4A4, INT8 W8A8 + ConvRot, W4A8, GGUF, fused QK-norm+RoPE, Sol-Attn. SM89 + SM120 |
|
| 22 |
+
| `image_server_kernels-0.4.3+cu129torch2.8-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.9 | 2.8 | built by us | **Current.** Same op set |
|
| 23 |
+
| `image_server_kernels-0.4.2+cu128torch2.8-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | Previous build, kept as a rollback |
|
| 24 |
+
| `image_server_kernels-0.4.2+cu129torch2.8-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.9 | 2.8 | built by us | Previous build, kept as a rollback |
|
| 25 |
+
| `image_server_kernels-0.3.0-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | Legacy. No Sol-Attn, no W4A8, no bundled GGUF kernels |
|
| 26 |
+
| `image_server_kernels-0.3.0+cu129torch2.8-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.9 | 2.8 | built by us | Legacy. Same omissions as the Windows 0.3.0 |
|
| 27 |
| `block_sparse_attn-0.0.2-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | Used by video pipeline |
|
| 28 |
| `block_sparse_attn-0.0.2-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.8 | 2.8 | built by us | Used by video pipeline |
|
| 29 |
| `q8_kernels-0.0.5-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | Used by LTX video |
|
| 30 |
| `q8_kernels-0.0.5-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.8 | 2.8 | built by us | Used by LTX video |
|
| 31 |
| `sageattention-2.2.0+cu128torch2.8.0-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | [woct0rdho/SageAttention](https://github.com/woct0rdho/SageAttention/releases/tag/v2.2.0-windows) | Mirror of upstream release |
|
| 32 |
+
| `sageattention-2.2.0+cu129torch2.8-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.9 | 2.8 | β | |
|
| 33 |
| `flash_attn-2.8.2+cu128torch2.8-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | [mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/tag/v0.4.10) | Mirror of upstream release |
|
| 34 |
| `flash_attn-2.8.3+cu130torch2.10-cp311-cp311-win_amd64.whl` | Windows x64 | 13.0 | 2.10 | [mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels) | Mirror of upstream release |
|
| 35 |
| `flash_attn-2.8.3+cu128torch2.8-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.8 | 2.8 | [mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/tag/v0.7.16) | Mirror of upstream release |
|
|
|
|
| 40 |
BASE=https://huggingface.co/deAPI-ai/image-server-wheels/resolve/main
|
| 41 |
|
| 42 |
# Windows
|
| 43 |
+
pip install $BASE/image_server_kernels-0.4.3+cu128torch2.8-cp311-cp311-win_amd64.whl
|
| 44 |
pip install $BASE/q8_kernels-0.0.5-cp311-cp311-win_amd64.whl
|
| 45 |
pip install $BASE/block_sparse_attn-0.0.2-cp311-cp311-win_amd64.whl
|
| 46 |
pip install $BASE/flash_attn-2.8.2+cu128torch2.8-cp311-cp311-win_amd64.whl
|
|
|
|
| 49 |
pip install --no-deps $BASE/qwen_tts-0.1.1+deapi3-py3-none-any.whl
|
| 50 |
|
| 51 |
# Linux
|
| 52 |
+
pip install $BASE/image_server_kernels-0.4.3+cu129torch2.8-cp311-cp311-linux_x86_64.whl
|
| 53 |
pip install $BASE/q8_kernels-0.0.5-cp311-cp311-linux_x86_64.whl
|
| 54 |
pip install $BASE/block_sparse_attn-0.0.2-cp311-cp311-linux_x86_64.whl
|
| 55 |
pip install $BASE/sageattention-2.2.0+cu129torch2.8-cp311-cp311-linux_x86_64.whl
|
| 56 |
pip install --no-deps $BASE/flash_attn-2.8.3+cu128torch2.8-cp311-cp311-linux_x86_64.whl
|
| 57 |
```
|
| 58 |
|
| 59 |
+
Wheels are built for **Python 3.11** and the CUDA/torch pair named in each filename.
|
| 60 |
+
Installing one against a different torch build is the first thing to suspect if the
|
| 61 |
+
import fails.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 62 |
|
| 63 |
## Credits
|
| 64 |
|
|
|
|
| 66 |
[mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels)
|
| 67 |
β all credit for those builds goes to the upstream author.
|
| 68 |
|
| 69 |
+
The Windows `sageattention` wheel is **a mirror, not our build**. The library is
|
| 70 |
+
[thu-ml/SageAttention](https://github.com/thu-ml/SageAttention) (Apache-2.0); the wheel
|
| 71 |
+
is built and published by
|
| 72 |
[woct0rdho](https://github.com/woct0rdho/SageAttention/releases), and we found it
|
| 73 |
through [wildminder/AI-windows-whl](https://github.com/wildminder/AI-windows-whl),
|
| 74 |
which indexes Windows CUDA wheels. All credit for the library and the build goes to
|
| 75 |
+
them. The file here is byte-identical to the upstream release
|
| 76 |
(`sha256:4379951403809dfcd5b1e10d35e287abf42afafb7b27615c09d6062cfbdf230a`).
|
| 77 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 78 |
We mirror third-party wheels so the install scripts have a single source of truth and
|
| 79 |
do not break if upstream release URLs change.
|
| 80 |
|
| 81 |
+
The remaining wheels (`ace_step`, `qwen_tts`, `image_server_kernels`,
|
| 82 |
+
`block_sparse_attn`, `q8_kernels`) were built in-house.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|