dawidope commited on
Commit
985f585
Β·
verified Β·
1 Parent(s): 903354c

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +18 -47
README.md CHANGED
@@ -18,15 +18,18 @@ Prebuilt Python 3.11 wheels.
18
  |---|---|---|---|---|---|
19
  | `ace_step-1.6.0-py3-none-any.whl` | any | β€” | β€” | built by us | Pure-Python, cross-platform |
20
  | `qwen_tts-0.1.1+deapi3-py3-none-any.whl` | any | β€” | β€” | built by us | Pure-Python, patched fork (`+deapi3`) |
21
- | `image_server_kernels-0.4.2+cu128torch2.8-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | **Current.** Adds Sol-Attn sparse attention, W4A8 linear, and in-tree llama.cpp GGUF CUDA kernels (`_C_gguf`), on top of: FP8 dense + grouped GEMM, NVFP4 W4A4, INT8 W8A8 + ConvRot, fused QK-norm+RoPE. SM89 + SM120 |
22
- | `image_server_kernels-0.3.0-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | Previous build, kept as a fallback. No Sol-Attn, no W4A8, no bundled GGUF kernels |
23
- | `image_server_kernels-0.3.0+cu129torch2.8-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.9 | 2.8 | built by us | Linux build is still at 0.3.0 β€” there is no Linux 0.4.2 yet |
 
 
 
24
  | `block_sparse_attn-0.0.2-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | Used by video pipeline |
25
  | `block_sparse_attn-0.0.2-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.8 | 2.8 | built by us | Used by video pipeline |
26
  | `q8_kernels-0.0.5-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | Used by LTX video |
27
  | `q8_kernels-0.0.5-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.8 | 2.8 | built by us | Used by LTX video |
28
  | `sageattention-2.2.0+cu128torch2.8.0-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | [woct0rdho/SageAttention](https://github.com/woct0rdho/SageAttention/releases/tag/v2.2.0-windows) | Mirror of upstream release |
29
- | `sageattention-2.2.0+cu129torch2.8-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.9 | 2.8 | provenance unconfirmed β€” see Credits | |
30
  | `flash_attn-2.8.2+cu128torch2.8-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | [mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/tag/v0.4.10) | Mirror of upstream release |
31
  | `flash_attn-2.8.3+cu130torch2.10-cp311-cp311-win_amd64.whl` | Windows x64 | 13.0 | 2.10 | [mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels) | Mirror of upstream release |
32
  | `flash_attn-2.8.3+cu128torch2.8-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.8 | 2.8 | [mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/tag/v0.7.16) | Mirror of upstream release |
@@ -37,7 +40,7 @@ Prebuilt Python 3.11 wheels.
37
  BASE=https://huggingface.co/deAPI-ai/image-server-wheels/resolve/main
38
 
39
  # Windows
40
- pip install $BASE/image_server_kernels-0.4.2+cu128torch2.8-cp311-cp311-win_amd64.whl
41
  pip install $BASE/q8_kernels-0.0.5-cp311-cp311-win_amd64.whl
42
  pip install $BASE/block_sparse_attn-0.0.2-cp311-cp311-win_amd64.whl
43
  pip install $BASE/flash_attn-2.8.2+cu128torch2.8-cp311-cp311-win_amd64.whl
@@ -46,39 +49,16 @@ pip install --no-deps $BASE/ace_step-1.6.0-py3-none-any.whl
46
  pip install --no-deps $BASE/qwen_tts-0.1.1+deapi3-py3-none-any.whl
47
 
48
  # Linux
49
- pip install $BASE/image_server_kernels-0.3.0+cu129torch2.8-cp311-cp311-linux_x86_64.whl
50
  pip install $BASE/q8_kernels-0.0.5-cp311-cp311-linux_x86_64.whl
51
  pip install $BASE/block_sparse_attn-0.0.2-cp311-cp311-linux_x86_64.whl
52
  pip install $BASE/sageattention-2.2.0+cu129torch2.8-cp311-cp311-linux_x86_64.whl
53
  pip install --no-deps $BASE/flash_attn-2.8.3+cu128torch2.8-cp311-cp311-linux_x86_64.whl
54
  ```
55
 
56
- `sageattention` is required by the MiniMax-H3 video model, which imports it
57
- unconditionally. Every other model that can use it falls back to SDPA when it is
58
- absent, so on a box that does not serve H3 the wheel is optional.
59
-
60
- > `image_server_kernels` is at **0.4.2 on Windows but only 0.3.0 on Linux** β€” the
61
- > Linux build of 0.4.2 has not been made yet. Sol-Attn, W4A8 and the bundled GGUF
62
- > kernels are therefore Windows-only for now.
63
-
64
- ## What is verified in `image_server_kernels-0.4.2`
65
-
66
- Built and tested on Windows 11 / RTX 5090 Laptop (SM120), CUDA 12.8, torch 2.8.0+cu128.
67
-
68
- - `scripts/smoke_test_rebuild.py` β€” ALL PASS: `fp8_dense_gemm` (per-tensor cos 0.99965,
69
- per-channel+bias 0.99972), `dynamic_per_token_scaled_fp8_quant` 0.99965,
70
- `fp8_dense_gemm_pre_quantized` 1.00000, `fp8_dense_gemm_sm120` 1.00000 via native
71
- CUTLASS 3.x, `rmsnorm_forward` 0.999999.
72
- - `scripts/validate_gguf_kernels.py` against real GGUF tensors β€” ALL PASS for
73
- Q4_K, Q5_K, Q6_K, Q5_0, Q5_1, Q8_0 (cos 0.99984–1.00000), which covers the bf16
74
- K-quant dequant path the in-tree patch exists for.
75
- - `sol_attn` on real MiniMax-H3 activations (S=15,479, 5 capture points):
76
- **2.22x vs SageAttention2 at cos 0.9805** at `tau=1.0`. Both speed and accuracy
77
- rise with sequence length (4k β†’ 1.68x/0.965, 15.5k β†’ 2.90x/0.980 in one run),
78
- so longer clips do better than this figure.
79
-
80
- Not exercised on this box: `sol_attn` on Ada SM89 (compiles and links, but no Ada
81
- GPU here), `w4a8_linear`, `nvfp4_*`, `fp8_grouped_gemm`, `w8a16_dense_gemm`.
82
 
83
  ## Credits
84
 
@@ -86,26 +66,17 @@ GPU here), `w4a8_linear`, `nvfp4_*`, `fp8_grouped_gemm`, `w8a16_dense_gemm`.
86
  [mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels)
87
  β€” all credit for those builds goes to the upstream author.
88
 
89
- `sageattention` is **a mirror, not our build**. The library is
90
- [thu-ml/SageAttention](https://github.com/thu-ml/SageAttention) (Apache-2.0); the
91
- Windows wheel is built and published by
92
  [woct0rdho](https://github.com/woct0rdho/SageAttention/releases), and we found it
93
  through [wildminder/AI-windows-whl](https://github.com/wildminder/AI-windows-whl),
94
  which indexes Windows CUDA wheels. All credit for the library and the build goes to
95
- them. The Windows file here is byte-identical to the upstream release
96
  (`sha256:4379951403809dfcd5b1e10d35e287abf42afafb7b27615c09d6062cfbdf230a`).
97
 
98
- > The **Linux** `sageattention-2.2.0+cu129torch2.8` wheel's provenance is not
99
- > recorded β€” we have not confirmed whether it is our own build or a mirror. Fill
100
- > this in before relying on the attribution.
101
-
102
  We mirror third-party wheels so the install scripts have a single source of truth and
103
  do not break if upstream release URLs change.
104
 
105
- `ace_step`, `qwen_tts`, `image_server_kernels`, `block_sparse_attn` and `q8_kernels`
106
- were built in-house.
107
-
108
- Sol-Attn in `image_server_kernels` wraps CUDA kernels from comfy-kitchen
109
- (Apache-2.0); see `third_party/comfy_kitchen_sol/PROVENANCE.md` in the
110
- `image-server-kernels` repo. The bundled GGUF kernels are llama.cpp's `ggml-cuda`
111
- (MIT), vendored per `third_party/llamacpp_gguf/PROVENANCE.md`.
 
18
  |---|---|---|---|---|---|
19
  | `ace_step-1.6.0-py3-none-any.whl` | any | β€” | β€” | built by us | Pure-Python, cross-platform |
20
  | `qwen_tts-0.1.1+deapi3-py3-none-any.whl` | any | β€” | β€” | built by us | Pure-Python, patched fork (`+deapi3`) |
21
+ | `image_server_kernels-0.4.3+cu128torch2.8-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | **Current.** FP8 dense + grouped GEMM, NVFP4 W4A4, INT8 W8A8 + ConvRot, W4A8, GGUF, fused QK-norm+RoPE, Sol-Attn. SM89 + SM120 |
22
+ | `image_server_kernels-0.4.3+cu129torch2.8-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.9 | 2.8 | built by us | **Current.** Same op set |
23
+ | `image_server_kernels-0.4.2+cu128torch2.8-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | Previous build, kept as a rollback |
24
+ | `image_server_kernels-0.4.2+cu129torch2.8-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.9 | 2.8 | built by us | Previous build, kept as a rollback |
25
+ | `image_server_kernels-0.3.0-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | Legacy. No Sol-Attn, no W4A8, no bundled GGUF kernels |
26
+ | `image_server_kernels-0.3.0+cu129torch2.8-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.9 | 2.8 | built by us | Legacy. Same omissions as the Windows 0.3.0 |
27
  | `block_sparse_attn-0.0.2-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | Used by video pipeline |
28
  | `block_sparse_attn-0.0.2-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.8 | 2.8 | built by us | Used by video pipeline |
29
  | `q8_kernels-0.0.5-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | built by us | Used by LTX video |
30
  | `q8_kernels-0.0.5-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.8 | 2.8 | built by us | Used by LTX video |
31
  | `sageattention-2.2.0+cu128torch2.8.0-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | [woct0rdho/SageAttention](https://github.com/woct0rdho/SageAttention/releases/tag/v2.2.0-windows) | Mirror of upstream release |
32
+ | `sageattention-2.2.0+cu129torch2.8-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.9 | 2.8 | β€” | |
33
  | `flash_attn-2.8.2+cu128torch2.8-cp311-cp311-win_amd64.whl` | Windows x64 | 12.8 | 2.8 | [mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/tag/v0.4.10) | Mirror of upstream release |
34
  | `flash_attn-2.8.3+cu130torch2.10-cp311-cp311-win_amd64.whl` | Windows x64 | 13.0 | 2.10 | [mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels) | Mirror of upstream release |
35
  | `flash_attn-2.8.3+cu128torch2.8-cp311-cp311-linux_x86_64.whl` | Linux x86_64 | 12.8 | 2.8 | [mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/tag/v0.7.16) | Mirror of upstream release |
 
40
  BASE=https://huggingface.co/deAPI-ai/image-server-wheels/resolve/main
41
 
42
  # Windows
43
+ pip install $BASE/image_server_kernels-0.4.3+cu128torch2.8-cp311-cp311-win_amd64.whl
44
  pip install $BASE/q8_kernels-0.0.5-cp311-cp311-win_amd64.whl
45
  pip install $BASE/block_sparse_attn-0.0.2-cp311-cp311-win_amd64.whl
46
  pip install $BASE/flash_attn-2.8.2+cu128torch2.8-cp311-cp311-win_amd64.whl
 
49
  pip install --no-deps $BASE/qwen_tts-0.1.1+deapi3-py3-none-any.whl
50
 
51
  # Linux
52
+ pip install $BASE/image_server_kernels-0.4.3+cu129torch2.8-cp311-cp311-linux_x86_64.whl
53
  pip install $BASE/q8_kernels-0.0.5-cp311-cp311-linux_x86_64.whl
54
  pip install $BASE/block_sparse_attn-0.0.2-cp311-cp311-linux_x86_64.whl
55
  pip install $BASE/sageattention-2.2.0+cu129torch2.8-cp311-cp311-linux_x86_64.whl
56
  pip install --no-deps $BASE/flash_attn-2.8.3+cu128torch2.8-cp311-cp311-linux_x86_64.whl
57
  ```
58
 
59
+ Wheels are built for **Python 3.11** and the CUDA/torch pair named in each filename.
60
+ Installing one against a different torch build is the first thing to suspect if the
61
+ import fails.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
62
 
63
  ## Credits
64
 
 
66
  [mjun0812/flash-attention-prebuild-wheels](https://github.com/mjun0812/flash-attention-prebuild-wheels)
67
  β€” all credit for those builds goes to the upstream author.
68
 
69
+ The Windows `sageattention` wheel is **a mirror, not our build**. The library is
70
+ [thu-ml/SageAttention](https://github.com/thu-ml/SageAttention) (Apache-2.0); the wheel
71
+ is built and published by
72
  [woct0rdho](https://github.com/woct0rdho/SageAttention/releases), and we found it
73
  through [wildminder/AI-windows-whl](https://github.com/wildminder/AI-windows-whl),
74
  which indexes Windows CUDA wheels. All credit for the library and the build goes to
75
+ them. The file here is byte-identical to the upstream release
76
  (`sha256:4379951403809dfcd5b1e10d35e287abf42afafb7b27615c09d6062cfbdf230a`).
77
 
 
 
 
 
78
  We mirror third-party wheels so the install scripts have a single source of truth and
79
  do not break if upstream release URLs change.
80
 
81
+ The remaining wheels (`ace_step`, `qwen_tts`, `image_server_kernels`,
82
+ `block_sparse_attn`, `q8_kernels`) were built in-house.