liangsu9988 commited on
Commit
df59ca3
·
verified ·
1 Parent(s): d20c7c7

Initialize legacy kernels compatibility mirror

Browse files
Files changed (1) hide show
  1. README.md +6 -45
README.md CHANGED
@@ -1,48 +1,9 @@
1
- ---
2
- library_name: kernels
3
- license: apache-2.0
4
- tags:
5
- - cuda
6
- - native-cuda
7
- - flashrt
8
- - moe
9
- - nvfp4
10
- - blackwell
11
- ---
12
 
13
- # grouped-moe-gemv
 
14
 
15
- Native CUDA FlashRT grouped MoE GEMV kernels for dynamic device-side routing.
16
- Version 2 covers BF16-activation/NVFP4-weight W4A16 and fully NVFP4 W4A4.
17
 
18
- Available functions:
19
-
20
- - `w4a16_decode_gemv_bf16`
21
- - `grouped_w4a16_gemv_bf16`
22
- - `quantize_activations_nvfp4_bf16`
23
- - `quantize_weights_nvfp4_bf16`
24
- - `grouped_w4a4_gemv_bf16`
25
- - `grouped_w4a4_gemv_from_bf16`
26
-
27
- W4A4 routing is token-major: packed activations `[M,K/2]`, device indices
28
- `[M,top_k]`, packed expert library `[E,N,K/2]`, output `[M,top_k,N]`.
29
- Use `M=routed_pairs, top_k=1` when each expert receives a distinct activation.
30
- See the repository README for static-buffer and CUDA Graph usage.
31
-
32
- Load version 2 with both current and legacy `kernels` clients:
33
-
34
- ```python
35
- from kernels import get_kernel
36
-
37
- try:
38
- moe = get_kernel(
39
- "flashrt/grouped-moe-gemv", version=2, trust_remote_code=True
40
- )
41
- except TypeError: # kernels==0.12.x
42
- moe = get_kernel("flashrt/grouped-moe-gemv", version=2)
43
- ```
44
-
45
- Use `quantize_activations_nvfp4_bf16` once for `[M,K]`, followed by
46
- `grouped_w4a4_gemv_bf16` for all `[M,top_k]` routes. The convenience function
47
- `grouped_w4a4_gemv_from_bf16` performs both calls and accepts preallocated
48
- `packed`, `sfa`, and `out` buffers for CUDA Graph capture.
 
1
+ # flashrt/grouped-moe-gemv
 
 
 
 
 
 
 
 
 
 
2
 
3
+ This repository is a compatibility mirror for older `kernels` clients
4
+ that resolve repositories through the default Hugging Face model repo API.
5
 
6
+ Canonical Kernel Hub repo: https://huggingface.co/kernels/flashrt/grouped-moe-gemv
 
7
 
8
+ Do not edit this mirror by hand. It is generated from the Kernel Hub
9
+ `vN` branches and contains the same `build/**` artifacts.