Uploaded using `kernel-builder`.
Browse files
README.md
CHANGED
|
@@ -13,9 +13,9 @@ tags:
|
|
| 13 |
# Gated Delta Attention
|
| 14 |
|
| 15 |
BF16 Gated DeltaNet recurrent/chunk/WY kernels from FlashRT, packaged for
|
| 16 |
-
Hugging Face Kernel Hub. The
|
| 17 |
-
|
| 18 |
-
|
| 19 |
|
| 20 |
## Available functions
|
| 21 |
|
|
@@ -25,11 +25,15 @@ native CUDA FLA-style MMA prefill path.
|
|
| 25 |
- `gated_delta_chunk_bf16`
|
| 26 |
- `gated_delta_chunk_smem_bf16`
|
| 27 |
- `lin_split_qkv_broadcast_bf16`
|
|
|
|
| 28 |
- `lin_split_qkv_gqa_bf16`
|
| 29 |
- `split_q_gate_bf16`
|
| 30 |
- `gdn_gating_bf16`
|
|
|
|
| 31 |
- `gdn_gating_strided_bf16`
|
|
|
|
| 32 |
- `gdn_chunk_from_conv_smem_bf16`
|
|
|
|
| 33 |
- `gdn_wy_norm_cumsum_pack_qk_bf16`
|
| 34 |
- `gdn_wy_kkt_b64_bf16`
|
| 35 |
- `gdn_wy_solve_tril_b64_f32`
|
|
@@ -47,7 +51,7 @@ native CUDA FLA-style MMA prefill path.
|
|
| 47 |
```python
|
| 48 |
from kernels import get_kernel
|
| 49 |
|
| 50 |
-
gdn = get_kernel("flashrt/gated-delta-attention", version=
|
| 51 |
out = gdn.gated_delta_recurrent_bf16(q, k, v, g, beta, state)
|
| 52 |
```
|
| 53 |
|
|
@@ -55,6 +59,20 @@ The WY helpers use the Qwen3.6 profile: `conv_out=(S,10240)`,
|
|
| 55 |
Q/K heads `16`, value heads `48`, head dimension `128`, and 64-token WY
|
| 56 |
blocks.
|
| 57 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 58 |
The FLA-style path keeps the hot prefill chain in CUDA kernels:
|
| 59 |
|
| 60 |
```python
|
|
|
|
| 13 |
# Gated Delta Attention
|
| 14 |
|
| 15 |
BF16 Gated DeltaNet recurrent/chunk/WY kernels from FlashRT, packaged for
|
| 16 |
+
Hugging Face Kernel Hub. The v4 API adds a model-neutral parameterized
|
| 17 |
+
`Hv/Hk` producer and fused recurrent prefill path. Validated profiles include
|
| 18 |
+
`Hv/Hk/D=48/16/128` and `32/16/128`.
|
| 19 |
|
| 20 |
## Available functions
|
| 21 |
|
|
|
|
| 25 |
- `gated_delta_chunk_bf16`
|
| 26 |
- `gated_delta_chunk_smem_bf16`
|
| 27 |
- `lin_split_qkv_broadcast_bf16`
|
| 28 |
+
- `lin_split_qkv_broadcast_h_bf16`
|
| 29 |
- `lin_split_qkv_gqa_bf16`
|
| 30 |
- `split_q_gate_bf16`
|
| 31 |
- `gdn_gating_bf16`
|
| 32 |
+
- `gdn_gating_h_bf16`
|
| 33 |
- `gdn_gating_strided_bf16`
|
| 34 |
+
- `gdn_gating_strided_h_bf16`
|
| 35 |
- `gdn_chunk_from_conv_smem_bf16`
|
| 36 |
+
- `gdn_chunk_from_conv_smem_h_bf16`
|
| 37 |
- `gdn_wy_norm_cumsum_pack_qk_bf16`
|
| 38 |
- `gdn_wy_kkt_b64_bf16`
|
| 39 |
- `gdn_wy_solve_tril_b64_f32`
|
|
|
|
| 51 |
```python
|
| 52 |
from kernels import get_kernel
|
| 53 |
|
| 54 |
+
gdn = get_kernel("flashrt/gated-delta-attention", version=4, trust_remote_code=True)
|
| 55 |
out = gdn.gated_delta_recurrent_bf16(q, k, v, g, beta, state)
|
| 56 |
```
|
| 57 |
|
|
|
|
| 59 |
Q/K heads `16`, value heads `48`, head dimension `128`, and 64-token WY
|
| 60 |
blocks.
|
| 61 |
|
| 62 |
+
The generic H32 producer profile uses `conv_out=(S,8192)`, Q/K heads `16`,
|
| 63 |
+
value heads `32`, and head dimension `128`. Q/K are broadcast `16 -> 32`.
|
| 64 |
+
Per-head `neg_exp_A_log` and `dt_bias` must remain FP32.
|
| 65 |
+
|
| 66 |
+
```python
|
| 67 |
+
S, Hv, Hk, D = 64, 32, 16, 128
|
| 68 |
+
q, k, v = gdn.lin_split_qkv_broadcast_h_bf16(conv_out, Hv, Hk)
|
| 69 |
+
g, beta = gdn.gdn_gating_h_bf16(a, b, neg_exp_A_log, dt_bias)
|
| 70 |
+
out = gdn.gdn_chunk_from_conv_smem_h_bf16(
|
| 71 |
+
conv_out, a, b, neg_exp_A_log, dt_bias, state,
|
| 72 |
+
num_v_heads=Hv, num_k_heads=Hk,
|
| 73 |
+
)
|
| 74 |
+
```
|
| 75 |
+
|
| 76 |
The FLA-style path keeps the hot prefill chain in CUDA kernels:
|
| 77 |
|
| 78 |
```python
|