naklitechie commited on
Commit
164d941
·
verified ·
1 Parent(s): 1ccf338

Sync from GitHub

Browse files
classic/engines/NOTICE CHANGED
@@ -10,7 +10,10 @@ fork of gemma4-webgpu:
10
 
11
  Changes from the original: rewritten for mixture-of-experts models whose experts stream from OPFS (Q8_0 and Q4_0
12
  split planes, slot-indexed expert GEMVs, GPU router top-k, chunked prefill, Q6_K head planes). `ternary_bonsai_2_dflash.js` is built from NakliTechie/dflash-mlx-bonsai2 (Apache-2.0). `lfm2_5.js`,
13
- `gemma-4-e2b.js` and `ternary_bonsai_2_27b.js` are built from webml-community Hugging Face Spaces by the same author, MIT. Everything else in
 
 
 
14
  this folder is MIT (../LICENSE), Copyright (c) 2026 Chirag Patnaik.
15
 
16
  The engine files are copies of LocalMind's (https://github.com/NakliTechie/LocalMind), kept in step by
 
10
 
11
  Changes from the original: rewritten for mixture-of-experts models whose experts stream from OPFS (Q8_0 and Q4_0
12
  split planes, slot-indexed expert GEMVs, GPU router top-k, chunked prefill, Q6_K head planes). `ternary_bonsai_2_dflash.js` is built from NakliTechie/dflash-mlx-bonsai2 (Apache-2.0). `lfm2_5.js`,
13
+ `gemma-4-e2b.js` and `ternary_bonsai_2_27b.js` are built from webml-community Hugging Face Spaces by the same author, MIT.
14
+ `clover_moe_ssd.js` follows the math of Diffuspace's modeling_clover_moe.py (Apache-2.0, Clover-35B-A3B-Preview); no
15
+ code is copied. `vendor/kohra-denoise.js` is kohra.js's denoise.js, unchanged (NakliTechie/kohra.js, Apache-2.0,
16
+ engines/LICENSE-APACHE), checked by scripts/sync-kohra.mjs. Everything else in
17
  this folder is MIT (../LICENSE), Copyright (c) 2026 Chirag Patnaik.
18
 
19
  The engine files are copies of LocalMind's (https://github.com/NakliTechie/LocalMind), kept in step by
classic/engines/clover_moe_ssd.js ADDED
@@ -0,0 +1,498 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /* clover_moe_ssd.js — Clover-35B-A3B-Preview (Diffuspace, Apache-2.0; GGUF arch `clover_moe`) in a browser tab,
2
+ * larger than RAM. Clover is a block-diffusion model on Qwen3.6-35B-A3B: it fills a 32-token block of mask tokens
3
+ * over several denoising passes, then commits the block into a causal prefix. The backbone, the expert paging and
4
+ * the commit pass are Qwen35MoeSsd's (a commit is one prefillChunk); what is new is the preview pass:
5
+ * - DeltaNet layers: the 4-tap conv also looks right, inside the block (conv1d.weight_right, 3 taps), and the
6
+ * delta rule becomes a softmax over the block's 32 keys plus one "history" slot, whose value is the read-only
7
+ * clean recurrent state read with the query and decayed by the position's gate. Nothing is written back.
8
+ * - full-attention layers: every block token sees the whole prefix and the whole block (no causal mask). The
9
+ * block's K/V go into the cache rows after the prefix as scratch; a commit overwrites them.
10
+ * - the MoE: one shared expert subset per block (the coreset), the smallest set of experts, by summed routing mass
11
+ * of every token's top 8, that covers moe_block_coreset_top_p (0.9); each token then routes inside it.
12
+ * - logits come back for every block row, and row i scores slot i (not the next token).
13
+ * The math follows the authors' reference (modeling_clover_moe.py: bd_gdn_conv1d, bd_gdn_local_memory, coreset_torch).
14
+ * The denoising loop is not here: kohra's denoise module drives forward() below.
15
+ *
16
+ * const m = await CloverMoeSsd.load(null, { url: '…/Clover-35B-A3B-Preview-Q4_0.gguf', poolBytes });
17
+ * await m.forward({ mode: 'commit', start: 0, ids: prompt, wantLogits: 'none' });
18
+ * const logits = await m.forward({ mode: 'preview', start: m.position, ids: block, wantLogits: 'all' });
19
+ */
20
+
21
+ import { Qwen35MoeSsd } from './qwen35_moe_ssd.js';
22
+ import { STORAGE, COPY_SRC, COPY_DST, MAP_READ } from './qwen3_moe_ssd.js';
23
+ import { denoise, PRESETS } from './vendor/kohra-denoise.js';
24
+
25
+ // The authors' Q4_0 file: Q4_0 matrices, experts, embedding and output, F32 norms and scalars (the kernels
26
+ // Qwen35MoeSsd already has). Their Q8_0 file keeps the embedding, output and alpha/beta in BF16, which no kernel here reads.
27
+ export const CLOVER_35B_A3B = {
28
+ repo: 'Diffuspace/Clover-35B-A3B-Preview-GGUF',
29
+ file: 'Clover-35B-A3B-Preview-Q4_0.gguf',
30
+ revision: '32e22a02b885e173177b7ce7661ad13448b4f89e',
31
+ sha256: '3e2cf14ca47f6e41c251f08237ac79cd37ad81a2ea2a220fff731e34c66930d5',
32
+ size: 19586909600,
33
+ };
34
+
35
+ const QB = 'struct Q { T: u32, pos0: u32, S: u32, pad: u32 }';
36
+ const NEG = '-3.4e38';
37
+
38
+ const PREVIEW_KERNELS = {
39
+ // bd_gdn_conv1d: per channel, the causal 4-tap conv over [3 committed inputs | the block] plus the right taps
40
+ // r[d-1]·x[t+d] (d = 1..3, inside the block), then SiLU. The committed inputs are only read.
41
+ bdConvB: `
42
+ struct P { n: u32 }
43
+ ${QB}
44
+ @group(0) @binding(0) var<storage, read> mixed: array<f32>;
45
+ @group(0) @binding(1) var<storage, read> w: array<f32>;
46
+ @group(0) @binding(2) var<storage, read> wr: array<f32>;
47
+ @group(0) @binding(3) var<storage, read> st: array<f32>;
48
+ @group(0) @binding(4) var<storage, read_write> y: array<f32>;
49
+ @group(0) @binding(5) var<uniform> p: P;
50
+ @group(0) @binding(6) var<uniform> qd: Q;
51
+ fn xin(c: u32, i: u32) -> f32 { if (i < 3u) { return st[c * 3u + i]; } return mixed[(i - 3u) * p.n + c]; }
52
+ @compute @workgroup_size(256) fn main(@builtin(global_invocation_id) gi: vec3<u32>) {
53
+ let c = gi.x; if (c >= p.n) { return; }
54
+ let w0 = w[c * 4u]; let w1 = w[c * 4u + 1u]; let w2 = w[c * 4u + 2u]; let w3 = w[c * 4u + 3u];
55
+ for (var t = 0u; t < qd.T; t++) {
56
+ var sum = 0.0;
57
+ sum += xin(c, t) * w0; sum += xin(c, t + 1u) * w1; sum += xin(c, t + 2u) * w2; sum += xin(c, t + 3u) * w3;
58
+ for (var d = 1u; d < 4u; d++) { if (t + d < qd.T) { sum += mixed[(t + d) * p.n + c] * wr[c * 3u + d - 1u]; } }
59
+ y[t * p.n + c] = sum / (1.0 + exp(-sum));
60
+ }
61
+ }`,
62
+ // L2-normalised q and k (x / sqrt(Σx² + eps)) for (key head wg.x, token wg.y), into one buffer: q rows [T][kH·dS]
63
+ // from 0, k rows from kOff (one buffer keeps bdGdnB within 10 storage bindings).
64
+ bdQkNormB: `
65
+ struct P { kH: u32, dS: u32, keyDim: u32, convDim: u32, kOff: u32, eps: f32 }
66
+ @group(0) @binding(0) var<storage, read> cv: array<f32>;
67
+ @group(0) @binding(1) var<storage, read_write> qk: array<f32>;
68
+ @group(0) @binding(2) var<uniform> p: P;
69
+ var<workgroup> rq: array<f32, 128>;
70
+ var<workgroup> rk: array<f32, 128>;
71
+ @compute @workgroup_size(128) fn main(@builtin(workgroup_id) wg: vec3<u32>, @builtin(local_invocation_id) l: vec3<u32>) {
72
+ let kh = wg.x; let t = wg.y; let j = l.x; let cb = t * p.convDim;
73
+ let qv = cv[cb + kh * p.dS + j]; let kv = cv[cb + p.keyDim + kh * p.dS + j];
74
+ rq[j] = qv * qv; rk[j] = kv * kv; workgroupBarrier();
75
+ for (var s = 64u; s > 0u; s >>= 1u) { if (j < s) { rq[j] += rq[j + s]; rk[j] += rk[j + s]; } workgroupBarrier(); }
76
+ let o = t * p.keyDim + kh * p.dS + j;
77
+ qk[o] = qv / sqrt(rq[0] + p.eps); qk[p.kOff + o] = kv / sqrt(rk[0] + p.eps);
78
+ }`,
79
+ // bd_gdn_local_memory for (value head wg.x, query token wg.y); thread j = value row j. k-head = h % kH (the GGUF's
80
+ // tiled head order, as gdnStep). Logits over the block's keys: (qn·kn)·logitScale[h]; the history slot's logit:
81
+ // bias[h] + c[h]·log(T/32); its value: the clean state read with qn·scale, times exp(g), g = softplus(a + dt)·A.
82
+ bdGdnB: `
83
+ struct P { kH: u32, dS: u32, keyDim: u32, convDim: u32, vH: u32, kOff: u32, scale: f32 }
84
+ ${QB}
85
+ @group(0) @binding(0) var<storage, read> qk: array<f32>;
86
+ @group(0) @binding(1) var<storage, read> cv: array<f32>;
87
+ @group(0) @binding(2) var<storage, read> al: array<f32>;
88
+ @group(0) @binding(3) var<storage, read> dt: array<f32>;
89
+ @group(0) @binding(4) var<storage, read> aa: array<f32>;
90
+ @group(0) @binding(5) var<storage, read> ls: array<f32>;
91
+ @group(0) @binding(6) var<storage, read> hb: array<f32>;
92
+ @group(0) @binding(7) var<storage, read> hc: array<f32>;
93
+ @group(0) @binding(8) var<storage, read> M: array<f32>;
94
+ @group(0) @binding(9) var<storage, read_write> o: array<f32>;
95
+ @group(0) @binding(10) var<uniform> p: P;
96
+ @group(0) @binding(11) var<uniform> qd: Q;
97
+ var<workgroup> qs: array<f32, 128>;
98
+ var<workgroup> wt: array<f32, 33>;
99
+ fn log1p_(u: f32) -> f32 { let y = 1.0 + u; return log(y) - ((y - 1.0) - u) / y; }
100
+ @compute @workgroup_size(128) fn main(@builtin(workgroup_id) wg: vec3<u32>, @builtin(local_invocation_id) l: vec3<u32>) {
101
+ let h = wg.x; let t = wg.y; let j = l.x; let dS = p.dS; let kh = h % p.kH; let T = qd.T;
102
+ let qb = t * p.keyDim + kh * dS;
103
+ qs[j] = qk[qb + j];
104
+ workgroupBarrier();
105
+ if (j < T) {
106
+ let kb = j * p.keyDim + kh * dS; var acc = 0.0;
107
+ for (var i = 0u; i < dS; i++) { acc += qs[i] * qk[p.kOff + kb + i]; }
108
+ wt[j] = acc * ls[h];
109
+ }
110
+ if (j == 32u) { wt[T] = hb[h] + hc[h] * log(f32(T) / 32.0); }
111
+ workgroupBarrier();
112
+ if (j == 0u) {
113
+ var mx = wt[0];
114
+ for (var s = 1u; s <= T; s++) { mx = max(mx, wt[s]); }
115
+ var sum = 0.0;
116
+ for (var s = 0u; s <= T; s++) { let e = exp(wt[s] - mx); wt[s] = e; sum += e; }
117
+ for (var s = 0u; s <= T; s++) { wt[s] = wt[s] / sum; }
118
+ }
119
+ workgroupBarrier();
120
+ let row = (h * dS + j) * dS; var rd = 0.0;
121
+ for (var i = 0u; i < dS; i++) { rd += M[row + i] * (qs[i] * p.scale); }
122
+ let x = al[t * p.vH + h] + dt[h];
123
+ let sp = select(log1p_(exp(x)), x, x > 20.0);
124
+ var acc = wt[T] * (rd * exp(sp * aa[h]));
125
+ var loc = 0.0;
126
+ for (var s = 0u; s < T; s++) { loc += wt[s] * cv[s * p.convDim + 2u * p.keyDim + h * dS + j]; }
127
+ o[t * p.vH * dS + h * dS + j] = acc + loc;
128
+ }`,
129
+ // Scores for T queries over all S = pos0 + T keys, unmasked: the block sees the prefix and itself both ways.
130
+ attnScoreAllB: `enable f16;
131
+ struct P { heads: u32, kvHeads: u32, hd: u32, scale: f32 }
132
+ ${QB}
133
+ @group(0) @binding(0) var<storage, read> q: array<f32>;
134
+ @group(0) @binding(1) var<storage, read> kc: array<f16>;
135
+ @group(0) @binding(2) var<storage, read_write> sc: array<f32>;
136
+ @group(0) @binding(3) var<uniform> p: P;
137
+ @group(0) @binding(4) var<uniform> qd: Q;
138
+ @compute @workgroup_size(256) fn main(@builtin(global_invocation_id) g: vec3<u32>, @builtin(num_workgroups) ng: vec3<u32>) {
139
+ let idx = g.y * ng.x * 256u + g.x; let S = qd.S; let T = qd.T;
140
+ if (idx >= p.heads * T * S) { return; }
141
+ let h = idx / (T * S); let rem = idx % (T * S); let t = rem / S; let pos = rem % S;
142
+ let kvh = h / (p.heads / p.kvHeads);
143
+ let qo = t * p.heads * p.hd + h * p.hd; let ko = pos * p.kvHeads * p.hd + kvh * p.hd;
144
+ var acc = 0.0;
145
+ for (var d = 0u; d < p.hd; d++) { acc += q[qo + d] * f32(kc[ko + d]); }
146
+ sc[idx] = acc * p.scale;
147
+ }`,
148
+ // coreset_torch, one workgroup, thread e = expert (E ≤ 256). The block's mass on expert e: Σ over tokens and their
149
+ // top-k picks of e of the pick's full-softmax probability; normalised, sorted (descending, ties by index), kept
150
+ // while the mass before it is under topP (the first always). Router logits of every other expert become -3.4e38,
151
+ // so topkB, run again, routes each token inside the subset (picks past the subset get weight 0).
152
+ coresetB: `
153
+ struct P { E: u32, K: u32, topP: f32 }
154
+ ${QB}
155
+ @group(0) @binding(0) var<storage, read_write> lg: array<f32>;
156
+ @group(0) @binding(1) var<storage, read> sel: array<u32>;
157
+ @group(0) @binding(2) var<uniform> p: P;
158
+ @group(0) @binding(3) var<uniform> qd: Q;
159
+ var<workgroup> red: array<f32, 256>;
160
+ var<workgroup> pr: array<f32, 256>;
161
+ var<workgroup> srt: array<f32, 256>;
162
+ var<workgroup> keep: array<u32, 256>;
163
+ var<workgroup> stat: array<f32, 2>;
164
+ @compute @workgroup_size(256) fn main(@builtin(local_invocation_id) l: vec3<u32>) {
165
+ let e = l.x; let E = p.E; let on = e < E;
166
+ var mine = 0.0;
167
+ for (var t = 0u; t < qd.T; t++) {
168
+ let v = select(${NEG}, lg[t * E + min(e, E - 1u)], on);
169
+ red[e] = v; workgroupBarrier();
170
+ for (var s = 128u; s > 0u; s >>= 1u) { if (e < s) { red[e] = max(red[e], red[e + s]); } workgroupBarrier(); }
171
+ if (e == 0u) { stat[0] = red[0]; } workgroupBarrier();
172
+ let ex = select(0.0, exp(v - stat[0]), on);
173
+ red[e] = ex; workgroupBarrier();
174
+ for (var s = 128u; s > 0u; s >>= 1u) { if (e < s) { red[e] += red[e + s]; } workgroupBarrier(); }
175
+ if (e == 0u) { stat[1] = red[0]; } workgroupBarrier();
176
+ for (var j = 0u; j < p.K; j++) { if (sel[t * 2u * p.K + j] == e) { mine += ex / stat[1]; } }
177
+ workgroupBarrier();
178
+ }
179
+ red[e] = mine; workgroupBarrier();
180
+ for (var s = 128u; s > 0u; s >>= 1u) { if (e < s) { red[e] += red[e + s]; } workgroupBarrier(); }
181
+ let tot = max(red[0], 1e-9);
182
+ let pe = select(-1.0, mine / tot, on);
183
+ pr[e] = pe; workgroupBarrier();
184
+ var rank = 0u;
185
+ for (var f = 0u; f < 256u; f++) { let pf = pr[f]; if (pf > pe || (pf == pe && f < e)) { rank++; } }
186
+ srt[rank] = pe; workgroupBarrier();
187
+ if (e == 0u) {
188
+ var cum = 0.0;
189
+ for (var r = 0u; r < E; r++) { keep[r] = select(0u, 1u, cum < p.topP || r == 0u); cum += srt[r]; }
190
+ }
191
+ workgroupBarrier();
192
+ if (on && keep[rank] == 0u) { for (var t = 0u; t < qd.T; t++) { lg[t * E + e] = ${NEG}; } }
193
+ }`,
194
+ // Per block row: the argmax with the mask id excluded (lowest id on a tie) and its softmax probability at T = 1
195
+ // over the rest of the vocabulary: out[2t] = id, out[2t + 1] = probability bits. One workgroup per row.
196
+ // A = -exp(A_log) in place: clover_moe GGUFs store ssm_a as A_log itself (checked against the BF16 checkpoint,
197
+ // 2026-10-30), where the qwen35moe files store -exp(A_log), which gdnStep and bdGdnB read.
198
+ negExp: `
199
+ struct P { n: u32 }
200
+ @group(0) @binding(0) var<storage, read_write> a: array<f32>;
201
+ @group(0) @binding(1) var<uniform> p: P;
202
+ @compute @workgroup_size(64) fn main(@builtin(global_invocation_id) gi: vec3<u32>) { if (gi.x < p.n) { a[gi.x] = -exp(a[gi.x]); } }`,
203
+ top1B: `
204
+ struct P { V: u32, mask: u32 }
205
+ @group(0) @binding(0) var<storage, read> lg: array<f32>;
206
+ @group(0) @binding(1) var<storage, read_write> out: array<u32>;
207
+ @group(0) @binding(2) var<uniform> p: P;
208
+ var<workgroup> rv: array<f32, 256>;
209
+ var<workgroup> ri: array<u32, 256>;
210
+ @compute @workgroup_size(256) fn main(@builtin(workgroup_id) wg: vec3<u32>, @builtin(local_invocation_id) l: vec3<u32>) {
211
+ let b = wg.x * p.V; let t = l.x;
212
+ var bv = ${NEG}; var bi = 0xffffffffu;
213
+ for (var i = t; i < p.V; i += 256u) { if (i != p.mask) { let v = lg[b + i]; if (v > bv) { bv = v; bi = i; } } }
214
+ rv[t] = bv; ri[t] = bi; workgroupBarrier();
215
+ for (var s = 128u; s > 0u; s >>= 1u) {
216
+ if (t < s) { let o = rv[t + s]; let oi = ri[t + s]; if (o > rv[t] || (o == rv[t] && oi < ri[t])) { rv[t] = o; ri[t] = oi; } }
217
+ workgroupBarrier();
218
+ }
219
+ let mx = rv[0]; let id = ri[0]; workgroupBarrier();
220
+ var sum = 0.0;
221
+ for (var i = t; i < p.V; i += 256u) { if (i != p.mask) { sum += exp(lg[b + i] - mx); } }
222
+ rv[t] = sum; workgroupBarrier();
223
+ for (var s = 128u; s > 0u; s >>= 1u) { if (t < s) { rv[t] += rv[t + s]; } workgroupBarrier(); }
224
+ if (t == 0u) { out[2u * wg.x] = id; out[2u * wg.x + 1u] = bitcast<u32>(1.0 / rv[0]); }
225
+ }`,
226
+ };
227
+
228
+ export class CloverMoeSsd extends Qwen35MoeSsd {
229
+ static archs = ['clover_moe'];
230
+
231
+ static async load(modelId = null, opts = {}) {
232
+ const source = opts.source || (opts.url ? { repo: null, file: opts.url.split('/').pop(), revision: null } : CLOVER_35B_A3B);
233
+ return super.load(modelId, { ...opts, source });
234
+ }
235
+
236
+ constructor(device, manifest, gguf, opts) {
237
+ super(device, manifest, gguf, opts);
238
+ const kv = gguf.kv, g = (k) => kv[`clover_moe.${k}`];
239
+ this.block = g('block_size') || 32;
240
+ if (this.block > 32) throw new Error(`bdGdnB holds at most 32 block keys, got a block of ${this.block}`);
241
+ this.coreset = g('moe_block_coreset_enabled') !== false;
242
+ this.coresetTopP = g('moe_block_coreset_top_p') ?? 0.9;
243
+ this.maskId = this.tokenizer.ids.get('<|mask|>');
244
+ if (this.maskId == null) throw new Error('no <|mask|> token in the vocabulary');
245
+ if (this.cfg.experts > 256) throw new Error('coresetB handles at most 256 experts');
246
+ // capture: per-layer block rows of the last preview (test/browser/clover-states.html, the oracle gate)
247
+ this.capture = !!opts.capture;
248
+ }
249
+
250
+ get kernels() {
251
+ if (!this._ckernels) this._ckernels = { ...super.kernels, ...PREVIEW_KERNELS };
252
+ return this._ckernels;
253
+ }
254
+ isGuarded(name) { return !(name in PREVIEW_KERNELS) && super.isGuarded(name); }
255
+
256
+ async uploadDense(onProgress) {
257
+ await super.uploadDense(onProgress);
258
+ const s = this.cfg.ssm, enc = this.device.createCommandEncoder(), pass = enc.beginComputePass(), u = this.uniform([s.vHeads]);
259
+ for (let l = 0; l < this.cfg.layers; l++) {
260
+ if ((l + 1) % this.cfg.attnInterval === 0) continue;
261
+ this.dispatch(pass, 'negExp', this.bind('negExp', [this.w[`blk.${l}.ssm_a`].raw, u]), Math.ceil(s.vHeads / 64));
262
+ }
263
+ pass.end(); this.device.queue.submit([enc.finish()]);
264
+ await this.device.queue.onSubmittedWorkDone();
265
+ }
266
+
267
+ initBuffers() {
268
+ super.initBuffers();
269
+ if (!this.batchPrefill) throw new Error('CloverMoeSsd needs batchPrefill');
270
+ if (this.chunkTokens < this.block) throw new Error(`a chunk of ${this.chunkTokens} tokens < block ${this.block}`);
271
+ this.initPreview();
272
+ }
273
+
274
+ // Buffers and bind groups of the preview pass. It shares the chunk buffers (this.b) with prefillChunk: the two
275
+ // never run at once (both hold the engine lock through forward()).
276
+ initPreview() {
277
+ const c = this.cfg, s = c.ssm, B = this.block, b = this.b, W = this.w, H = c.hidden;
278
+ const keyDim = s.kHeads * s.dState, convDim = 2 * keyDim + s.dInner;
279
+ const A = (n, cat = 'batch') => this.buffer(n * 4, STORAGE | COPY_SRC | COPY_DST, cat);
280
+ this.p = { qk: A(2 * B * keyDim), logits: A(B * c.vocab), top1: A(2 * B) };
281
+ this.rbTop1 = this.buffer(2 * B * 4, MAP_READ | COPY_DST, 'batch');
282
+ this.rbBlockLogits = this.buffer(B * c.vocab * 4, MAP_READ | COPY_DST, 'batch');
283
+ if (this.capture) { this.p.cap = A(c.layers * B * H); this.rbCap = this.buffer(c.layers * B * H * 4, MAP_READ | COPY_DST, 'batch'); }
284
+ const U = (w) => this.uniform(w), qb = this.qb, raw = (name) => {
285
+ const t = W[name]; if (!t || !t.raw) throw new Error(`missing F32 tensor ${name}`); return t.raw;
286
+ };
287
+ const u = {
288
+ conv: U([convDim]), qkn: U([s.kHeads, s.dState, keyDim, convDim, B * keyDim, { f: c.eps }]),
289
+ gdn: U([s.kHeads, s.dState, keyDim, convDim, s.vHeads, B * keyDim, { f: 1 / Math.sqrt(s.dState) }]),
290
+ att: U([c.heads, c.kvHeads, c.headDim, { f: 1 / Math.sqrt(c.headDim) }]),
291
+ core: U([c.experts, c.topK, { f: this.coresetTopP }]), lm: U([c.vocab, H]), top1: U([c.vocab, this.maskId]),
292
+ };
293
+ for (let l = 0; l < c.layers; l++) {
294
+ const n = (t) => `blk.${l}.${t}`, g = {};
295
+ if (this.recurrent[l]) {
296
+ g.conv = this.bind('bdConvB', [b.mixed, raw(n('ssm_conv1d.weight')), raw(n('ssm_conv1d.weight_right')), this.conv[l], b.cv, u.conv, qb]);
297
+ g.qkn = this.bind('bdQkNormB', [b.cv, this.p.qk, u.qkn]);
298
+ g.gdn = this.bind('bdGdnB', [this.p.qk, b.cv, b.al, raw(n('ssm_dt.bias')), raw(n('ssm_a')),
299
+ raw(n('ssm_bd_logit_scale.weight')), raw(n('ssm_bd_history.bias')), raw(n('ssm_bd_history.weight')), this.ssm[l], b.go, u.gdn, qb]);
300
+ } else {
301
+ g.score = this.bind('attnScoreAllB', [b.q, this.kc[l], b.sc, u.att, qb]);
302
+ }
303
+ g.core = this.bind('coresetB', [b.rl, b.sel, u.core, qb]);
304
+ this.layers[l].P = g;
305
+ }
306
+ this.gP = {
307
+ rmsOut: this.bind('rmsnormB', [b.x, raw('output_norm.weight'), b.xn, U([H, { f: c.eps }])]),
308
+ lm: this.bind(this.qk.mmT, [b.xn, W['output.weight'].q, W['output.weight'].s, this.p.logits, u.lm, qb]),
309
+ top1: this.bind('top1B', [this.p.logits, this.p.top1, u.top1]),
310
+ };
311
+ if (this.qk.mmT !== 'matmulQ4T' && this.qk.mmT !== 'matmulQ8T') throw new Error('the batched LM head needs a Q4_0 or Q8_0 output matrix');
312
+ if (W['output.weight'].q == null) throw new Error('the batched LM head needs a Q4_0 or Q8_0 output matrix');
313
+ }
314
+
315
+ // Layer l of the preview over the block's T rows, up to its routing (encodeTrunkB with the preview kernels).
316
+ encodePreviewTrunk(pass, l, T, S) {
317
+ const c = this.cfg, s = c.ssm, g = this.layers[l].B, gp = this.layers[l].P, d = (nm, gr, x, y, z) => this.dispatch(pass, nm, gr, x, y, z);
318
+ const H = c.hidden, QN = c.heads * c.headDim, kvn = c.kvHeads * c.headDim, convDim = 2 * s.kHeads * s.dState + s.dInner;
319
+ const tt = Math.ceil(T / 16), mm = (n) => Math.ceil(n / 16), QK = this.qk;
320
+ d('rmsnormB', g.rmsA, T);
321
+ if (this.recurrent[l]) {
322
+ d(QK.mmT, g.qkv, mm(convDim), tt); d(QK.mmT, g.z, mm(s.dInner), tt); d(QK.mmT, g.alpha, mm(s.vHeads), tt);
323
+ d('bdConvB', gp.conv, Math.ceil(convDim / 256));
324
+ d('bdQkNormB', gp.qkn, s.kHeads, T);
325
+ d('bdGdnB', gp.gdn, s.vHeads, T);
326
+ d('gdnNormGateB', g.norm, s.vHeads, T);
327
+ d(QK.mmT, g.out, mm(H), tt);
328
+ } else {
329
+ d(QK.mmT, g.q, mm(2 * QN), tt); d(QK.mmT, g.k, mm(kvn), tt); d(QK.mmT, g.v, mm(kvn), tt);
330
+ d('qkNormRopePB', g.ropeQ, c.heads, T); d('qkNormRopePB', g.ropeK, c.kvHeads, T);
331
+ d('kvStoreB', g.kv, Math.ceil(kvn / 256), T); // scratch rows pos0.. after the prefix
332
+ d('attnScoreAllB', gp.score, Math.ceil(c.heads * T * S / 256));
333
+ d('softmaxB', g.soft, c.heads * T);
334
+ d('attnOutPosB', g.attOut, Math.ceil(QN / 256), T);
335
+ d('sigmoidMulB', g.gate, Math.ceil(QN / 256), T);
336
+ d(QK.mmT, g.o, mm(H), tt);
337
+ }
338
+ d('addInPlaceB', g.addO, Math.ceil(H / 256), T);
339
+ d('rmsnormB', g.rmsF, T);
340
+ d('matmulF32T', g.router, c.experts, T);
341
+ d('topkB', g.topk, T);
342
+ if (this.coreset) { d('coresetB', gp.core, 1); d('topkB', g.topk, T); }
343
+ d(QK.mmT, g.shg, mm(c.shexpFf), tt); d(QK.mmT, g.shu, mm(c.shexpFf), tt);
344
+ d('siluMul2B', g.shact, Math.ceil(c.shexpFf / 256), T);
345
+ d(QK.mmT, g.shd, mm(H), tt);
346
+ d('matmulF32T', g.shgate, 1, T);
347
+ }
348
+
349
+ // One denoising pass over the block `ids` (mask slots hold this.maskId) at positions position..position+T-1.
350
+ // Reads the committed cache and writes nothing to it (KV rows past the prefix are scratch). want: 'all' → T × vocab
351
+ // logits (Float32Array, row t scores slot t); 'top1' → { token: Int32Array(T), conf: Float32Array(T) }, the argmax
352
+ // over the vocabulary without the mask id and its probability at T = 1.
353
+ async preview(ids, want = 'all') {
354
+ const c = this.cfg, dev = this.device, K = c.topK, H = c.hidden, T = ids.length, pos0 = this.position, S = pos0 + T;
355
+ if (T < 1 || T > this.block || this.block % T) throw new Error(`a preview block of ${T} must divide ${this.block}`);
356
+ if (S > this.maxCtx) throw this.contextFull();
357
+ const tStart = performance.now();
358
+ dev.queue.writeBuffer(this.a.stop, 0, new Uint32Array(4));
359
+ dev.queue.writeBuffer(this.b.ids, 0, Uint32Array.from(ids));
360
+ dev.queue.writeBuffer(this.qb, 0, new Uint32Array([T, pos0, S, 0]));
361
+ dev.queue.writeBuffer(this.b.rope, 0, this.ropeRows(this.textPositions(T), T));
362
+ const selBytes = T * 2 * K * 4, rows = T * H * 4;
363
+ let enc = dev.createCommandEncoder(), pass = enc.beginComputePass();
364
+ this.dispatch(pass, this.qk.embB, this.gB.embed, Math.ceil(H / 256), T);
365
+ this.encodePreviewTrunk(pass, 0, T, S);
366
+ pass.end();
367
+ enc.copyBufferToBuffer(this.b.sel, 0, this.rbSelB, 0, selBytes);
368
+ dev.queue.submit([enc.finish()]);
369
+ const units = new Set();
370
+ for (let l = 0; l < c.layers; l++) {
371
+ const t0 = performance.now();
372
+ await this.rbSelB.mapAsync(1, 0, selBytes);
373
+ this.counters.gpuWaitMs += performance.now() - t0;
374
+ const sel = new Uint32Array(this.rbSelB.getMappedRange(0, selBytes).slice(0)), wts = new Float32Array(sel.buffer);
375
+ this.rbSelB.unmap();
376
+ // Picks with weight 0 (a token routed past the coreset) need no load: they take any loaded slot.
377
+ const pairs = new Uint32Array(T * K), live = new Set();
378
+ for (let t = 0; t < T; t++) for (let j = 0; j < K; j++) {
379
+ const e = sel[t * 2 * K + j]; pairs[t * K + j] = e;
380
+ if (wts[t * 2 * K + K + j] > 0) live.add(e);
381
+ }
382
+ const unique = [...live];
383
+ for (const e of unique) units.add(l * c.experts + e);
384
+ const tq = performance.now();
385
+ const slotsU = await this.xs.ensure(l, unique);
386
+ this.counters.ensureMs += performance.now() - tq;
387
+ const slotOf = new Map(unique.map((e, i) => [e, slotsU[i]]));
388
+ dev.queue.writeBuffer(this.b.slots, 0, pairs.map((e) => slotOf.get(e) ?? slotsU[0]));
389
+ enc = dev.createCommandEncoder(); pass = enc.beginComputePass();
390
+ this.encodeExpertsB(pass, l, T);
391
+ pass.end();
392
+ if (this.capture) enc.copyBufferToBuffer(this.b.x, 0, this.p.cap, l * this.block * H * 4, rows);
393
+ if (l + 1 < c.layers) {
394
+ pass = enc.beginComputePass();
395
+ this.encodePreviewTrunk(pass, l + 1, T, S);
396
+ pass.end();
397
+ enc.copyBufferToBuffer(this.b.sel, 0, this.rbSelB, 0, selBytes);
398
+ } else {
399
+ pass = enc.beginComputePass();
400
+ this.dispatch(pass, 'rmsnormB', this.gP.rmsOut, T);
401
+ this.dispatch(pass, this.qk.mmT, this.gP.lm, Math.ceil(c.vocab / 16), Math.ceil(T / 16));
402
+ if (want === 'top1') this.dispatch(pass, 'top1B', this.gP.top1, T);
403
+ pass.end();
404
+ if (want === 'top1') enc.copyBufferToBuffer(this.p.top1, 0, this.rbTop1, 0, 2 * T * 4);
405
+ else enc.copyBufferToBuffer(this.p.logits, 0, this.rbBlockLogits, 0, T * c.vocab * 4);
406
+ }
407
+ dev.queue.submit([enc.finish()]);
408
+ this.xs.release(slotsU);
409
+ }
410
+ let result;
411
+ if (want === 'top1') {
412
+ await this.rbTop1.mapAsync(1, 0, 2 * T * 4);
413
+ const u = new Uint32Array(this.rbTop1.getMappedRange(0, 2 * T * 4).slice(0)), f = new Float32Array(u.buffer);
414
+ this.rbTop1.unmap();
415
+ result = { token: new Int32Array(T), conf: new Float32Array(T) };
416
+ for (let t = 0; t < T; t++) { result.token[t] = u[2 * t]; result.conf[t] = f[2 * t + 1]; }
417
+ } else {
418
+ await this.rbBlockLogits.mapAsync(1, 0, T * c.vocab * 4);
419
+ result = new Float32Array(this.rbBlockLogits.getMappedRange(0, T * c.vocab * 4).slice(0));
420
+ this.rbBlockLogits.unmap();
421
+ }
422
+ this.counters.previews = (this.counters.previews || 0) + 1;
423
+ this.counters.previewExperts = (this.counters.previewExperts || 0) + units.size;
424
+ this.counters.previewMs = (this.counters.previewMs || 0) + performance.now() - tStart;
425
+ return result;
426
+ }
427
+
428
+ // The last preview's per-layer residual rows: layers × T × hidden (capture: true only).
429
+ async readCapture(T = this.block) {
430
+ const c = this.cfg, n = c.layers * this.block * c.hidden * 4, enc = this.device.createCommandEncoder();
431
+ enc.copyBufferToBuffer(this.p.cap, 0, this.rbCap, 0, n);
432
+ this.device.queue.submit([enc.finish()]);
433
+ await this.rbCap.mapAsync(1);
434
+ const all = new Float32Array(this.rbCap.getMappedRange().slice(0));
435
+ this.rbCap.unmap();
436
+ return Array.from({ length: c.layers }, (_, l) => all.subarray(l * this.block * c.hidden, (l * this.block + T) * c.hidden));
437
+ }
438
+
439
+ // The callback kohra's denoise module drives (one call at a time, under the engine lock):
440
+ // commit: appends ids at positions [start, start + ids.length); start is the cache length, or 0 for a new prompt
441
+ // (which reuses the cached prefix it shares). Returns null (Clover commits never need logits).
442
+ // preview: a denoising pass over the block at the cache end; see preview().
443
+ async forward(args) {
444
+ const release = await this.acquire();
445
+ try { return await this.forwardLocked(args); } catch (e) { throw this.stepError(e); } finally { release(); }
446
+ }
447
+ async forwardLocked({ mode, start, ids, wantLogits = mode === 'preview' ? 'all' : 'none' }) {
448
+ ids = Array.from(ids);
449
+ if (mode === 'preview') {
450
+ if (start !== this.position) throw new Error(`preview at ${start}, but the cache holds ${this.position} tokens`);
451
+ if (wantLogits !== 'all' && wantLogits !== 'top1') throw new Error(`preview returns 'all' or 'top1', not ${wantLogits}`);
452
+ return await this.preview(ids, wantLogits);
453
+ }
454
+ if (mode !== 'commit') throw new Error(`forward mode ${mode}`);
455
+ if (wantLogits !== 'none') throw new Error('Clover commits return no logits');
456
+ if (start === 0) { await this.prefillSegments([{ ids }], 'none', { sysEnd: this.checkpointEnd(ids) }); return null; }
457
+ if (start !== this.position) throw new Error(`commit at ${start}, but the cache holds ${this.position} tokens`);
458
+ for (let i = 0; i < ids.length; i += this.chunkTokens) {
459
+ const chunk = ids.slice(i, i + this.chunkTokens);
460
+ if (chunk.length < 2) await this.step(chunk[0], 'none'); else await this.prefillChunk(chunk, 'none');
461
+ }
462
+ return null;
463
+ }
464
+
465
+ // A chat turn (generate() holds the engine lock around it): kohra's denoise loop with Clover's decoding rules
466
+ // (PRESETS.clover: threshold 0.95, the reference generation config's penalties), yielding text block by block.
467
+ // opts beyond maxNewTokens / signal / enableThinking go to denoise() (e.g. penalties: null, top1: true).
468
+ async *generateTurn(messages, { maxNewTokens = 512, signal, enableThinking = true, ...loop } = {}) {
469
+ const promptIds = this.tokenizer.encode(this.chatPrompt(messages, { enableThinking }));
470
+ const blocks = [], out = [];
471
+ let wake = null, done = false, failed = null, stop = false;
472
+ const STOP = new Error('stopped');
473
+ const forward = (a) => { if (stop || (signal && signal.aborted)) throw STOP; return this.forwardLocked(a); };
474
+ const run = denoise({ ...PRESETS.clover, top1: true, ...loop, forward, promptIds, maxNewTokens, maskId: this.maskId, blockSize: this.block,
475
+ onBlock: ({ tokens }) => { blocks.push(tokens); if (wake) wake(); } })
476
+ .then((r) => { this.lastDenoise = { previews: r.previews, commits: r.commits, seconds: r.seconds, tokens: r.tokenIds.length }; },
477
+ (e) => { if (e !== STOP) failed = e; })
478
+ .finally(() => { done = true; if (wake) wake(); });
479
+ try {
480
+ for (;;) {
481
+ while (blocks.length) {
482
+ for (const token of blocks.shift()) { out.push(token); yield { text: this.tokenizer.decode(out), token, tokens: out.length }; }
483
+ }
484
+ if (done) break;
485
+ await new Promise((r) => { wake = r; });
486
+ wake = null;
487
+ }
488
+ if (failed) throw failed;
489
+ } finally { stop = true; await run; }
490
+ }
491
+
492
+ stats() {
493
+ const n = Math.max(1, this.counters.previews || 0);
494
+ return { ...super.stats(), previews: this.counters.previews || 0, previewMsEach: (this.counters.previewMs || 0) / n, expertsPerPreview: (this.counters.previewExperts || 0) / n };
495
+ }
496
+ }
497
+
498
+ export { PREVIEW_KERNELS as CLOVER_PREVIEW_KERNELS };
classic/engines/qwen35_moe_ssd.js CHANGED
@@ -741,6 +741,7 @@ function guarded(src) {
741
  }
742
 
743
  export class Qwen35MoeSsd extends Qwen3MoeSsd {
 
744
  // Without a url, the pinned Hugging Face file; with one (a local serve, a layer cut), the OPFS
745
  // directory is named after the file, never after rung 2a's model.
746
  static async load(modelId = null, opts = {}) {
@@ -751,7 +752,7 @@ export class Qwen35MoeSsd extends Qwen3MoeSsd {
751
  constructor(device, manifest, gguf, opts) {
752
  super(device, manifest, gguf, opts);
753
  const c = this.cfg;
754
- if (c.arch !== 'qwen35moe') throw new Error(`Qwen35MoeSsd needs a qwen35moe GGUF, got ${c.arch}`);
755
  if (c.ssm.dConv !== 4) throw new Error(`gdnConv assumes a 4-tap conv, got ${c.ssm.dConv}`);
756
  if (c.ssm.dState !== 128 || c.ssm.dInner !== c.ssm.vHeads * c.ssm.dState) throw new Error('gdnStep assumes 128-wide heads');
757
  // Weight type: Q8_0 (Qwen3.6, MiniCPM-V Q8_0) or Q4_0 (MiniCPM-V Q4_0: Q4_0 matrices and experts, Q6_K output).
 
741
  }
742
 
743
  export class Qwen35MoeSsd extends Qwen3MoeSsd {
744
+ static archs = ['qwen35moe'];
745
  // Without a url, the pinned Hugging Face file; with one (a local serve, a layer cut), the OPFS
746
  // directory is named after the file, never after rung 2a's model.
747
  static async load(modelId = null, opts = {}) {
 
752
  constructor(device, manifest, gguf, opts) {
753
  super(device, manifest, gguf, opts);
754
  const c = this.cfg;
755
+ if (!this.constructor.archs.includes(c.arch)) throw new Error(`${this.constructor.name} needs a ${this.constructor.archs.join(' or ')} GGUF, got ${c.arch}`);
756
  if (c.ssm.dConv !== 4) throw new Error(`gdnConv assumes a 4-tap conv, got ${c.ssm.dConv}`);
757
  if (c.ssm.dState !== 128 || c.ssm.dInner !== c.ssm.vHeads * c.ssm.dState) throw new Error('gdnStep assumes 128-wide heads');
758
  // Weight type: Q8_0 (Qwen3.6, MiniCPM-V Q8_0) or Q4_0 (MiniCPM-V Q4_0: Q4_0 matrices and experts, Q6_K output).
classic/engines/qwen3_moe_ssd.js CHANGED
@@ -43,12 +43,13 @@ export { FORMAT as STORE_FORMAT, OPFS_ROOT as STORE_ROOT };
43
 
44
  // ── GGUF ────────────────────────────────────────────────────────────────────
45
  // qwen3moe (rung 2a) and qwen35moe (Qwen3.5/3.6 MoE, rung 2b: qwen35_moe_ssd.js) share the
46
- // expert layout; qwen35moe adds the Gated DeltaNet (ssm.*), partial RoPE and a shared expert.
 
47
  export function configFromGguf(kv) {
48
  const a = kv['general.architecture'];
49
- if (a !== 'qwen3moe' && a !== 'qwen35moe') throw new Error(`expected a qwen3moe or qwen35moe GGUF, got ${a}`);
50
  const g = (k) => kv[`${a}.${k}`];
51
- const extra = a !== 'qwen35moe' ? {} : {
52
  ropeDims: g('rope.dimension_count'), shexpFf: g('expert_shared_feed_forward_length'),
53
  attnInterval: g('full_attention_interval') || 4,
54
  ssm: { dConv: g('ssm.conv_kernel'), dInner: g('ssm.inner_size'), dState: g('ssm.state_size'), vHeads: g('ssm.time_step_rank'), kHeads: g('ssm.group_count') },
 
43
 
44
  // ── GGUF ────────────────────────────────────────────────────────────────────
45
  // qwen3moe (rung 2a) and qwen35moe (Qwen3.5/3.6 MoE, rung 2b: qwen35_moe_ssd.js) share the
46
+ // expert layout; qwen35moe adds the Gated DeltaNet (ssm.*), partial RoPE and a shared expert. clover_moe (Clover,
47
+ // block diffusion on Qwen3.6: clover_moe_ssd.js) is qwen35moe plus per-DeltaNet-layer preview tensors.
48
  export function configFromGguf(kv) {
49
  const a = kv['general.architecture'];
50
+ if (a !== 'qwen3moe' && a !== 'qwen35moe' && a !== 'clover_moe') throw new Error(`expected a qwen3moe, qwen35moe or clover_moe GGUF, got ${a}`);
51
  const g = (k) => kv[`${a}.${k}`];
52
+ const extra = a === 'qwen3moe' ? {} : {
53
  ropeDims: g('rope.dimension_count'), shexpFf: g('expert_shared_feed_forward_length'),
54
  attnInterval: g('full_attention_interval') || 4,
55
  ssm: { dConv: g('ssm.conv_kernel'), dInner: g('ssm.inner_size'), dState: g('ssm.state_size'), vHeads: g('ssm.time_step_rank'), kHeads: g('ssm.group_count') },
classic/engines/vendor/kohra-denoise.js ADDED
@@ -0,0 +1,308 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // denoise.js: the block-diffusion decoding loop, with no runtime inside.
2
+ //
3
+ // The caller supplies one async forward pass:
4
+ //
5
+ // forward({ mode, start, ids, wantLogits }) -> Float32Array | null
6
+ //
7
+ // mode 'preview' the denoise pass over the current block: reads the committed cache, writes
8
+ // nothing, returns [ids.length * V] fp32 logits (row-major). wantLogits = 'all'.
9
+ // mode 'commit' the causal pass: appends ids at absolute positions [start, start + ids.length).
10
+ // Prefill is the first commit; commits arrive in order with no gaps.
11
+ // wantLogits 'last' returns the last position's V logits; 'none' returns null.
12
+ // Optional fast path (option top1): a greedy, penalty-free preview asks wantLogits 'top1' and gets
13
+ // { token: Int32Array(n), conf: Float32Array(n) }: the per-row argmax with the mask id excluded and
14
+ // its softmax probability at T = 1 (mask logit at -inf). A runtime without it throws, and the loop
15
+ // falls back to 'all' for the rest of the run.
16
+ //
17
+ // Logits are same-position: row i scores slot start + i (not the next token).
18
+ //
19
+ // `ids` is an Int32Array; masked slots hold `maskId`. The loop reads the returned array during the
20
+ // step and never keeps it, so the caller owns buffers and disposal.
21
+ // kohra's ONNX path (kohra.js) and diskformer's clover_moe engine both drive this loop.
22
+
23
+ const DEFAULTS = {
24
+ maskId: 151669,
25
+ eosIds: [], // stop tokens; the response is trimmed before the first one
26
+ blockSize: 32,
27
+ maxNewTokens: 128,
28
+ threshold: null, // number: fill every masked position past it per preview (min 1).
29
+ // null: fixed reveal schedule over `steps` previews.
30
+ strict: false, // threshold compare: true uses conf > threshold, false conf >= threshold
31
+ steps: 128, // fixed-schedule mode only: total previews, split evenly over blocks
32
+ maxStepsPerBlock: null, // threshold mode: throw if a block still has masks after this many previews
33
+ temperature: 0, // 0 = argmax; > 0 = sample softmax(logits / temperature)
34
+ topK: 0, // sampling filters (temperature > 0 only); 0 / 1 = off
35
+ topP: 1,
36
+ remasking: 'low_confidence', // or 'random'
37
+ suppressMask: false, // set the mask token's logit to -inf before scoring
38
+ seed: false, // a commit's last logit picks the next block's first token
39
+ alignToGrid: false, // blocks on the absolute pos // blockSize grid (first block holds the prompt tail)
40
+ fullLastBlock: false, // denoise the last block at full blockSize, then trim to maxNewTokens
41
+ stopAtEos: false, // stop after a resolved block that holds a stop token
42
+ penalties: null, // see applyPenalties
43
+ top1: false, // try the 'top1' preview fast path when greedy and penalty-free
44
+ onStep: null, // ({ canvas, P, start, end, fresh, block, numBlocks, previews, elapsedMs })
45
+ onBlock: null, // ({ tokens, block }) after each resolved block (response tokens only)
46
+ yieldEvery: 1, // release the event loop every N previews
47
+ };
48
+
49
+ // Clover-35B-A3B-Preview: generation_config.json plus the denoising config in modeling_clover_moe.py.
50
+ export const PRESETS = {
51
+ clover: {
52
+ maskId: 151669,
53
+ eosIds: [248046, 248044],
54
+ blockSize: 32,
55
+ threshold: 0.95,
56
+ strict: true,
57
+ maxStepsPerBlock: 32,
58
+ suppressMask: true,
59
+ fullLastBlock: true,
60
+ stopAtEos: true,
61
+ penalties: {
62
+ presence: 0.9, frequency: 0, repetition: 1.05,
63
+ length: 10, lengthStart: 8192, lengthWindow: 8192,
64
+ thinkEnd: 10, thinkEndStart: 4096, thinkEndWindow: 4096, thinkEndId: 248069,
65
+ stopIds: [248044, 248046], // eos ids plus pad (248044)
66
+ },
67
+ },
68
+ };
69
+
70
+ // Cooperative yield that lets the UI paint without waiting on the compositor.
71
+ // requestAnimationFrame stalls in a hidden tab; a MessageChannel macrotask always fires.
72
+ export const yieldToEventLoop = (() => {
73
+ if (typeof MessageChannel === 'undefined') {
74
+ return () => new Promise((resolve) => setTimeout(resolve, 0));
75
+ }
76
+ const ch = new MessageChannel();
77
+ const queue = [];
78
+ // ref/unref exist only in Node: hold the port open while a yield is pending, so an
79
+ // idle port never keeps the process alive and a pending one never lets it exit.
80
+ ch.port1.onmessage = () => { queue.shift()?.(); if (!queue.length) ch.port1.unref?.(); };
81
+ ch.port1.unref?.();
82
+ return () => new Promise((resolve) => { queue.push(resolve); ch.port1.ref?.(); ch.port2.postMessage(0); });
83
+ })();
84
+
85
+ // Per-block reveal counts for the linear-alpha (MDLM) scheduler: round remaining/(steps-i)
86
+ // per step, skip zero entries. Mirrors dllm get_num_transfer_tokens.
87
+ export function transferSchedule(nMasked, steps) {
88
+ const ks = [];
89
+ let remaining = nMasked;
90
+ for (let i = 0; i < steps && remaining > 0; i++) {
91
+ const k = Math.min(Math.round(remaining / (steps - i)), remaining);
92
+ if (k > 0) { ks.push(k); remaining -= k; }
93
+ }
94
+ return ks;
95
+ }
96
+
97
+ // Response-only penalties, in CloverPenaltyState.apply order (sampling_clover_moe.py):
98
+ // frequency, presence, repetition over committed response tokens, then the length ramp on
99
+ // stop ids and the think-end ramp. `counts` holds committed response tokens only; tokens still
100
+ // in the block never penalise themselves. Mutates rows of `logits` in place.
101
+ export function applyPenalties(logits, rows, V, { counts, length, block, maskId, p }) {
102
+ // Every operation rounds to fp32 the way PyTorch's fp32 kernels do, with scalars cast to fp32.
103
+ const fp = Math.fround;
104
+ const ramp = (row, start, window, strength) =>
105
+ fp(fp(Math.min(Math.max(fp(fp(length + row + 1 - start) / fp(window)), 0), 1)) * fp(strength));
106
+ let closeOpen = false;
107
+ if (p.thinkEnd) {
108
+ closeOpen = !counts.has(p.thinkEndId);
109
+ for (let i = 0; i < block.length && closeOpen; i++) {
110
+ if (block[i] !== maskId && block[i] === p.thinkEndId) closeOpen = false;
111
+ }
112
+ }
113
+ const freq = fp(p.frequency ?? 0), pres = fp(p.presence ?? 0), rep = fp(p.repetition ?? 1);
114
+ for (let r = 0; r < rows; r++) {
115
+ const off = r * V;
116
+ for (const [tok, n] of counts) {
117
+ let l = logits[off + tok];
118
+ if (freq) l = fp(l - fp(n * freq));
119
+ if (pres) l = fp(l - pres);
120
+ if (rep !== 1) l = fp(l < 0 ? l * rep : l / rep);
121
+ logits[off + tok] = l;
122
+ }
123
+ if (p.length && p.stopIds?.length) {
124
+ const add = ramp(r, p.lengthStart, p.lengthWindow, p.length);
125
+ for (const s of p.stopIds) logits[off + s] = fp(logits[off + s] + add);
126
+ }
127
+ if (closeOpen) {
128
+ logits[off + p.thinkEndId] = fp(logits[off + p.thinkEndId] + ramp(r, p.thinkEndStart, p.thinkEndWindow, p.thinkEnd));
129
+ }
130
+ }
131
+ }
132
+
133
+ // Pick a token for one logits row and its confidence (softmax at T = 1 of the same row).
134
+ function pick(logits, off, V, cfg) {
135
+ let maxL = -Infinity;
136
+ for (let v = 0; v < V; v++) if (logits[off + v] > maxL) maxL = logits[off + v];
137
+ let sumExp = 0;
138
+ for (let v = 0; v < V; v++) sumExp += Math.exp(logits[off + v] - maxL);
139
+ let best = 0;
140
+ if (cfg.temperature > 0) {
141
+ const keep = samplingFilter(logits, off, V, cfg);
142
+ let bestScore = -Infinity;
143
+ for (let v = 0; v < V; v++) {
144
+ if (keep && !keep(logits[off + v], v)) continue;
145
+ // Gumbel-max on logits / temperature == a draw from softmax(logits / temperature).
146
+ const s = logits[off + v] / cfg.temperature - Math.log(-Math.log(cfg.random() + 1e-12) + 1e-12);
147
+ if (s > bestScore) { bestScore = s; best = v; }
148
+ }
149
+ } else {
150
+ for (let v = 1; v < V; v++) if (logits[off + v] > logits[off + best]) best = v;
151
+ }
152
+ return { tok: best, conf: Math.exp(logits[off + best] - maxL) / sumExp };
153
+ }
154
+
155
+ // top-k then top-p over logits / temperature, as predict_with_confidence does. Returns null when off.
156
+ function samplingFilter(logits, off, V, cfg) {
157
+ const k = cfg.topK > 0 && cfg.topK < V ? cfg.topK : 0;
158
+ const topP = cfg.topP != null && cfg.topP < 1 ? cfg.topP : 0;
159
+ if (!k && !topP) return null;
160
+ const order = Array.from({ length: V }, (_, v) => v).sort((a, b) => logits[off + b] - logits[off + a]);
161
+ let n = k || V;
162
+ if (topP) {
163
+ const T = cfg.temperature, top = logits[off + order[0]] / T;
164
+ let z = 0;
165
+ for (let i = 0; i < n; i++) z += Math.exp(logits[off + order[i]] / T - top);
166
+ let cum = 0;
167
+ for (let i = 0; i < n; i++) {
168
+ const pr = Math.exp(logits[off + order[i]] / T - top) / z;
169
+ if (cum > topP) { n = i; break; } // drop once the mass before this token passes top_p
170
+ cum += pr;
171
+ }
172
+ }
173
+ const allowed = new Set(order.slice(0, Math.max(n, 1)));
174
+ return (_, v) => allowed.has(v);
175
+ }
176
+
177
+ // Run the loop. `promptIds` is an array of token ids. Returns
178
+ // { tokenIds, stopToken, canvas, P, previews, commits, seconds }.
179
+ export async function denoise({ forward, promptIds, ...options }) {
180
+ const cfg = { random: Math.random, ...DEFAULTS, ...options };
181
+ const MASK = cfg.maskId, bs = cfg.blockSize;
182
+ const eos = new Set(cfg.eosIds);
183
+ const P = promptIds.length;
184
+ const maxNew = cfg.maxNewTokens;
185
+
186
+ // Blocks to denoise, left to right. Prompt-aligned: blocks start at P. Grid-aligned:
187
+ // blocks live on pos // blockSize, so the first one holds the prompt tail.
188
+ const first = cfg.alignToGrid ? Math.floor(P / bs) * bs : P;
189
+ const limit = P + maxNew;
190
+ const blocks = [];
191
+ for (let s = first; s < limit; s += bs) blocks.push([s, cfg.fullLastBlock ? s + bs : Math.min(s + bs, limit)]);
192
+ const numBlocks = blocks.length;
193
+ const T = numBlocks ? blocks[numBlocks - 1][1] : P;
194
+ const stepsPerBlock = Math.ceil(cfg.steps / Math.max(numBlocks, 1));
195
+
196
+ const canvas = new Int32Array(Math.max(T, P)).fill(MASK);
197
+ canvas.set(promptIds, 0);
198
+
199
+ const counts = new Map(); // committed response tokens, for penalties
200
+ let responseLength = 0;
201
+ let previews = 0, commits = 0;
202
+ let top1 = cfg.top1 && !(cfg.temperature > 0) && !cfg.penalties && cfg.remasking !== 'random';
203
+ const t0 = performance.now();
204
+
205
+ const commit = async (s, e, next) => {
206
+ const wantLogits = cfg.seed && next ? 'last' : 'none';
207
+ const out = await forward({ mode: 'commit', start: s, ids: canvas.slice(s, e), wantLogits });
208
+ commits++;
209
+ if (wantLogits === 'last' && canvas[next] === MASK) {
210
+ const V = out.length;
211
+ let best = 0;
212
+ for (let v = 1; v < V; v++) if (out[v] > out[best]) best = v;
213
+ canvas[next] = best;
214
+ }
215
+ };
216
+
217
+ if (numBlocks && first > 0) await commit(0, first, first); // prefill
218
+
219
+ let stopToken = null, genEnd = limit;
220
+ for (let b = 0; b < numBlocks; b++) {
221
+ const [start, end] = blocks[b];
222
+ const masked = () => { let n = 0; for (let i = start; i < end; i++) if (canvas[i] === MASK) n++; return n; };
223
+ const nMasked = masked();
224
+ const schedule = cfg.threshold == null ? transferSchedule(nMasked, stepsPerBlock) : null;
225
+ let step = 0;
226
+
227
+ while (masked() > 0) {
228
+ if (schedule && step >= schedule.length) break;
229
+ if (!schedule && cfg.maxStepsPerBlock != null && step >= cfg.maxStepsPerBlock) {
230
+ throw new Error(`denoise: block ${b} still has masks after ${step} previews`);
231
+ }
232
+ const ids = canvas.slice(start, end);
233
+ const cand = [];
234
+ let fast = null;
235
+ if (top1) {
236
+ try { fast = await forward({ mode: 'preview', start, ids, wantLogits: 'top1' }); } catch { top1 = false; }
237
+ }
238
+ if (fast) {
239
+ previews++;
240
+ for (let p = start; p < end; p++) {
241
+ if (canvas[p] === MASK) cand.push({ p, tok: fast.token[p - start], conf: fast.conf[p - start] });
242
+ }
243
+ } else {
244
+ const logits = await forward({ mode: 'preview', start, ids, wantLogits: 'all' });
245
+ previews++;
246
+ const rows = end - start, V = logits.length / rows;
247
+ if (cfg.suppressMask) for (let r = 0; r < rows; r++) logits[r * V + MASK] = -Infinity;
248
+ if (cfg.penalties) {
249
+ applyPenalties(logits, rows, V, { counts, length: responseLength, block: ids, maskId: MASK, p: cfg.penalties });
250
+ }
251
+ for (let p = start; p < end; p++) {
252
+ if (canvas[p] !== MASK) continue;
253
+ const { tok, conf } = pick(logits, (p - start) * V, V, cfg);
254
+ cand.push({ p, tok, conf: cfg.remasking === 'random' ? cfg.random() : conf });
255
+ }
256
+ }
257
+
258
+ let chosen;
259
+ if (schedule) {
260
+ cand.sort((x, y) => y.conf - x.conf);
261
+ chosen = cand.slice(0, schedule[step]);
262
+ } else {
263
+ const thr = cfg.threshold;
264
+ chosen = cand.filter((c) => (cfg.strict ? c.conf > thr : c.conf >= thr));
265
+ if (chosen.length === 0) { // progress floor: the single most confident
266
+ let top = cand[0];
267
+ for (const c of cand) if (c.conf > top.conf) top = c;
268
+ chosen = [top];
269
+ }
270
+ }
271
+ step++;
272
+
273
+ const fresh = new Set();
274
+ for (const { p, tok } of chosen) { canvas[p] = tok; fresh.add(p); }
275
+ cfg.onStep?.({ canvas, P, start, end, fresh, block: b, numBlocks, previews, elapsedMs: performance.now() - t0 });
276
+ if (cfg.yieldEvery && previews % cfg.yieldEvery === 0) await yieldToEventLoop();
277
+ }
278
+
279
+ // Response tokens of this block, in position order, cut at the first stop token.
280
+ // Stop only after the whole block resolves: an earlier slot can fill after a later EOS.
281
+ const respStart = Math.max(start, P), respEnd = Math.min(end, limit);
282
+ let cut = respEnd;
283
+ if (cfg.stopAtEos) {
284
+ for (let i = respStart; i < respEnd; i++) if (eos.has(canvas[i])) { cut = i + 1; stopToken = canvas[i]; break; }
285
+ }
286
+ if (cfg.penalties) {
287
+ for (let i = respStart; i < cut; i++) counts.set(canvas[i], (counts.get(canvas[i]) ?? 0) + 1);
288
+ }
289
+ responseLength += cut - respStart;
290
+ cfg.onBlock?.({ tokens: Array.from(canvas.subarray(respStart, stopToken == null ? cut : cut - 1)), block: b });
291
+ if (stopToken != null) { genEnd = cut - 1; break; }
292
+ if (b + 1 < numBlocks && end < limit) await commit(start, end, blocks[b + 1][0]);
293
+ }
294
+
295
+ // Trim at the first stop token after the prompt (fixed-schedule runs fill every block first).
296
+ if (stopToken == null) {
297
+ for (let i = P; i < limit; i++) if (eos.has(canvas[i])) { genEnd = i; stopToken = canvas[i]; break; }
298
+ }
299
+ return {
300
+ tokenIds: Array.from(canvas.subarray(P, Math.min(genEnd, limit))),
301
+ stopToken,
302
+ canvas,
303
+ P,
304
+ previews,
305
+ commits,
306
+ seconds: (performance.now() - t0) / 1000,
307
+ };
308
+ }