Stefatorus Claude Opus 5.5 commited on
Commit
146ba97
·
1 Parent(s): 2dc8d66

Faster WebGPU sampling: fp16 compute, GPU-resident steps, non-blocking thinker preview

Browse files

- 002/003 and 001 load the v2 build from webgpu/v2/ in each model repo: one graph run per Euler step (CFG mix +
update inside the graph), z stays on the GPU between steps, fp16 compute (fp32 for graph I/O, CFG/Euler,
timestep/resolution/count embeddings and Range), 1x1 conv + depth-to-space instead of ConvTranspose.
- 003 @ 512: 275 -> 161 ms/step, 14 -> 8.1 s/image; 256 px and 001/002: ~6 -> ~3.4 s (RTX 4060, Edge).
- Show thinker: async readback, colouring in a Web Worker, frames dropped while busy; no cost to sampling.
- "Fast" toggle: 25 steps (~4.1 s at 512); default stays 50.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018EtDckYfNgMbkwqVmTsdha

Files changed (7) hide show
  1. README.md +18 -12
  2. css/style.css +2 -0
  3. index.html +2 -1
  4. js/agate.js +76 -10
  5. js/app.js +12 -9
  6. js/thinker-worker.js +11 -0
  7. js/thinker.js +65 -32
README.md CHANGED
@@ -37,14 +37,14 @@ this browser.
37
  | [Preview 001](https://huggingface.co/Logolabs/agate-preview-001) | 256 px | the first preview | 530 MB |
38
 
39
  Each model is downloaded once and kept in the browser's Cache Storage. The files live in each release's model repo
40
- under `webgpu/` (e.g. [Logolabs/agate-preview-003/webgpu](https://huggingface.co/Logolabs/agate-preview-003/tree/main/webgpu)),
41
  with a `manifest.json` holding sizes and sha256s (the cache keys). `models/` in this Space is the original 001 build
42
  (without the plan output), kept for `?models=./models/`.
43
 
44
  ## Show thinker
45
 
46
  Agate's thinker lays the picture out on a 16 × 16 grid (the plan) that steers the renderer. With **Show thinker**
47
- (on by default; it costs about 2-4% per step) the page shows, at every step, the plan of the conditional branch as
48
  colours and the image the model currently expects (x₁ = z + (1 − t)·v, with the SD latent→RGB approximation) —
49
  the same live preview as the [ComfyUI nodes](https://github.com/logolabs/agate-comfyui): the plan's cells are
50
  projected on its top three principal components, fitted at the first step with the 2% / 98% quantiles fixed then,
@@ -55,18 +55,24 @@ so a colour keeps its meaning while the plan evolves.
55
  A browser with WebGPU: a recent Chrome or Edge (113+). Safari and Firefox may lack WebGPU or need it enabled in
56
  their settings; without it the page offers a WebAssembly (CPU) path that works but takes minutes per image.
57
 
58
- ## Speed and accuracy (RTX 4060 laptop, Edge, WebGPU, warm)
59
 
60
- | model | per step (CFG batch of 2) | 50-step image | model setup per visit |
61
  |---|---|---|---|
62
- | 003 at 512 px | ~275 ms | ~14 s | ~15 s |
63
- | 003 at 256 px | ~120 ms | ~6 s | ~15 s |
64
- | 002 / 001 | ~105 ms | ~5.3 s | ~14 s |
65
-
66
- With identical initial noise, the browser output matches the PyTorch packages (TAESD decoder): 001 within
67
- 5/255 per pixel, 002 within 14/255 (PSNR 52.8 dB), 003 within 29/255 at 512 px (PSNR 54.8 dB) and 47/255 at
68
- 256 px (PSNR 48.9 dB), on a few pixels only (mean difference 0.1-0.2/255). The difference comes from storing the
69
- weights as fp16 to halve the download; an fp32 build matches to 1/255. Run the self-test with `?parity=1`.
 
 
 
 
 
 
70
  Seeds use a JavaScript PRNG, so a seed gives the same image in every browser, but not the same image as the
71
  PyTorch pipeline with that seed. The Python packages decode with the full SD-VAE by default; this page uses TAESD.
72
 
 
37
  | [Preview 001](https://huggingface.co/Logolabs/agate-preview-001) | 256 px | the first preview | 530 MB |
38
 
39
  Each model is downloaded once and kept in the browser's Cache Storage. The files live in each release's model repo
40
+ under `webgpu/v2/` (e.g. [Logolabs/agate-preview-003/webgpu/v2](https://huggingface.co/Logolabs/agate-preview-003/tree/main/webgpu/v2)),
41
  with a `manifest.json` holding sizes and sha256s (the cache keys). `models/` in this Space is the original 001 build
42
  (without the plan output), kept for `?models=./models/`.
43
 
44
  ## Show thinker
45
 
46
  Agate's thinker lays the picture out on a 16 × 16 grid (the plan) that steers the renderer. With **Show thinker**
47
+ (on by default; it does not slow sampling: previews are read back asynchronously and coloured in a Web Worker) the page shows, at every step, the plan of the conditional branch as
48
  colours and the image the model currently expects (x₁ = z + (1 − t)·v, with the SD latent→RGB approximation) —
49
  the same live preview as the [ComfyUI nodes](https://github.com/logolabs/agate-comfyui): the plan's cells are
50
  projected on its top three principal components, fitted at the first step with the 2% / 98% quantiles fixed then,
 
55
  A browser with WebGPU: a recent Chrome or Edge (113+). Safari and Firefox may lack WebGPU or need it enabled in
56
  their settings; without it the page offers a WebAssembly (CPU) path that works but takes minutes per image.
57
 
58
+ ## Speed and accuracy (RTX 4060 laptop, Edge, WebGPU, warm, 50 steps)
59
 
60
+ | model | per step (CFG batch of 2) | image | before (2026-09-29 morning) |
61
  |---|---|---|---|
62
+ | 003 at 512 px | ~161 ms | **~8.1 s** (Fast, 25 steps: ~4.1 s) | 275 ms, 14 s |
63
+ | 003 at 256 px | ~70 ms | ~3.5 s | 120 ms, 6.1 s |
64
+ | 002 / 001 | ~65 ms | ~3.3 s | 105 ms, 5.3 s |
65
+
66
+ How: each step is one graph run that also does the CFG mix and the Euler update, so the latent never leaves the
67
+ GPU between steps (the only readback is the final latent); the network computes in **fp16** (WebGPU `shader-f16`;
68
+ WGSL has no bf16), with fp32 kept for the graph I/O, the CFG/Euler arithmetic, the timestep / resolution / count
69
+ embeddings (sin(1000·t·f) needs fp32) and `Range`; the transposed-conv upsamplers run as 1x1 conv + depth-to-space.
70
+ The thinker preview never blocks sampling: its tensors are downloaded asynchronously, coloured in a Web Worker,
71
+ and frames are dropped while one is pending (measured cost: none).
72
+
73
+ fp16 compute moves the result slightly: against the fp32 PyTorch reference with the same initial noise, 001 is
74
+ within 16/255 (PSNR 48 dB), 002 36 dB, 003 at 256 px 42 dB and 003 at 512 px 31 dB (mean 0.6-1.8/255). For scale,
75
+ the Python package's own CUDA path (bf16) is 14 dB from that fp32 reference for the same noise. Run `?parity=1`.
76
  Seeds use a JavaScript PRNG, so a seed gives the same image in every browser, but not the same image as the
77
  PyTorch pipeline with that seed. The Python packages decode with the full SD-VAE by default; this page uses TAESD.
78
 
css/style.css CHANGED
@@ -130,3 +130,5 @@ input[type=text] { width: 100%; font: 15px/1.45 var(--sans); color: var(--ink);
130
  .thinker #pred-canvas { image-rendering: auto; }
131
  .thinker figcaption { margin-top: 6px; }
132
  .thinker-note { grid-column: 1 / -1; margin: 0; font-size: 13px; color: var(--mute); }
 
 
 
130
  .thinker #pred-canvas { image-rendering: auto; }
131
  .thinker figcaption { margin-top: 6px; }
132
  .thinker-note { grid-column: 1 / -1; margin: 0; font-size: 13px; color: var(--mute); }
133
+ .label .toggle.fast { margin-left: 10px; text-transform: uppercase; letter-spacing: .1em; color: var(--ink); }
134
+ .label .toggle.fast input { accent-color: var(--red); margin: 0 4px 0 0; vertical-align: -1px; }
index.html CHANGED
@@ -68,7 +68,8 @@
68
  </div>
69
  </div>
70
  <div class="field">
71
- <label class="label" for="steps">Steps <b id="steps-v">50</b></label>
 
72
  <input id="steps" type="range" min="4" max="50" step="1" value="50">
73
  </div>
74
  <div class="field">
 
68
  </div>
69
  </div>
70
  <div class="field">
71
+ <label class="label" for="steps">Steps <b id="steps-v">50</b>
72
+ <span class="toggle fast" title="25 steps: about twice as fast; detail holds, the layout can change"><input type="checkbox" id="fast"> Fast</span></label>
73
  <input id="steps" type="range" min="4" max="50" step="1" value="50">
74
  </div>
75
  <div class="field">
js/agate.js CHANGED
@@ -15,7 +15,7 @@ export { ort };
15
  // (001 without the plan output) is used only through ?models=. dir: the layout under a local
16
  // models base (?models=<url> or window.AGATE_MODEL_BASE given explicitly, e.g. for local testing). cache: Cache
17
  // Storage name (001 keeps the name it always had, so returning visitors do not download it again).
18
- const HUB = (v) => `https://huggingface.co/Logolabs/agate-preview-${v}/resolve/main/webgpu/`;
19
  export const VERSIONS = {
20
  "003": { remote: HUB("003"), dir: "003/", cache: "agate-preview-003-web", res: [512, 256], note: "Multi-resolution model, 512 px (or 256). Quoted text is spelled out, counts are coded, “no X” becomes a negative prompt." },
21
  "002": { remote: HUB("002"), dir: "002/", cache: "agate-preview-002-web", res: [256], note: "Same network as 001, trained 38,710 steps longer before the same anneal. 256 px, plain prompts." },
@@ -99,6 +99,9 @@ export class Agate {
99
 
100
  get mr() { return this.manifest?.arch === "fcdm_t2mr"; }
101
 
 
 
 
102
  fileKey(f) { return new Request(`${this.base}${f}?sha256=${this.manifest.files[f].sha256}`); }
103
 
104
  async getFile(f, onBytes = () => {}) {
@@ -115,7 +118,7 @@ export class Agate {
115
  this.resolution = this.mr ? Number(resolution || manifest.resolution) : 256;
116
  // 003: download everything (the two graphs are ~1 MB each and share one weights file), so a resolution
117
  // switch needs no network
118
- const need = this.mr ? Object.keys(manifest.files) : FILES_T2;
119
  const total = need.reduce((s, f) => s + manifest.files[real(f)].bytes, 0);
120
  let loaded = 0, fromCache = 0;
121
  const bufs = {};
@@ -151,7 +154,10 @@ export class Agate {
151
  }
152
  onProgress({ phase: "init", file: "generator", loaded: total, total });
153
  const ts = performance.now();
154
- if (this.mr) {
 
 
 
155
  this.weights = bufs["generator.weights"]; // kept for a resolution switch (no re-read)
156
  await this.createGenerator(this.resolution, bufs);
157
  } else {
@@ -166,12 +172,20 @@ export class Agate {
166
 
167
  // 003: the generator graph for one resolution; the weights file is shared by both graphs.
168
  async createGenerator(res, bufs = {}) {
169
- const r = this.manifest.resolutions[String(res)];
170
- if (!r) throw new Error(`resolution ${res} not available`);
171
- const graph = bufs[r.graph] || await this.getFile(r.graph);
172
- const weights = this.weights || await this.getFile(r.external_data);
 
 
 
 
 
 
 
 
173
  if (this.sessions.gen) { try { await this.sessions.gen.release(); } catch { /* */ } this.sessions.gen = null; }
174
- this.sessions.gen = await ort.InferenceSession.create(graph, this.sessionOptions({ externalData: [{ path: r.external_data, data: weights }] }));
175
  this.resolution = Number(res);
176
  }
177
 
@@ -240,9 +254,57 @@ export class Agate {
240
  // -> { rgba (watermarked unless watermark=false), width, height, latent, timings, prepared }
241
  get hasPlan() { return !!this.sessions?.gen?.outputNames?.includes("plan"); }
242
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
243
  // onPreview({ step, steps, plan, x1, hw }) after every step, when given and the graph outputs the plan: plan =
244
  // the thinker output for the conditional branch (Float32Array 640 x 16 x 16), x1 = z + (1 - t) v (guided).
245
- async generate({ prompt, negative = "", seed = 0, steps = 50, cfg = 3.0, noise = null, watermark = true, onStep = () => {}, onPreview = null, shouldStop = () => false }) {
246
  const C = this.manifest.latent_ch, D = this.manifest.ctx_dim;
247
  const rc = this.mr ? this.manifest.resolutions[String(this.resolution)] : { latent_hw: this.manifest.latent_hw, shift: 1 };
248
  const hw = rc.latent_hw, shift = Number(rc.shift || 1);
@@ -268,10 +330,13 @@ export class Agate {
268
  }
269
 
270
  let z = noise ? Float32Array.from(noise) : Agate.gaussianNoise(seed, per);
271
- const zz = new Float32Array(2 * per), tt = new Float32Array(2);
272
  const grid = Array.from({ length: steps + 1 }, (_, i) => Agate.shiftT(i / steps, shift));
273
  t0 = performance.now();
274
  const stepMs = [];
 
 
 
 
275
  for (let i = 0; i < steps; i++) {
276
  if (shouldStop()) throw new Error("stopped");
277
  const ts = performance.now();
@@ -296,6 +361,7 @@ export class Agate {
296
  onStep(i + 1, steps, z);
297
  await new Promise((r) => setTimeout(r, 0)); // let the progress bar paint
298
  }
 
299
  T.samplerMs = performance.now() - t0;
300
  T.firstStepMs = stepMs[0];
301
  T.stepMs = stepMs.length > 1 ? (T.samplerMs - stepMs[0]) / (stepMs.length - 1) : stepMs[0];
 
15
  // (001 without the plan output) is used only through ?models=. dir: the layout under a local
16
  // models base (?models=<url> or window.AGATE_MODEL_BASE given explicitly, e.g. for local testing). cache: Cache
17
  // Storage name (001 keeps the name it always had, so returning visitors do not download it again).
18
+ const HUB = (v) => `https://huggingface.co/Logolabs/agate-preview-${v}/resolve/main/webgpu/v2/`; // v2: fast fp16 step graphs
19
  export const VERSIONS = {
20
  "003": { remote: HUB("003"), dir: "003/", cache: "agate-preview-003-web", res: [512, 256], note: "Multi-resolution model, 512 px (or 256). Quoted text is spelled out, counts are coded, “no X” becomes a negative prompt." },
21
  "002": { remote: HUB("002"), dir: "002/", cache: "agate-preview-002-web", res: [256], note: "Same network as 001, trained 38,710 steps longer before the same anneal. 256 px, plain prompts." },
 
99
 
100
  get mr() { return this.manifest?.arch === "fcdm_t2mr"; }
101
 
102
+ // the fast build (2026-09-29): one Euler step per run, fp16 compute, everything stays on the GPU between steps
103
+ get fast() { return !!this.manifest?.step; }
104
+
105
  fileKey(f) { return new Request(`${this.base}${f}?sha256=${this.manifest.files[f].sha256}`); }
106
 
107
  async getFile(f, onBytes = () => {}) {
 
118
  this.resolution = this.mr ? Number(resolution || manifest.resolution) : 256;
119
  // 003: download everything (the two graphs are ~1 MB each and share one weights file), so a resolution
120
  // switch needs no network
121
+ const need = this.mr || manifest.step ? Object.keys(manifest.files) : FILES_T2;
122
  const total = need.reduce((s, f) => s + manifest.files[real(f)].bytes, 0);
123
  let loaded = 0, fromCache = 0;
124
  const bufs = {};
 
154
  }
155
  onProgress({ phase: "init", file: "generator", loaded: total, total });
156
  const ts = performance.now();
157
+ if (this.fast) {
158
+ this.weights = bufs[manifest.step.external_data]; // kept for a resolution switch (no re-read)
159
+ await this.createGenerator(this.resolution, bufs);
160
+ } else if (this.mr) {
161
  this.weights = bufs["generator.weights"]; // kept for a resolution switch (no re-read)
162
  await this.createGenerator(this.resolution, bufs);
163
  } else {
 
172
 
173
  // 003: the generator graph for one resolution; the weights file is shared by both graphs.
174
  async createGenerator(res, bufs = {}) {
175
+ let graphName, ext, extra = {};
176
+ if (this.fast) {
177
+ graphName = this.manifest.step.graphs[String(res)];
178
+ ext = this.manifest.step.external_data;
179
+ extra = { preferredOutputLocation: "gpu-buffer" }; // z_next / x1 / plan stay on the GPU
180
+ } else {
181
+ const r = this.manifest.resolutions[String(res)];
182
+ graphName = r?.graph; ext = r?.external_data;
183
+ }
184
+ if (!graphName) throw new Error(`resolution ${res} not available`);
185
+ const graph = bufs[graphName] || await this.getFile(graphName);
186
+ const weights = this.weights || await this.getFile(ext);
187
  if (this.sessions.gen) { try { await this.sessions.gen.release(); } catch { /* */ } this.sessions.gen = null; }
188
+ this.sessions.gen = await ort.InferenceSession.create(graph, this.sessionOptions({ externalData: [{ path: ext, data: weights }], ...extra }));
189
  this.resolution = Number(res);
190
  }
191
 
 
254
  // -> { rgba (watermarked unless watermark=false), width, height, latent, timings, prepared }
255
  get hasPlan() { return !!this.sessions?.gen?.outputNames?.includes("plan"); }
256
 
257
+ // GPU-resident sampler for the step graphs: z_next of one run is the z input of the next, ctx / mask / counts
258
+ // are uploaded once, and nothing is read back until the end. The preview (plan + x1) is requested only when
259
+ // onPreview is set, previewReady() says the viewer is free and no earlier readback is still in flight; its
260
+ // download is started but never awaited by the loop (frames are dropped instead).
261
+ async sampleFast({ z, grid, cfg, hw, C, L, D, ctx, mask, cnt, onStep, onPreview, previewReady, shouldStop, stepMs }) {
262
+ const dev = ort.env.webgpu?.device;
263
+ const bufs = [];
264
+ const gpuT = (data, dims) => {
265
+ if (!dev) return new ort.Tensor("float32", data, dims);
266
+ const buf = dev.createBuffer({ size: Math.ceil(data.byteLength / 16) * 16, usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_SRC | GPUBufferUsage.COPY_DST });
267
+ dev.queue.writeBuffer(buf, 0, data);
268
+ bufs.push(buf);
269
+ return ort.Tensor.fromGpuBuffer(buf, { dataType: "float32", dims });
270
+ };
271
+ const fixed = { ctx: gpuT(ctx, [2, L, D]), mask: gpuT(mask, [2, L]) };
272
+ if (this.sessions.gen.inputNames.includes("counts")) fixed.counts = gpuT(cnt || new Float32Array(2 * L), [2, L]);
273
+ const steps = grid.length - 1;
274
+ let zT = new ort.Tensor("float32", z, [1, C, hw, hw]), inflight = false;
275
+ const one = (v) => new ort.Tensor("float32", new Float32Array([v]), [1]);
276
+ const cfgT = one(cfg);
277
+ try {
278
+ for (let i = 0; i < steps; i++) {
279
+ if (shouldStop()) throw new Error("stopped");
280
+ const ts = performance.now();
281
+ const want = !!onPreview && this.hasPlan && !inflight && previewReady();
282
+ const out = await this.sessions.gen.run({ z: zT, t: one(grid[i]), dt: one(grid[i + 1] - grid[i]), cfg: cfgT, ...fixed },
283
+ want ? ["z_next", "x1", "plan"] : ["z_next"]);
284
+ if (i > 0) zT.dispose?.();
285
+ zT = out.z_next;
286
+ if (want) {
287
+ inflight = true;
288
+ const step = i + 1;
289
+ Promise.all([out.plan.getData(true), out.x1.getData(true)])
290
+ .then(([plan, x1]) => onPreview({ step, steps, plan, x1, hw }))
291
+ .catch((e) => console.warn("preview", e))
292
+ .finally(() => { inflight = false; });
293
+ }
294
+ stepMs.push(performance.now() - ts);
295
+ onStep(i + 1, steps, null);
296
+ if ((i & 3) === 3) await new Promise((r) => setTimeout(r, 0)); // let the page paint now and then
297
+ }
298
+ const zf = await zT.getData(true); // the only synchronising readback
299
+ return Float32Array.from(zf);
300
+ } finally {
301
+ for (const b of bufs) { try { b.destroy(); } catch { /* */ } }
302
+ }
303
+ }
304
+
305
  // onPreview({ step, steps, plan, x1, hw }) after every step, when given and the graph outputs the plan: plan =
306
  // the thinker output for the conditional branch (Float32Array 640 x 16 x 16), x1 = z + (1 - t) v (guided).
307
+ async generate({ prompt, negative = "", seed = 0, steps = 50, cfg = 3.0, noise = null, watermark = true, onStep = () => {}, onPreview = null, previewReady = () => true, shouldStop = () => false }) {
308
  const C = this.manifest.latent_ch, D = this.manifest.ctx_dim;
309
  const rc = this.mr ? this.manifest.resolutions[String(this.resolution)] : { latent_hw: this.manifest.latent_hw, shift: 1 };
310
  const hw = rc.latent_hw, shift = Number(rc.shift || 1);
 
330
  }
331
 
332
  let z = noise ? Float32Array.from(noise) : Agate.gaussianNoise(seed, per);
 
333
  const grid = Array.from({ length: steps + 1 }, (_, i) => Agate.shiftT(i / steps, shift));
334
  t0 = performance.now();
335
  const stepMs = [];
336
+ if (this.fast) {
337
+ z = await this.sampleFast({ z, grid, cfg, hw, C, L, D, ctx, mask, cnt: feeds.counts?.data, onStep, onPreview, previewReady, shouldStop, stepMs });
338
+ } else {
339
+ const zz = new Float32Array(2 * per), tt = new Float32Array(2);
340
  for (let i = 0; i < steps; i++) {
341
  if (shouldStop()) throw new Error("stopped");
342
  const ts = performance.now();
 
361
  onStep(i + 1, steps, z);
362
  await new Promise((r) => setTimeout(r, 0)); // let the progress bar paint
363
  }
364
+ }
365
  T.samplerMs = performance.now() - t0;
366
  T.firstStepMs = stepMs[0];
367
  T.stepMs = stepMs.length > 1 ? (T.samplerMs - stepMs[0]) / (stepMs.length - 1) : stepMs[0];
js/app.js CHANGED
@@ -116,7 +116,8 @@ async function pickRes(r) {
116
  }
117
  }
118
 
119
- ui.steps.oninput = () => { $("steps-v").textContent = ui.steps.value; };
 
120
  ui.cfg.oninput = () => { $("cfg-v").textContent = Number(ui.cfg.value).toFixed(1); };
121
  ui.dice.onclick = () => { ui.seed.value = Math.floor(Math.random() * 2 ** 31); };
122
  ui.crisp.checked = store.get("agate-crisp") === "1";
@@ -169,18 +170,19 @@ async function pngWithMetadata() {
169
  return new Blob([bytes], { type: "image/png" });
170
  }
171
 
 
 
172
  function previewFn() {
173
  thinkerView.reset();
174
- let planMs = 0, n = 0;
175
- window.__agate.thinker = { steps: 0, planMs: 0 };
176
- return ({ step, steps, plan, x1, hw }) => {
177
- const t0 = performance.now();
178
  ui.thinker.hidden = false;
179
- thinkerView.drawPlan(plan);
180
- thinkerView.drawPrediction(x1, hw);
181
  ui.thinkerStep.textContent = `${step} / ${steps}`;
182
- planMs += performance.now() - t0; n++;
183
- window.__agate.thinker = { steps: n, drawMsMean: planMs / n };
 
 
184
  };
185
  }
186
 
@@ -197,6 +199,7 @@ async function generate(opts = {}) {
197
  watermark: opts.watermark ?? true,
198
  onStep: (i, n) => setProgress(i / n, `Step ${i} / ${n}`, ""),
199
  onPreview: (opts.thinker ?? ui.showThinker.checked) ? previewFn() : null,
 
200
  shouldStop: () => stop,
201
  });
202
  const ctx = ui.canvas.getContext("2d");
 
116
  }
117
  }
118
 
119
+ ui.steps.oninput = () => { $("steps-v").textContent = ui.steps.value; $("fast").checked = Number(ui.steps.value) === 25; };
120
+ $("fast").onchange = () => { ui.steps.value = $("fast").checked ? 25 : 50; $("steps-v").textContent = ui.steps.value; };
121
  ui.cfg.oninput = () => { $("cfg-v").textContent = Number(ui.cfg.value).toFixed(1); };
122
  ui.dice.onclick = () => { ui.seed.value = Math.floor(Math.random() * 2 ** 31); };
123
  ui.crisp.checked = store.get("agate-crisp") === "1";
 
170
  return new Blob([bytes], { type: "image/png" });
171
  }
172
 
173
+ // The previews: frames go to thinkerView, which colours them in a Web Worker and drops frames while busy. The
174
+ // sampler never waits for any of it (agate.js sampleFast); with Show thinker off nothing is requested at all.
175
  function previewFn() {
176
  thinkerView.reset();
177
+ let n = 0, t0 = performance.now();
178
+ window.__agate.thinker = { submitted: 0 };
179
+ thinkerView.onDrawn = ({ step, steps, workerMs, dropped, drawn }) => {
 
180
  ui.thinker.hidden = false;
 
 
181
  ui.thinkerStep.textContent = `${step} / ${steps}`;
182
+ window.__agate.thinker = { submitted: n, drawn, dropped, lastWorkerMs: workerMs, lastStep: step };
183
+ };
184
+ return ({ step, steps, plan, x1, hw }) => {
185
+ if (thinkerView.submit(plan, x1, hw, { step, steps })) n++;
186
  };
187
  }
188
 
 
199
  watermark: opts.watermark ?? true,
200
  onStep: (i, n) => setProgress(i / n, `Step ${i} / ${n}`, ""),
201
  onPreview: (opts.thinker ?? ui.showThinker.checked) ? previewFn() : null,
202
+ previewReady: () => !thinkerView.pending,
203
  shouldStop: () => stop,
204
  });
205
  const ctx = ui.canvas.getContext("2d");
js/thinker-worker.js ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // Web Worker: colours the thinker previews off the main thread (see thinker.js).
2
+ import { planPixels, predPixels } from "./thinker.js";
3
+
4
+ let gen = -1, state = {};
5
+ self.onmessage = (e) => {
6
+ const { gen: g, plan, x1, hw, meta } = e.data;
7
+ if (g !== gen) { gen = g; state = {}; } // a new generation: fit a new basis at its first frame
8
+ const t0 = performance.now();
9
+ const p = planPixels(plan, state), q = predPixels(x1, hw);
10
+ self.postMessage({ gen: g, plan: p, pred: q, hw, meta, ms: performance.now() - t0 }, [p.buffer, q.buffer]);
11
+ };
js/thinker.js CHANGED
@@ -75,42 +75,75 @@ function quantile(sorted, q) { // torch.quantile, linear interpo
75
  return sorted[lo] + (sorted[hi] - sorted[lo]) * (pos - lo);
76
  }
77
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
78
  export class ThinkerView {
79
- constructor(planCanvas, predCanvas) { this.planCanvas = planCanvas; this.predCanvas = predCanvas; this.reset(); }
 
 
 
 
 
 
 
 
80
 
81
- reset() { this.basis = null; this.lo = null; this.hi = null; }
82
 
83
- // plan: Float32Array (C * g * g), channel-major (1, C, g, g)
84
- drawPlan(plan, C = 640, g = 16) {
85
- const n = g * g, X = new Float64Array(n * C);
86
- for (let c = 0; c < C; c++) for (let p = 0; p < n; p++) X[p * C + c] = plan[c * n + p];
87
- for (let c = 0; c < C; c++) { let m = 0; for (let p = 0; p < n; p++) m += X[p * C + c]; m /= n; for (let p = 0; p < n; p++) X[p * C + c] -= m; }
88
- if (!this.basis) this.basis = topComponents(X, n, C);
89
- const Y = this.basis.map((v) => { const y = new Float64Array(n); for (let p = 0; p < n; p++) { let s = 0; const r = p * C; for (let c = 0; c < C; c++) s += X[r + c] * v[c]; y[p] = s; } return y; });
90
- if (!this.lo) {
91
- this.lo = Y.map((y) => quantile(Float64Array.from(y).sort(), 0.02));
92
- this.hi = Y.map((y) => quantile(Float64Array.from(y).sort(), 0.98));
93
- }
94
- const img = new ImageData(g, g);
95
- for (let p = 0; p < n; p++) {
96
- for (let k = 0; k < 3; k++) img.data[p * 4 + k] = Math.round(Math.min(1, Math.max(0, (Y[k][p] - this.lo[k]) / (this.hi[k] - this.lo[k] + 1e-8))) * 255);
97
- img.data[p * 4 + 3] = 255;
98
- }
99
- const cv = this.planCanvas; cv.width = g; cv.height = g;
100
- cv.getContext("2d").putImageData(img, 0, 0);
101
  }
102
 
103
- // x1: Float32Array (4 * hw * hw) in the model's (scaled SD-VAE) latent space
104
- drawPrediction(x1, hw) {
105
- const n = hw * hw, img = new ImageData(hw, hw);
106
- for (let p = 0; p < n; p++) {
107
- for (let k = 0; k < 3; k++) {
108
- let s = 0; for (let c = 0; c < 4; c++) s += x1[c * n + p] * RGB_FACTORS[c][k];
109
- img.data[p * 4 + k] = Math.floor(Math.min(1, Math.max(0, (s + 1) / 2)) * 255);
110
- }
111
- img.data[p * 4 + 3] = 255;
112
- }
113
- const cv = this.predCanvas; cv.width = hw; cv.height = hw;
114
- cv.getContext("2d").putImageData(img, 0, 0);
115
  }
 
 
 
116
  }
 
75
  return sorted[lo] + (sorted[hi] - sorted[lo]) * (pos - lo);
76
  }
77
 
78
+ // state: { basis, lo, hi } (filled at the first call of a generation). -> RGBA Uint8ClampedArray (g * g * 4)
79
+ export function planPixels(plan, state, C = 640, g = 16) {
80
+ const n = g * g, X = new Float64Array(n * C);
81
+ for (let c = 0; c < C; c++) for (let p = 0; p < n; p++) X[p * C + c] = plan[c * n + p];
82
+ for (let c = 0; c < C; c++) { let m = 0; for (let p = 0; p < n; p++) m += X[p * C + c]; m /= n; for (let p = 0; p < n; p++) X[p * C + c] -= m; }
83
+ if (!state.basis) state.basis = topComponents(X, n, C);
84
+ const Y = state.basis.map((v) => { const y = new Float64Array(n); for (let p = 0; p < n; p++) { let s = 0; const r = p * C; for (let c = 0; c < C; c++) s += X[r + c] * v[c]; y[p] = s; } return y; });
85
+ if (!state.lo) {
86
+ state.lo = Y.map((y) => quantile(Float64Array.from(y).sort(), 0.02));
87
+ state.hi = Y.map((y) => quantile(Float64Array.from(y).sort(), 0.98));
88
+ }
89
+ const px = new Uint8ClampedArray(n * 4);
90
+ for (let p = 0; p < n; p++) {
91
+ for (let k = 0; k < 3; k++) px[p * 4 + k] = Math.round(Math.min(1, Math.max(0, (Y[k][p] - state.lo[k]) / (state.hi[k] - state.lo[k] + 1e-8))) * 255);
92
+ px[p * 4 + 3] = 255;
93
+ }
94
+ return px;
95
+ }
96
+
97
+ // x1: Float32Array (4 * hw * hw) in the model's (scaled SD-VAE) latent space -> RGBA (hw * hw * 4)
98
+ export function predPixels(x1, hw) {
99
+ const n = hw * hw, px = new Uint8ClampedArray(n * 4);
100
+ for (let p = 0; p < n; p++) {
101
+ for (let k = 0; k < 3; k++) {
102
+ let s = 0; for (let c = 0; c < 4; c++) s += x1[c * n + p] * RGB_FACTORS[c][k];
103
+ px[p * 4 + k] = Math.floor(Math.min(1, Math.max(0, (s + 1) / 2)) * 255);
104
+ }
105
+ px[p * 4 + 3] = 255;
106
+ }
107
+ return px;
108
+ }
109
+
110
+ // Draws previews; the colouring runs in a Web Worker (thinker-worker.js) when available, so the page and the
111
+ // sampler never wait for it. A frame arriving while the previous one is still being coloured is dropped.
112
  export class ThinkerView {
113
+ constructor(planCanvas, predCanvas, onDrawn = () => {}) {
114
+ this.planCanvas = planCanvas; this.predCanvas = predCanvas; this.onDrawn = onDrawn;
115
+ this.gen = 0; this.busy = false; this.dropped = 0; this.drawn = 0;
116
+ try {
117
+ this.worker = new Worker(new URL("./thinker-worker.js", import.meta.url), { type: "module" });
118
+ this.worker.onmessage = (e) => this._paint(e.data);
119
+ } catch { this.worker = null; }
120
+ this.reset();
121
+ }
122
 
123
+ reset() { this.gen++; this.state = {}; this.busy = false; this.dropped = 0; this.drawn = 0; }
124
 
125
+ get pending() { return this.busy; }
126
+
127
+ // plan: Float32Array (640 * 16 * 16), x1: Float32Array (4 * hw * hw); both are transferred to the worker
128
+ submit(plan, x1, hw, meta = {}) {
129
+ if (this.busy) { this.dropped++; return false; }
130
+ this.busy = true;
131
+ const job = { gen: this.gen, plan, x1, hw, meta };
132
+ if (this.worker) this.worker.postMessage(job, [plan.buffer, x1.buffer]);
133
+ else setTimeout(() => this._paint({ gen: job.gen, hw, meta, plan: planPixels(plan, this.state), pred: predPixels(x1, hw) }), 0);
134
+ return true;
 
 
 
 
 
 
 
 
135
  }
136
 
137
+ _paint({ gen, plan, pred, hw, meta, ms }) {
138
+ this.busy = false;
139
+ if (gen !== this.gen) return;
140
+ const put = (cv, px, s) => { cv.width = s; cv.height = s; cv.getContext("2d").putImageData(new ImageData(px, s, s), 0, 0); };
141
+ put(this.planCanvas, plan, 16);
142
+ put(this.predCanvas, pred, hw);
143
+ this.drawn++;
144
+ this.onDrawn({ ...meta, workerMs: ms, dropped: this.dropped, drawn: this.drawn });
 
 
 
 
145
  }
146
+
147
+ // synchronous path (tests): colour on this thread
148
+ drawPlan(plan) { const px = planPixels(plan, this.state); const cv = this.planCanvas; cv.width = 16; cv.height = 16; cv.getContext("2d").putImageData(new ImageData(px, 16, 16), 0, 0); }
149
  }