feat: consolidate model variants in crivo-web

bd8c5ee 25 days ago

2.19 kB

# Playground benchmark

Run `bun run browser`, open `http://localhost:4180` in a fresh browser tab, and paste
`playground.js` into DevTools before the first inference. Leave the default example
unchanged. The script operates the actual selectors and Run button and observes the
production worker messages; it does not substitute an inference implementation.

Inspect `window.crivoBenchmark.status` and `.error`. Once status is `complete`, save
`JSON.stringify(window.crivoBenchmark, null, 2)`. Each of the 17 supported configurations
runs once to initialize, three additional times to warm up, then 20 measured times.
All three questions are sent together in one request. Warm samples must report zero
model loading time. A 50 ms pause between requests lets the UI finish handling results.
Keep other workloads idle; hardware, power mode, browser and background activity affect timing.

`browser.json` records the measured input and all timing samples from the local playground.
Inference timing includes tokenization, the batched ONNX forward pass and answer mapping.
Worker total includes validation and other worker overhead, but excludes rendering and
main-thread messaging. First-run model setup and first inference are retained separately.
The README reports median and nearest-rank p95 inference times (19th of 20 sorted samples).

For the recorded run, exact locally converted weights were preloaded into the browser's
`transformers-cache` CacheStorage under their production Hugging Face resolve URLs.
These were the same artifacts published at the revision recorded in `browser.json`.
Model setup therefore measures cached asset reads and session initialization, **not downloads**.
To reproduce cached setup without local artifacts, run each selection once to download
its assets, reload the page, and then start the script. Do not clear browser storage.
Network speed is deliberately excluded. ONNX Runtime/CDN files may also be browser-cached.

The 1B FP32/WASM configuration is excluded because it exceeds the tested WASM memory
budget. INT8/WebGPU is unsupported. INT8 here is weight-only storage with FP32 arithmetic;
it is not expected to consistently outperform FP32.

	# Playground benchmark

	Run `bun run browser`, open `http://localhost:4180` in a fresh browser tab, and paste
	`playground.js` into DevTools before the first inference. Leave the default example
	unchanged. The script operates the actual selectors and Run button and observes the
	production worker messages; it does not substitute an inference implementation.

	Inspect `window.crivoBenchmark.status` and `.error`. Once status is `complete`, save
	`JSON.stringify(window.crivoBenchmark, null, 2)`. Each of the 17 supported configurations
	runs once to initialize, three additional times to warm up, then 20 measured times.
	All three questions are sent together in one request. Warm samples must report zero
	model loading time. A 50 ms pause between requests lets the UI finish handling results.
	Keep other workloads idle; hardware, power mode, browser and background activity affect timing.

	`browser.json` records the measured input and all timing samples from the local playground.
	Inference timing includes tokenization, the batched ONNX forward pass and answer mapping.
	Worker total includes validation and other worker overhead, but excludes rendering and
	main-thread messaging. First-run model setup and first inference are retained separately.
	The README reports median and nearest-rank p95 inference times (19th of 20 sorted samples).

	For the recorded run, exact locally converted weights were preloaded into the browser's
	`transformers-cache` CacheStorage under their production Hugging Face resolve URLs.
	These were the same artifacts published at the revision recorded in `browser.json`.
	Model setup therefore measures cached asset reads and session initialization, not downloads.
	To reproduce cached setup without local artifacts, run each selection once to download
	its assets, reload the page, and then start the script. Do not clear browser storage.
	Network speed is deliberately excluded. ONNX Runtime/CDN files may also be browser-cached.

	The 1B FP32/WASM configuration is excluded because it exceeds the tested WASM memory
	budget. INT8/WebGPU is unsupported. INT8 here is weight-only storage with FP32 arithmetic;
	it is not expected to consistently outperform FP32.