Instructions to use Steven10429/laya-typed-decisions-webgpu-q4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Laya
How to use Steven10429/laya-typed-decisions-webgpu-q4 with Laya:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
laya typed-decisions · q4 split ONNX for the browser (WebGPU)
A 4-bit weight-only quantization of the typed-decisions checkpoint of
convaiinnovations/laya (ModernBERT-large, English, fine-tuned for
typed choice / score / noul decisions), in the split layout that
laya-ts loads with ONNX Runtime Web.
467 MB vs 1.69 GB fp32.
Recipe (reproducible, see laya-skill):
laya-ts/scripts/export_onnx.py --repo convaiinnovations/laya --subfolder typed-decisions(laya @6d942c9, torch vs ONNX within 1e-4)MatMulNBitsQuantizer(ONNX Runtime 1.30, 4 bit, block 32, symmetric) on every weight MatMul of the encoder (112) and the head (8). The token embedding stays fp32.
Measured (Apple M4 Pro, Chrome, laya-ts + onnxruntime-web 1.30)
| variant | size | WebGPU p50 | WASM p50 | agreement with fp32 (120 decisions) | max prob drift |
|---|---|---|---|---|---|
| fp32 | 1.69 GB | 265 ms | 2443 ms | — | — |
| q4 (this repo) | 467 MB | 361 ms | ~2.8 s | 110 / 120 | 0.22 |
| q8 (not published) | 705 MB | ≈ WASM: 8-bit MatMulNBits has no WebGPU kernel | 2.8 s | 119 / 120 | 0.013 |
The 120 decisions are 40 labelled support tickets × 3 questions. Accuracy on them (dept / urgency / churn): fp32 0.875 / 0.425 / 0.700, q4 0.875 / 0.450 / 0.625. q4 flips about 8 % of decisions, so re-calibrate thresholds on your own held-out data. English only; use the multilingual export for other languages.
Use
import { Agent } from "laya-ts";
const agent = await Agent.load("https://huggingface.co/Steven10429/laya-typed-decisions-webgpu-q4/resolve/main/");
Live demo: https://stevenli-phoenix-work.itch.io/laya-webgpu
License & attribution
Apache-2.0, same as the original weights by ConvAI Innovations / Nandakishor M (NandhaKishorM/laya). This repository only changes format and precision.
Model tree for Steven10429/laya-typed-decisions-webgpu-q4
Base model
convaiinnovations/laya