ONNX export of NCM small models

README.md

@@ -0,0 +1,38 @@
+---
+license: mit
+library_name: onnx
+base_model: hazyresearch/ncm-checkpoints
+tags:
+  - onnx
+  - webgpu
+  - next-concept-modeling
+---
+
+# NCM small models, ONNX for the browser
+
+ONNX exports of a few of the small [Next-Concept Modeling](https://huggingface.co/papers/2610.14162)
+checkpoints by Hazy Research ([hazyresearch/ncm-checkpoints](https://huggingface.co/hazyresearch/ncm-checkpoints),
+code: [HazyResearch/ncm](https://github.com/HazyResearch/ncm), MIT). All credit for the models goes to the authors.
+They are used by the in-browser demo at [mrfakename/ncm-webgpu](https://huggingface.co/spaces/mrfakename/ncm-webgpu).
+
+| Folder | Source checkpoint |
+| --- | --- |
+| `tinystories/` | `tinystories` |
+| `lm1b-small/` | `lm1b/small` |
+| `sudoku-easy/`, `sudoku-medium/`, `sudoku-hard/` | `sudoku/{easy,medium,hard}` |
+
+Each `model.onnx` (+ `model.onnx.data`, fp32) is one graph that runs the full coarse-to-fine
+generation, every level unrolled with shared weights, followed by the VQ-VAE decoder read-out of each level.
+
+- Inputs: `prefix` (int64, 1 x prefix_len). The sampled models also take `l0` (level-0 codes, drawn
+  from `level0_unigram.json`), `u` (one uniform in [0, 1) per generated code) and `temperature`.
+  Codes are drawn by inverse-CDF on `softmax(logits / temperature)`, which matches `torch.multinomial`
+  with the released `top_k=0, top_p=1` settings.
+- Outputs: `codes`, all levels concatenated, plus the decoded tokens of every level.
+  For `lm1b-small` this is the top-128 logits per position, so the browser can run the released
+  frequency and adjacency penalties (`token_unigram.json`, `meta.json`).
+- `meta.json` has the level sizes and special tokens. `fixtures.json` has reference inputs and outputs
+  from PyTorch. `verification.json` compares ONNX Runtime (CPU) with PyTorch.
+
+Exported with `torch.onnx.export` (opset 17). Linear and area interpolation are rewritten as
+constant matrix products, which is the same linear map.

verification.json

@@ -0,0 +1,3 @@
+{
+ "interp_matmul_max_abs_err": 2.384185791015625e-07
+}