File size: 7,560 Bytes
b18482b e99098b b18482b e99098b b18482b e99098b b18482b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 | ---
license: cc-by-nc-4.0
language:
- en
library_name: pytorch
pipeline_tag: text-generation
tags:
- ncm
- coarse-to-fine
- vq-vae
- tool-use
- from-scratch
- tiny
datasets:
- HuggingFaceFW/fineweb-edu
- roneneldan/TinyStories
- wikimedia/wikipedia
- abisee/cnn_dailymail
- corbt/enron-emails
- Yale-LILY/aeslc
- HuggingFaceTB/everyday-conversations-llama3.1-2k
- HuggingFaceTB/smol-smoltalk
- Salesforce/xlam-function-calling-60k
---
An independent, from-scratch implementation of **Next-Concept Modeling (NCM)**, Kumbong, Khare et al., *Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation*, 2026 ([arXiv:2610.14162](https://arxiv.org/abs/2610.14162), official code [HazyResearch/ncm](https://github.com/HazyResearch/ncm)). All credit for the method goes to the authors.
**Files:** `tiny-ncm.pt` (weights + config, 146 MB), `bpe.json` (text tokenizer), `registry.json` (tool list).
Run it with the code at https://github.com/amar683/tiny-ncm:
```bash
git clone https://github.com/amar683/tiny-ncm && cd tiny-ncm
python3 -m venv .venv && .venv/bin/pip install -r requirements.txt
hf download amartomar683/tiny-ncm tiny-ncm.pt --local-dir weights
.venv/bin/python chat.py --dry-run
```
# Model card: tiny-ncm
## Summary
- **Type:** NCM (coarse-to-fine, block-wise) language model for tool use, trained from scratch.
- **Method:** Next-Concept Modeling, Kumbong, Khare et al., 2026 ([arXiv:2610.14162](https://arxiv.org/abs/2610.14162)); see below.
- **Size:** 36.4M parameters: generator 20.9M (d=384, 8 layers, 3 query heads / 1 KV head) and multiscale residual VQ
tokenizer 15.5M (5 levels, 1024-entry codebooks). Byte-level BPE vocabulary of 16,384. Context 4096.
- **Checkpoint:** `runs/task6` (released as `tiny-ncm.pt`, fp32, 146 MB). Code: https://github.com/amar683/tiny-ncm (MIT).
- **Hardware:** one MacBook (Apple M5, 24 GB), PyTorch MPS, bf16. About 12 hours of training in total (VQ tokenizer 1.3 h, generator 10.5 h).
## Method: the NCM paper
This model is an independent, from-scratch implementation of **Next-Concept Modeling (NCM)** from
> **Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation**
> Hermann Kumbong, Ishan S. Khare, Iddah Mlauzi, Andrew Woen, Charles Ding, E. Kelly Buchanan, Simran Arora, Azalia Mirhoseini, Christopher Ré
> Stanford University and Together AI, 2026. [arXiv:2610.14162](https://arxiv.org/abs/2610.14162) · official code: [HazyResearch/ncm](https://github.com/HazyResearch/ncm)
All credit for the method goes to the authors. This project is not affiliated with them, and no code was taken from the official repository.
**From the paper:** a multiscale residual VQ-VAE turns text into coarse-to-fine levels of discrete codes; a transformer generates
one level at a time, predicting all codes of a level in parallel; nested dropout in the tokenizer; conditioning (code) corruption
and per-level loss weighting when training the generator.
**Changed or added here:** text is split into 32-token blocks that are generated one after another with a KV cache (each block
coarse-to-fine over 5 levels), so a chat can grow to 4096 tokens; level positions attend to a 128-token window of context to fit
Apple-silicon speed; Muon optimizer; a small size (36M) trained in about 12 hours on one MacBook; chat and tool-use training,
plus the app-side checks in `assistant/`.
If you use this, please cite the paper:
```bibtex
@article{kumbong2026ncm,
title = {Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation},
author = {Kumbong, Hermann and Khare, Ishan S. and Mlauzi, Iddah and Woen, Andrew and Ding, Charles and
Buchanan, E. Kelly and Arora, Simran and Mirhoseini, Azalia and R{\'e}, Christopher},
journal = {arXiv preprint arXiv:2610.14162},
year = {2026}
}
```
## Intended use
A personal assistant that turns everyday requests into tool calls (alarms, reminders, timers, calendar, read-only Gmail,
web and Wikipedia lookups, weather, notes, calculator) and makes small talk. Use it together with the app in
`assistant/`, which checks every tool call against the user's words. **Not** for coding, reasoning, medical, legal or
financial advice, or anything where a wrong answer matters.
## Training data
| Data | License | Used for |
|---|---|---|
| [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) (sample-10BT) | ODC-By 1.0 | general English |
| [TinyStories](https://huggingface.co/datasets/roneneldan/TinyStories) | CDLA-Sharing 1.0 | simple English |
| [Wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia) (20231101.en) | CC BY-SA 3.0 / GFDL | text, lookup answers |
| [CNN/DailyMail](https://huggingface.co/datasets/abisee/cnn_dailymail) | Apache-2.0 | summaries |
| [Enron emails](https://huggingface.co/datasets/corbt/enron-emails), [AESLC](https://huggingface.co/datasets/Yale-LILY/aeslc) | not stated (public record) | email style, subject lines |
| [everyday-conversations](https://huggingface.co/datasets/HuggingFaceTB/everyday-conversations-llama3.1-2k), [smol-smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smol-smoltalk) | Apache-2.0 | small talk (non-coding only) |
| [xLAM function-calling 60k](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k) | CC-BY-4.0, "research purposes only" | reading tool schemas |
| Synthetic tool conversations (`ncm/synth_tools.py`, `make_copy_data.py`) | made for this project | tool calls |
The weights were **not** trained on web pages collected by `harvest_web.py` (that experiment, `runs/web1`, was not adopted).
**Weights license: CC BY-NC 4.0.** Because of the xLAM terms and the data with no stated license, the weights are released for
**research and non-commercial use**, with credit to the datasets above. xLAM asks users to cite APIGen:
Liu et al., *APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets*, 2024.
## Training
1. BPE tokenizer (16k) on the mix.
2. VQ tokenizer: 97.6% exact text reconstruction, 95% exact tool JSON.
3. Generator pretraining at context 1024 (Muon + AdamW, warmup-stable-decay schedule).
4. Chat/tool phases at 4096: synthetic tool data, then "copy-focused" data with fresh random names, times and places in every
example, then 2h45m of mostly English, then 2 × 1h of copy-focused data.
Details and every run: [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md).
## Evaluation
| Test | Model + app checks | Model alone |
|---|---|---|
| 500 new requests | 490 / 500 | n/a |
| 60 hard requests written before the fixes | 57 / 60 | n/a |
| 153 everyday requests, right tool | 153 / 153 | n/a |
| 153 everyday requests, every detail exact | 153 / 153 | 64 / 153 |
| Math (90) | 89 / 90 | n/a |
| Coding (40) | 5 / 40 | n/a |
## Limitations and risks
- **Copying.** The model often garbles names, numbers and times ("London" → "Singapore"); the app fixes them from the
user's words. Without the app, expect about half of the tool calls to have a wrong detail.
- **English.** Free-form replies are rough; the app replaces replies that don't match the tool result.
- **Reasoning.** No multi-step math, no code.
- **Web answers** are picked from the top search results by word matching; they can be wrong or mix methods.
- **No safety training.** The model has no refusal or content filtering. The app asks before any action that sends
messages or calls someone, and never guesses those tools from a garbled name.
- **Privacy.** Everything runs locally. Web questions go to DuckDuckGo and the pages' sites; Gmail access is read-only.
|