card: serving requirements (draft head + binding) and scope the bit-exact claim to the checkpoint
ea63c5a verified |
Download README.md from JasonW2025/Tesseract-R08-20261006: direct link, hf CLI and curl.
- Browser
- Download file 10.5 kB
-
https://huggingface.co/JasonW2025/Tesseract-R08-20261006/resolve/main/README.md
- Command line
-
hf download hf://JasonW2025/Tesseract-R08-20261006/README.md
-
curl -L -H "Authorization: Bearer $HF_TOKEN" -o README.md https://huggingface.co/JasonW2025/Tesseract-R08-20261006/resolve/main/README.md
10.5 kB
| license: other | |
| license_name: tesseract-research-license | |
| tags: | |
| - tesseract | |
| - tessera | |
| - fp8 | |
| - trellis-quantization | |
| - moe | |
| - speculative-decoding | |
| - dgx-spark | |
| - sm_121a | |
| library_name: custom | |
| pipeline_tag: text-generation | |
| # Tesseract R08-20261006 | |
| Research artifacts. Cannot be served using existing engines except our proprietary engine at the moment. | |
| Tesseract is a 126.3e9-parameter sparse language model served on a single NVIDIA | |
| DGX Spark (GB10, sm_121a). This repository is the checkpoint the serving lane | |
| actually ran: an fp8 dense stack with **Tessera WINDOW trellis** routed experts, | |
| an fp8 lm_head, and an MTP draft block for speculative decoding. | |
| > Research artifact, measured on one DGX Spark against this exact checkpoint. | |
| > Not a claim about other hardware, other quantisations, or another serving stack. | |
| ## Status — read this before quoting the flags | |
| The checkpoint files in this repository are a **bit-exact copy of the serving | |
| checkpoint**: every one of the 75,313 files inside `packed/` matches the | |
| checkpoint's own integrity manifest, and all 33 `ple-table/` shards, 52 dense | |
| shards and 123 `evidence/` records match by SHA-256. Two things are NOT the | |
| checkpoint: the checkpoint's own `README.md` is replaced by this card, and | |
| `draft-head/` is an additional serving artifact (see Serving requirements). | |
| **The repository does not claim its own qualification.** Its manifests say so, | |
| and they are published unmodified: | |
| | file | field | value | | |
| |---|---|---| | |
| | `complete.json` | `serving_qualified` | `false` | | |
| | `complete.json` | `quality_qualified` | `false` | | |
| | `complete.json` | `performance_qualified` | `false` | | |
| | `complete.json` | `status` | `complete_all_49_routed_owners` | | |
| | `serving-bundle.json` | `status` | `complete_candidate_for_qualification` | | |
| | `execution.json` | `whole_model_graph_qualified` | `false` | | |
| In other words the artifact is **structurally complete and self-declared | |
| unqualified**: it is a *candidate for* qualification, and the numbers in this | |
| card are the measurements we hold against it, not vendor certification. | |
| Editing the manifests to flip those flags would break the hash chain that makes | |
| this repository verifiable, so they stay as produced. | |
| Known exception, stated rather than hidden: `complete.json`'s entry for | |
| `config.json` describes an **older** copy of that file (4382 bytes, | |
| `76d73213…`) than the one shipped (4417 bytes, `2e186993…`) — the config was | |
| updated after the manifest was emitted. A third-party run of the manifest check | |
| will report exactly that one mismatch and no others. | |
| ### What was actually verified against this repository | |
| | check | result | | |
| |---|---| | |
| | packed/ archives vs `complete.json` | 49/49 archives, 75,313/75,313 files, 0 mismatched | | |
| | dense shards (52) + `ple-table` (33) | SHA-256 identical to the serving checkpoint | | |
| | `evidence/` (123 records) | SHA-256 identical | | |
| | `config.json` / tokenizer set | identical; manifest entry stale, see above | | |
| | serving identity of the checkpoint | `d4cdd7b9a35e79c3` — fixed greedy decode | | |
| | release gate on the checkpoint | pass on canary, KL, NLL, top-1, MMLU subset | | |
| `packed/*.tar` are upload-time archives: extract them into `packed/` before | |
| serving (the tar members are `layer-NN/<file>`; extract with `-C packed/`). | |
| ## Layout — packed/ ships as archives | |
| `packed/` is published as **one tar per layer** (`packed/layer-NN.tar`) because | |
| this checkpoint has 75,537 files and Hugging Face caps a repository at 20,000. | |
| Extract before serving: | |
| ```bash | |
| for t in packed/layer-*.tar; do tar -xf "$t" -C packed/; done | |
| ``` | |
| That restores the directory-per-layer layout the loader reads. Everything else | |
| (`config.json`, the dense `model-protected-*.safetensors`, `ple-table/`, | |
| `evidence/`) is present as plain files. | |
| ## Serving requirements | |
| The measured serving configuration needs three things this repository does not | |
| contain in the checkpoint itself: | |
| | requirement | identity | | |
| |---|---| | |
| | engine | a Tessera-capable build that loads the WINDOW trellis wire; not published here | | |
| | MTP draft head | `draft-head/draft-head.json`, sha256 `878d057fe94a82c2d9237cdfb07c56786b643ef3c3256eaa42f910792993faed` | | |
| | draft-head binding | `draft-head/provenance.json` pins config sha256 `2e1869932920dd1b6dc2931900dc74b1c09b3232b0a0ed2010dfba3cbfa37760` (the 4417-byte config shipped here) and tokenizer sha256 `0997f410…` | | |
| `draft-head/` was added on 2026-11-09 because the original upload omitted it: | |
| the checkpoint does not carry the draft head, the serving recipe requires it by | |
| pinned sha256, and without it the published set cannot arm MTP d3. Its | |
| provenance file independently confirms that the 4417-byte `config.json` is the | |
| serving config — so `complete.json`'s 4382-byte entry for `config.json` is the | |
| stale one (see Status). | |
| Measured serving configuration, for reproduction of the numbers above: KV cache | |
| bf16 at 65,536 tokens, prefill budget 4,096, fp8 dense weights + fp8 MMA head, | |
| MTP depth 3, temperature 0, thinking off, prefix-cache granularity 1,024. | |
| ## The wire | |
| | property | value | | |
| |---|---| | |
| | parameters | 126.3e9 (5.5e9 dense + 120.8e9 experts) | | |
| | layers / hidden / vocab | 48 / 2,560 / 248,320 | | |
| | experts | top-10 of 512 per token | | |
| | dense compute | fp8 (attention, dense FFN, lm_head MMA, MTP block) | | |
| | routed experts | Tessera WINDOW trellis: B4 rate-4 / T3 rate-3 / W832 per-column Bresenham | | |
| | bits per weight | 3.9 experts, 8.4 dense, 4.1 overall (this repository's accounting) | | |
| | KV cache | bf16, 65,536 tokens | | |
| | on-disk | 166.2 GiB, 75,537 files before packing | | |
| ## Memory on one DGX Spark (GB10, unified 121.63 GiB pool) | |
| Serving plan, from the engine's own boot ledger: | |
| | component | GiB | | |
| |---|---| | |
| | Tessera compact routed experts (resident planes) | 60.54 | | |
| | model weights (resident; n-gram table mmap'ed) | 5.37 | | |
| | spec snapshot rows | 3.45 | | |
| | layer objects (GR sites, GDN, QSA, PLE) | 3.00 | | |
| | activation rows incl. the 2,048-row scoring plane | 2.06 | | |
| | loader staging (pinned host, freed after the last layer) | 1.81 | | |
| | KV cache pool | 1.68 | | |
| | prefix cache | 1.40 | | |
| | request slot state | 0.86 | | |
| | vision tower weights | 0.84 | | |
| | draft block + draft head | 0.70 | | |
| | MoE scratch | 0.35 | | |
| | vision workspace | 0.29 | | |
| | draft-only physical head | 0.17 | | |
| | engine, GEMM workspace, dense prefill bridge, n-gram staging | 0.22 | | |
| | **total plan** | **82.58** (79.23 device + 3.35 pinned) + 4.00 headroom | | |
| Peak observed while idling after boot: 91 GB used of 121 GB, 30 GB available, | |
| engine RSS 6.5 GiB. The 60.54 GiB of expert planes dominate; the rest is KV, | |
| snapshots and activations. | |
| ## Measured on one DGX Spark | |
| | metric | value | note | | |
| |---|---|---| | |
| | cold prefill | **1,543 tok/s** | 7.2k distinct fillers, cache miss confirmed | | |
| | decode, one stream | **48.3 tok/s** | MTP d3, 2.54 tok/step, acceptance 0.513 | | |
| | decode, 8 streams | **122 tok/s** | | | |
| | TTFT | 141–177 ms short | t ≈ 143 ms + n/1664 | | |
| | NLL vs bf16 teacher | **4.3183** vs 4.2966 | 2,796 tokens, top-1 0.3530 vs 0.3484 | | |
| | MMLU | **88.07%** | n=570, 5-shot, temp 0 | | |
| | GSM8K | **96.59%** | n=1,319, 8-shot CoT | | |
| | tool-call, 24-case hard | **95.8%** | tool-name accuracy 100% | | |
| | TEB 3.0, hardmode | **89/100** | 104 scenarios; deployability 76, responsiveness 45 | | |
| | TEB suites | GSM8K 98.0% (200), MMLU 91.7% (300), IFEval 83.5%/88.4% | each suite's own report | | |
| | scored-prefill cost | **0.061 ms/prompt token** | was 3.905 before the chunking fix | | |
| | MMLU-Pro (full test set) | **67.95%** | 8,176/12,032, 14 categories, 5-shot answer-only, 11.4 h | | |
| | KL(teacher ‖ candidate) | **0.0553 nats/position** | full vocabulary, 2,796 positions, frozen population | | |
| ### KL vs the bf16 teacher | |
| Full-vocabulary KL over the frozen quality population: **0.0553 nats/position** | |
| (154.55 nats over 2,796 positions, every one of the 248,320 logits per | |
| position). Both sides are gated, not assumed: | |
| - the teacher logits reproduce the frozen teacher receipt's per-record NLL to | |
| **0.0000 nats** on all 32 records, which also fixes the row alignment; | |
| - the candidate logits reproduce the served API's logprobs to **1.1e-5** nats. | |
| For scale, the same population gives a mean NLL of 12,074.1 (candidate) vs | |
| 12,013.3 (teacher), i.e. +0.0217 nats/token, and top-1 agreement of 0.3530 vs | |
| 0.3484. KL is larger than the NLL gap because it counts redistributed tail mass | |
| that never changes the argmax. Per record it splits cleanly: thinking-mode | |
| records 5.4-10.7 nats, non-thinking records 0.3-3.2. | |
| ### MMLU-Pro, full test set | |
| All **12,032 questions** (14 categories), 5-shot answer-only, temperature 0, | |
| thinking off: **67.95% (8,176/12,032)** in 11.4 hours. Per category it ranges | |
| from 84.2% (psychology) to 58.2% (physics). 70% of questions were scored from a | |
| single request via the top-20 logprob candidates; the other 30% fell back to one | |
| request per choice letter when a letter sat outside those candidates. | |
| ## Thinking / reasoning settings | |
| Controls: `enable_thinking` (alias `thinking`) inside `chat_template_kwargs`, and | |
| `reasoning_effort` ∈ {none, minimal, low, medium, high, xhigh, max}. Reasoning is | |
| returned in `message.reasoning_content` and counted in | |
| `usage.completion_tokens_details.reasoning_tokens`. History knobs: | |
| `preserve_thinking`, `clear_thinking`, `drop_thinking`. | |
| Measured on this checkpoint, same prompt, temperature 0 — **four observable | |
| states**, not seven: | |
| | setting | thinking | reasoning tokens | | |
| |---|---|---| | |
| | `reasoning_effort: none` / `enable_thinking: false` | off | 0 | | |
| | `minimal` ≡ `low` | on | 56 | | |
| | `medium` | on | 68 | | |
| | `high` ≡ `xhigh` ≡ `max` ≡ `enable_thinking: true` | on | 39 | | |
| **Every accuracy number above was measured with thinking OFF** | |
| (`enable_thinking: false`, and `--no-think` for TEB). The effort ladder is not | |
| monotonic in token count on this checkpoint, so an effort level is not a | |
| fine-grained control. | |
| ## Serving identity | |
| `d4cdd7b9a35e79c3` — the SHA of a fixed greedy decode. Any change to the | |
| weights, the quantisation or the serving numerics changes it; it is the | |
| handshake this checkpoint is identified by. | |
| ## What this repository is not | |
| It is not the newest wire in the lab. A rate-3 (sub-4-bit, ~3.09 bpw) re-encode | |
| of the same official FP8 source exists internally (`SUB4`, October–November | |
| 2026) but its receipt is unfinished: per-layer gate results, serving gates | |
| (boot, C1/C4/MTP parity, NLL) and memory/fit numbers were never filled in, so it | |
| is not qualified and not published. Benchmark and compare against this | |
| repository only as the r08 wire. | |