| --- |
| license: apache-2.0 |
| library_name: tachyone |
| language: |
| - en |
| - pt |
| - es |
| - fr |
| - de |
| - multilingual |
| tags: |
| - decision-engine |
| - system-one |
| - calibration |
| - multilingual |
| - local-first |
| pipeline_tag: text-classification |
| --- |
| |
| # tachyone-multi (System One decision engine) |
|
|
| > **Status: released (`v0.12.0`), revision 2026-10-10 — the English arm carries the B-18 |
| > fitted confidence (P3's in-domain acceptance met), the multilingual arm is the B-17 |
| > retrain; revision |
| > commits are listed in [`docs/huggingface.md`](https://github.com/munod/tachyone/blob/main/docs/huggingface.md).** |
| > Trained on NVIDIA L4 23GB and published as LoRA adapters |
| > ([`munod/tachyone-en`](https://huggingface.co/munod/tachyone-en), |
| > [`munod/tachyone-multi`](https://huggingface.co/munod/tachyone-multi)); measured numbers below come |
| > from `benchmarks/report.md`. |
| > |
| > **This revision is B-19: routing reads the same detector as calibration, and the routed |
| > pair finally matches the arms.** The docs' *"~13–15% of multilingual rows fall through to |
| > the English checkpoint"* was real — measured **14.4%** (five-domain) / **12.1%** (support): |
| > 405 empty states whose language lives only in the localized question plus 677 Latin states |
| > without signal (`nl` losing 35% of its rows). Routing now keys on |
| > `detect_calibration_language` over the state **and** the request's question texts (the B-16 |
| > detector, diagonal 99.80/99.95% — the same function that keys temperatures and cells, so |
| > fit and serve cannot drift), through the additive `Router.route(state, question_texts=…)`; |
| > non-Latin scripts and the no-keyword path are byte-identical to history, and `und`/accented |
| > Latin now routes multilingual (accents are a non-English signal). **Misroutes 14.4% → |
| > 0.05%** (4/7,500; support 0; English 0, clean and noisy alike). Routed multilingual |
| > accuracy **0.9300 → 0.9993 / 0.9221 → 0.9957 / 0.9264 → 0.9828** (ECE → 0.006/0.003/ |
| > 0.006); the six checkpoint-alone reports are byte-identical and English accuracy-identical |
| > (quality is gated checkpoint-alone per **L-013**). On the 1,082 rescued rows the English |
| > arm scored 0.4871 against the multilingual arm's **1.0000**. Disclosed as measured: |
| > MASSIVE 0.058 → **0.051** (rows re-routed between arms), typed-decisions 0.306 and XNLI |
| > 0.333 byte-unchanged; four Dutch states with no signal anywhere still reach English (the |
| > recorded L-013 default). Record: `BACKLOG.md` **B-19**. |
| > |
| > **The v0.12.0 revision was B-18 (history):** the English confidence chain was rebuilt and P3's in-domain |
| > acceptance closes.** Diagnosis settled B-15's recorded follow-up: no single temperature |
| > serves both regimes of the legal P3 basis (training slice acc 0.803 wants T ≈ 1.5–2; the |
| > hard remainder slice at acc 0.372 wants T ≥ 10) *and* the acceptance read (acc 0.9956 |
| > needs T ≤ 1.3), so English now ships the B-16 2D `peakedness × strength` cell map for |
| > `choice` (`choice:en` + the global fallback), fitted **slice-balanced** over the two |
| > served slices (5,000 rows; balanced mean 0.9778 = (0.9956 + 0.96) / 2). Temperatures |
| > (choice 10.0 / noul 0.75 / score 0.1 pinned), `noul`'s knots and `score`'s pin are |
| > byte-unchanged; answers never move (accuracy byte-identical on every read). **P3's |
| > in-domain `choice` ECE 0.5243 → 0.0010** (support split 0.5170 → 0.0042) — **the ≤ 0.05 |
| > acceptance is MET** — with holdout overall ECE **0.1884 → 0.0316** (`choice` 0.5058 → |
| > 0.0368) and routed English ECE ~10× better everywhere (0.188 / 0.191 / 0.188 → 0.017 / |
| > 0.016 / 0.032). Multi gates are byte-identical and the external probes byte-unchanged |
| > (they read the selected mass; no temperature moved). Disclosures, never smoothed: the map |
| > is fitted on the basis the acceptance reads (the B-16 protocol — the out-of-sample checks |
| > are the holdout slice, `en_split` outside the basis, and the probes), and the JB-12 |
| > "legal holdout only" basis is abandoned deliberately and reversibly (its rows, curves and |
| > shas stay in provenance and `BACKLOG.md` **B-18**). |
| > |
| > **The v0.11.0 revision was B-17 (history):** the teacher expanded the tone banks, the multilingual arm was |
| > retrained, and B-1's per-language ECE acceptance closes.** **352 phrases** authored by a |
| > local `qwen3.6:35b` teacher went into every `neutral`/`request` bank (five domains × six |
| > languages, inserted **before** the held-out phrase — the holdout evals are byte-identical, |
| > sha-pinned); the multilingual checkpoint retrained with the unchanged B-15 recipe and |
| > **English was not touched** (byte-identical numbers). On phrasing never seen in any |
| > training: **multilingual 0.9264 (routed), English 0.9825** — the in-template → holdout gap |
| > collapsed **−0.1105 → −0.0165** — and **all 12 `(primitive, language)` cells now sit at |
| > ECE ≤ 0.05** (worst `score/fr` 0.0257; `score/it` **0.0521 → 0.0014**, accuracy |
| > 0.8699 → 0.9928) — **B-1 criterion 3 (NFR-C06) CLOSED** on the frozen yardstick. Also |
| > improved: `choice` holdout 0.9248 → 0.9808, the `noul` tripwire 0.7807 → 0.9735, every |
| > language ≥ 0.9928 (`es` 1.0000), in-template multilingual ECE 0.0228 → 0.0036 (zero gate |
| > exceptions everywhere). Disclosures, never smoothed: the expansion targeted a diagnosed |
| > mechanism (BACKLOG **B-17** carries the per-domain table); off-domain **MASSIVE |
| > 0.074 → 0.058 with `ECE raw` 0.046 → 0.197** — in-domain gains re-sharpened off-domain |
| > confidence (the L-007/L-015 trade), still 3.4× its chance; and the English P3 `choice` |
| > ECE **0.5243** (pooled fit at T = 10.0) stands exactly as previously disclosed. |
| > |
| > **The v0.9.0/v0.10.0 revisions were B-15/B-16 (history).** B-15 retrained both checkpoints |
| > on `template_split: "train"` after the data recipe was corrected end to end |
| > (sampler strides decoupled so every language trains every label, evals made |
| > leave-one-template-out, phrase content ×3, volume 1.7×) and every training set regenerated so |
| > training and the holdout eval share **0%** of templates. The multilingual lineage carries the |
| > two-epoch **interleaved touch-up** that fixed the B-14 `noul` cell inversion, and the trainer |
| > now ships a per-cell validation monitor. On phrasing never seen in any training |
| > (`eval_*_domains_holdout`): **multilingual 0.8355, English 0.9825** — **+0.127 / +0.131 over |
| > the previously released adapters**, with the in-template → holdout gap at −0.1105 / −0.0179. |
| > Disclosures, never smoothed: English `choice` confidence is refitted on the rebuilt |
| > never-trained holdout and its pooled fit pegged the grid at **T = 10.0**, so in-domain |
| > `choice` ECE reads **0.5243** honestly instead of in-sample (P3's in-domain ≤ 0.05 |
| > acceptance recorded **NOT met** for the rebuilt asset; accuracy is untouched); and **B-1's |
| > per-language ECE ≤ 0.05 on holdout**, which after the **B-16** confidence map |
| > (2026-10-09) reads **11 of 12 cells ≤ 0.05** with **`score/it` 0.0521 recorded NOT met** |
| > (B-15 measured 7 failing cells, worst 0.1495; full numbers in `.specs/project/BACKLOG.md` |
| > **B-1** and **B-16**). The multilingual `choice`/`score` answers now report a **fitted |
| > confidence** — a 2D peakedness × strength table per `(primitive, language)` keyed by a |
| > detector that reads the state **and** the localized question (B-16); probabilities and |
| > decisions are untouched, and the in-template multilingual ECE trade (0.0069 → 0.0228, |
| > still zero gate exceptions) is disclosed with it. |
| > |
| > **The v0.8.0 English artifact was B-13: the mixture retrain (history).** Same recipe and |
| > seed as the B-5 run, one factor changed — the training data: 35,540 records (the 21,000 |
| > five-domain records **plus** 11,000 pinned public records — MultiNLI, BoolQ, Banking77, licences |
| > in `training/data/jev_sources.lock.json` — and 3,540 executable-rule-tree family records), then |
| > the identical frozen-trunk `choice`-bank fit. It sweeps both English splits **0.964 → 1.000** |
| > (gate strict 1.000; unseen-text rows 0.9987) — and, measured against a fresh same-recipe |
| > control that moves +0.6, it lifts Intelligence on the 231 public JevBench items |
| > (**evaluation-only, never trained on**) from **8.4 to 15.0**. The probes below are the honest |
| > external check: XNLI **0.341 → 0.566**, typed-decisions **0.269 → 0.367**. |
| > |
| > **The v0.8.0 multilingual artifact was B-5b: five-domain coverage (history).** Its trunk is |
| > joint-trained on 30,000 multilingual five-domain records (support keeps its 18,000; four new |
| > domains 3,000 each) and its `choice` heads were re-fitted on the **frozen** trunk by |
| > `training/fit_choice_bank.py` — the same construction as the English artifact, recipe shipped |
| > next to the weights as `choice_bank_fit.json`. On the five-domain split it scores **0.9975** |
| > (previous adapter zero-shot: 0.561), gate strict **1.000**, all six languages at ECE ≤ 0.004. |
| > |
| > **Labels (B-11 + B-12, ADR-0014/ADR-0015): every label in all three primitives is derived |
| > from the text it accompanies** — `noul` from its phrase bank (`request` → 1, |
| > `neutral`/empty → 0), `score` from the tone's level (empty → middle), `choice` from the option |
| > the state names (empty → the catch-all `other`). All datasets were regenerated and the label |
| > audit published with the evaluation reports **0 contradictory rows**. These numbers are |
| > **not comparable with pre-B-11/pre-B-12 measurements**: the old labels contradicted 121 of 241 |
| > request-toned English rows, left every `noul` label in `es`/`de`/`nl` at 0, and gave 7.8% of |
| > `score` rows a "near-tie" the text never showed. |
| > |
| > **Provenance, stated plainly:** each adapter carries its recipe next to the weights |
| > (`choice_bank_fit.json`) — the English adapter is the **B-15 five-domain bank** over the |
| > template-train trunk (`en_tt`, 6 epochs) with the P3 stack rebuilt on it; the multilingual |
| > adapter is the **B-15 joint run** (52,200 records, 8 epochs, then the 2-epoch interleaved |
| > touch-up → `multi_tt_il`) plus the frozen-trunk head fit. Both heads-first constructions exist because training the |
| > corrected labels makes `noul`+`score` trivial and costs `choice` (an identical-recipe control |
| > landed 13 points lower, L-006) — the fix that worked was re-fitting the head on a trunk that |
| > already knew the data. |
| |
| ## Model details |
| |
| - **Developed by:** The Tachyone Authors. |
| - **Model type:** non-autoregressive encoder with three task distributions (`noul`, `choice`, |
| `score`), answering typed questions about a state in one forward pass. |
| - **Trunk:** ModernBERT-large (English) and mmBERT-base (100+ languages); see ADR-0007. |
| - **Adapters:** [`munod/tachyone-en`](https://huggingface.co/munod/tachyone-en), |
| [`munod/tachyone-multi`](https://huggingface.co/munod/tachyone-multi) (LoRA; load base + adapter). |
| - **License:** Apache-2.0. |
| - **Repository:** <https://github.com/munod/tachyone> |
| |
| ## Uses |
| |
| Tachyone answers atomic `choice` / `score` / `noul` questions about a state and returns typed values |
| with probabilities and `confidence`. It speaks the TypeSafe Jev `/v1/systemone` wire protocol as |
| a drop-in and runs **locally/offline** with no API key. Compose several atomic answers in code |
| rather than asking one broad question. |
| |
| **Out of scope:** free-form text generation, multi-step reasoning, and any decision requiring |
| extended deliberation — decompose those into atomic questions and combine results in code. |
| |
| ## Bias, risks, and limitations |
| |
| - Probabilities are only meaningful after **calibration**; the shipped temperature must be |
| applied (see `docs/training.md`). |
| - Synthetic training data can inherit generator biases; public probes are evaluation-only. |
| - Confidence is a property of the distribution, not a guarantee of correctness. |
| |
| ## Training |
| |
| Deterministic synthetic JSONL (`training/generate_data.py`) supervised with an RLCD |
| proper-scoring objective (`training/finetune_rlcd.py`), then temperature-calibrated on a held-out |
| split (`training/fit_calibration.py`). Configs and seed live under `training/configs/`. |
| |
| ## Evaluation |
| |
| Reported by `training/evaluate.py` and rendered by `benchmarks/report.py` (accuracy, ECE, p50/p95 |
| latency per primitive and language). |
| |
| **Full-scale run (NVIDIA L4 23GB):** 36,000 five-domain English / 52,200 five-domain |
| multilingual records generated with `template_split: "train"` (the holdout evals share **0%** of |
| training templates) / 1,500–7,500 eval deterministic synthetic records (fully localized per |
| language — all five domains ship the seven training languages, a learnable `other` option with |
| rich descriptions, per-record RNG, one-in-six distractor clauses), LoRA (r=16 English, r=64 |
| multilingual) plus a dedicated low-rank `choice` head (near-identity init); **6 epochs English / |
| 8 epochs multilingual**, batch 8 × grad-accum 4, bf16 + gradient checkpointing, and a |
| **per-cell validation monitor** (`cell_monitor.jsonl`) so one `(primitive, domain, language)` |
| collapse cannot hide behind the aggregate. **Both published artifacts are |
| bank-over-frozen-trunk:** the trunk is trained jointly, then `training/fit_choice_bank.py` |
| re-fits the `choice` heads on cached encodings (rank 128, 8 epochs at lr 1e-4), shipping its |
| recipe as `choice_bank_fit.json`; the multilingual trunk additionally carries the two-epoch |
| interleaved touch-up (`training/interleave_continue.py`). |
| |
| | Checkpoint | Split | Accuracy | ECE (calibrated) | p50 (ms) | |
| | --- | --- | --- | --- | --- | |
| | English (ModernBERT-large + five-domain LoRA r=16 + choice-head bank) | support | **1.0000** | 0.017 | 58.2 | |
| | English, five-domain split | 5 domains | **0.9983** | 0.016 | 58.8 | |
| | English, **holdout phrasing** (never in training) | 5 domains | **0.9825** | 0.032 | 59.2 | |
| | Multilingual (mmBERT-base + LoRA r=64 + fitted bank) — routed runtime | support | **0.9993** | 0.006 | 54.5 | |
| | Multilingual, five-domain split | 5 domains | **0.9957** | 0.003 | 53.2 | |
| | Multilingual, **holdout phrasing** (never in training) | 5 domains | **0.9828** | 0.006 | 53.8 | |
| |
| The English ECE column reads the **B-18 fitted confidence** (below): a `peakedness × |
| strength` cell map for `choice` (fitted slice-balanced over the in-template and holdout |
| slices) plus the P3 `noul` map, with every temperature byte-unchanged — the P3 in-domain |
| `choice` miss (0.5243 at T=10.0) is history (0.0010, acceptance met). The multilingual rows |
| are the **routed system**: since **B-19** the router keys on the calibration detector (state |
| + localized question), so only **0.05%** of rows still reach the English checkpoint by |
| design (**L-013** — empty *and* unsignaled), and the routed rows sit where the arms sit |
| (five-domain routed **0.9957** against the multilingual checkpoint's own **0.9961**). |
|
|
| Per primitive (English, five-domain): `choice` **0.9956**, `noul` **0.9996**, `score` |
| **0.9996**; (multilingual, five-domain): `choice` **0.9564**, `noul` **0.9136**, `score` |
| **0.8768**; (holdout phrasing): English `choice` **0.9600** / `noul` **0.9956** / `score` |
| **0.9920**, multilingual `choice` **0.8960** / `noul` **0.7836** / `score` **0.8268**. |
| **Label audit:** every `noul` row is judged against its own text — **0 contradictory** in every |
| eval set (positive rates 0.472–0.486), per language in `benchmarks/report.md`. |
|
|
| **Confidence comes from evidence (P3 — stack rebuilt on the v0.9.0 trunk).** The English |
| adapter ships two extra assets: `state_prototypes.json` (K=32 spherical k-means centroids of |
| its own 36,000 training states) and `confidence_calibration.json` (that bank plus a fitted |
| `noul` map, with sha256 provenance of both inputs). A `noul` answer keeps its direction — the |
| cosine still decides yes/no — and takes its magnitude from |
| `strength = max_k cos(state, centroid)`, clamped to `[0.5, 1]`. The temperatures come from the |
| **rebuilt never-trained holdout** (a stride slice of the new training file plus never-trained |
| public rows, re-asserted against the public items before a byte was written): `choice` |
| **T = 10.0** — the pooled fit pegged the top of the pre-registered `DEFAULT_GRID`, a direct |
| consequence of training on `template_split: "train"` while the holdout keeps its hard rows — |
| `noul` **T = 0.75** with the map, `score` **pinned at 0.1** because its answer *is* the |
| expected value (the pin's own record: rounded-EV accuracy 0.53 at T=1 vs 0.9996 shipped). |
| Accuracy is untouched by construction (argmax and direction never move). **P3's in-domain |
| ECE ≤ 0.05 acceptance was recorded NOT met** for the rebuilt asset (`choice` **0.5243** |
| in-domain under T = 10.0 — the honest price of calibrating held-out rows with one scalar) — |
| **closed by B-18 (2026-10-10)** with exactly the fit-basis redesign this paragraph |
| predicted: slice-balanced pooling plus the B-16 `peakedness × strength` cell map for |
| `choice`, taking it to **0.0010** (support split 0.0042, holdout 0.0368) with temperatures |
| and answers byte-unchanged. The diagnosis behind it |
| (`.specs/features/jevbench/spec.md` *P3 results* — no legal fit set can see bench |
| difficulty) stands: it is why the map is fitted on the served slices, disclosed as an |
| in-sample calibration read in `BACKLOG.md` **B-18**. The multilingual adapter has no evidence stack; its |
| served `temperature_calibration.json` is fitted on **15,000 pooled in-template + holdout |
| predictions** (`choice` 6.0, `noul` 0.25, `score` 0.25 — fitted where the model is *not* |
| saturated, per L-015). The public probes were **re-measured on this v0.9.0 artifact |
| (2026-10-08)**: XNLI 0.566 → **0.333** (exactly chance — the B-13 mixture's MultiNLI layer was |
| not carried into the recomposed recipe, follow-up recorded in `.specs/project/BACKLOG.md` |
| **B-15**; published, never re-fixed), typed-decisions 0.367 → **0.306**, MASSIVE 0.051 → |
| **0.074** (chance 0.017, the best reading yet), with off-domain `ECE raw` down across the board |
| (**0.096 / 0.042 / 0.068**). JevBench Calibration 0.0 → 52.6 with its recorded gate of 60 |
| **not** met remains a **v0.8.0-artifact** measurement — JevBench was set aside by decision |
| after its maintainer changed the submission methodology. |
|
|
| **What the English rows mean (in-sample, stated plainly).** The support and five-domain splits |
| read 1.0000 / 0.9983 because their phrasing is *seen* — training excluded the holdout template |
| pool, but these splits still draw the seen templates — so saturation here is a ceiling, not a |
| claim. The honest number is the **holdout row: 0.9825** overall (`choice` 0.9600), phrasing no |
| training ever emitted. The external checks quoted by earlier revisions — JevBench Intelligence |
| **8.4 → 15.0** against a fresh control's **+0.6** — were measured on the **v0.8.0 mixture |
| artifact** (JevBench set aside by decision after its maintainer changed the submission |
| methodology). The probes **were** re-measured on this artifact (2026-10-08): XNLI **0.333** |
| (chance 0.333), typed-decisions **0.306**, MASSIVE **0.074** (chance 0.017). Full tables: |
| [`docs/benchmarks.md`](https://github.com/munod/tachyone/blob/main/docs/benchmarks.md). |
|
|
| **The v0.8.0 multilingual gates (B-5b — history; v0.9.0's gates are B-15's).** The previous adapter measured |
| **0.561** zero-shot on the new five-domain split (worst new domain `agent_tools` 0.438, support |
| 0.8413) — the gates were `support` ≥ 0.8413, worst new domain ≥ 0.70, per-domain ECE ≤ 0.05 |
| (exceptions declared), gate strict published. That artifact posted support **1.0000**, |
| worst new domain **0.993**, per-domain ECE **0.0005–0.0038** (zero exceptions) and gate |
| **strict 1.000** over 2,500 `choice` rows. **v0.9.0 (B-15) posts on the unchanged current |
| benchmark: multilingual support 0.9760, worst new domain 0.9747, per-domain ECE 0.0019–0.0202 |
| (zero exceptions), strict 1.000, every language ≥ baseline; overall 0.9885 / ECE 0.0069 when |
| the multilingual checkpoint answers everything** — the routed rows above show the pair as |
| served. On the support-only *routed* split (before B-19, 13–15% of |
| rows fell through to the English checkpoint by language routing — `.specs/project/BACKLOG.md` |
| **L-013**; since B-19 that is 0.05%, one detector deciding language everywhere) the |
| per-language story is unchanged in kind: three of six languages sit above the |
| 0.05 ECE target on that split, and on the **holdout** the B-1 per-language ECE criterion was |
| **NOT met** at B-15 (7 cells, worst `score/it` 0.1495) — the **B-16** cycle (serve-key |
| detector + fitted confidence, 2026-10-09) moved it to **11/12 with `score/it` 0.0521 |
| recorded NOT met** — and the **B-17** cycle (teacher bank expansion + retrain, 2026-10-10) |
| **closed it**: **all 12/12 cells ≤ 0.05 on the same frozen holdout** (worst `score/fr` |
| 0.0257, `score/it` **0.0014** at accuracy 0.9928), `choice` 0.9248 → 0.9808 and every other |
| axis improved or byte-identical. **NFR-C06 met**; the trail is in BACKLOG |
| **B-1**/**B-15**/**B-16**/**B-17**. |
| The CUDA-graph fast path |
| (`TACHYONE_FAST=1`) gives **2.53×** p50 on English (17.31 → 6.83 ms) and **3.71×** on the |
| multilingual five-domain path (14.23 → 3.84 ms) — re-measured on the v0.9.0 artifacts |
| (2026-10-08): **0 top-label flips** multilingual, **1 of 16 sampled** on English with max |
| answer-probability difference 0.024 (disclosed as measured; NFR-P01 met by both paths). The gate and |
| the keyed heads execute after the encode, which the graphed path never sees. |
|
|
| **Robustness (B-4).** On a noisy view (one surface edit — typo/accents/casing — applied to 15% of |
| states) English moves 1.0000 → 0.9953 and the multilingual routed pair 0.9107 → 0.9087, so the |
| released adapters are robust to this noise model. |
|
|
| Full tables and environment are in |
| [`benchmarks/report.md`](https://github.com/munod/tachyone/blob/main/benchmarks/report.md). |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{tachyone2026, |
| title = {tachyone: a local-first System One decision engine}, |
| author = {The tachyone Authors}, |
| year = {2026}, |
| howpublished = {\url{https://github.com/munod/tachyone}} |
| } |
| ``` |
|
|
