| --- |
| license: other |
| language: |
| - fr |
| - en |
| tags: |
| - atom-native |
| - toroidal-field |
| - not-a-transformer |
| --- |
| |
| # ATOM |
|
|
| Atom-native model. UTF-8 Atomizer packets go into a toroidal field, integrated with RK4. Not a transformer. No attention, no BPE, no GPT-2 tokenizer on the main path. |
|
|
| Siren is the post this field is meant to hold. It does not hold it yet. |
|
|
| Chat Space: [thefinalboss/atom-chat](https://huggingface.co/spaces/thefinalboss/atom-chat). It loads the highest-step checkpoint, skipping `RELEASE`, smoke, pair probes, and role-loss probes. The 12.1M file below is a role-loss run, so the Space may still bind the 10.7M corpus tip. |
|
|
| Updated 2026-10-09. Superseded checkpoints were removed from this repo. |
|
|
| ## Tip |
|
|
| `checkpoints/bi_d64_ansbank/atom_native_step_12100000_bi_d64_ansbank.pt` |
|
|
| Uploaded 2026-10-02 11:28 UTC. Step 12,100,000. d=64, 64 modes. Resumed from the 10.7M Fractus-dataset tip. Planned target in the checkpoint was 13.5M ticks. Stopped at 12.1M. |
|
|
| Recipe saved in the file: |
|
|
| - batched byte-tick, `device=cuda`, `batch_size=1920`, `fast=compile`, TF32 on |
| - `field_bptt=256`, leak frozen at 0.05, `last_atom_dropout=0.5`, `role_loss=True` |
| - episode 8000, atom flush 64, seed `20260913` |
| - data: `data/answers/train_answers.txt` (answer bank, not the full Fractus jsonl) |
| - Atomizer `atomizer-v1-byte-span`, `max_span_bytes=1`, MERGE off, payload-copy off, hard obligatory readout, last-atom on |
| - `bytes_seen=23,558,400,000` (step × batch, not 12.1M raw bytes) |
| - AdamW in the file. Saved arg LR is `5e-4`; the optimizer group says `2e-3` |
|
|
| This is next-byte under teacher force plus a role-loss probe. It is not speech. |
|
|
| ## Parents, kept |
|
|
| - `checkpoints/bi_d64_fds/atom_native_step_10700000_bi_d64_fds.pt` — corpus tip the 12.1M run resumed from. French + English, plus `thefinalboss/fractus-datasets` converted to `Utilisateur:` / `Assistant:` text (2,265,000 pairs, 7.5 Go). Train CE fell from ~1.81 at 9.5M to ~1.54 at 10.6M. |
| - `checkpoints/bi_d64_day/atom_native_step_9500000_bi_d64_day.pt` — parent of the 10.7M file. |
|
|
| ## Last published generation |
|
|
| Generation at 10.6M, greedy, T=0.3, top-k=2. Not re-probed at 12.1M: |
|
|
| - Bonjour, comment ça va ? → `C'est de la fait pas de la famille de la fait d` |
| - Qui es-tu ? → `Je veux vie trais ?` then a pasted `Utilisateur:` |
| - Il était une fois → `C'est pas se sag` |
| - The cat sat on the → `The dis` |
| - Ouvre Siren. → `Alle se` |
|
|
| The field learned the dialogue file as text. It did not learn that the reply has to answer the prompt. |
|
|
| ## Removed 2026-10-09 |
|
|
| 66 `.pt` files. Pod mirrors (`gpu_bi_d64_*`), role and leak probes, smoke, the d32 byte-tick line, `RELEASE/` (pre root-cause, including the 2.9M noise tip), and the 11.0M / 11.4M / 11.7M cuts of this same ansbank run. The 11.7M file was the same d=64 brain with optimizer state; the tip is 12.1M. |
|
|
| ## What this is not |
|
|
| Not a product. Not Siren. Not a tool-caller. Weights are operable. Fluency is the gap between teacher-forced next-byte and a reply that depends on the prompt. |
|
|
