README and tips: 10.7M Fractus injection, role probes

README.md

@@ -13,34 +13,37 @@ tags:
 
 Atom-native model. UTF-8 Atomizer packets go into a toroidal field, integrated with RK4. Not a transformer. No attention, no BPE, no GPT-2 tokenizer on the main path.
 
-Siren is the post this field is meant to hold (OS chat, tools, code). It does not hold it yet. A tool call is a line ATOM writes. The OS executes it. That is not wired.
+Siren is the post this field is meant to hold. It does not hold it yet.
 
-Chat Space: [thefinalboss/atom-chat](https://huggingface.co/spaces/thefinalboss/atom-chat). It loads the highest-step checkpoint in `checkpoints/`, skipping `RELEASE`, smoke runs, and pair-probe files. It rechecks every 30 minutes.
+Chat Space: [thefinalboss/atom-chat](https://huggingface.co/spaces/thefinalboss/atom-chat). It loads the highest-step checkpoint, skipping `RELEASE`, smoke, pair probes, and role-loss probes.
 
-## Tip on this card
+## Tip
 
-`checkpoints/bi_d64_day/atom_native_step_9500000_bi_d64_day.pt`
+`checkpoints/bi_d64_fds/atom_native_step_10700000_bi_d64_fds.pt`
 
-Corpus tip. d=64, 64 modes, leak frozen at 0.05 (not learnable). Day run used batch 2112, BPTT 256, free-run aux horizon 16, prompt-dependence penalty 0.35 on batch halves whose gold bytes differ. French + English shards (~6.4 Go + ~6.6 Go) plus a 4.5 Mo OS-pair shard. Not a wipe: resumed from 7.6M, then 7.8M when the batch changed.
+Corpus tip after the Fractus-dataset injection. d=64, 64 modes, leak frozen at 0.05. Resumed from the 9.5M day run. French + English stayed in that pass, plus `thefinalboss/fractus-datasets` converted to `Utilisateur:` / `Assistant:` text (2,265,000 pairs, 7.5 Go). Train CE fell from ~1.81 at 9.5M to ~1.54 at 10.6M. That is next-byte under teacher force. It is not speech.
 
-Train CE at 9.4–9.5M: about 1.81. That is next-byte under teacher force. It is not speech.
+Generation at 10.6M, greedy, T=0.3, top-k=2:
 
-## What was measured
+- Bonjour, comment ça va ? → `C'est de la fait pas de la famille de la fait d`
+- Qui es-tu ? → `Je veux vie trais ?` then a pasted `Utilisateur:`
+- Il était une fois → `C'est pas se sag`
+- The cat sat on the → `The dis`
+- Ouvre Siren. → `Alle se`
 
-Leak. A learnable field leak drifted to ~0.36 and collapsed context. Frozen at 0.05. CE went from ~2.45 just after the cut to ~1.82 at 7.2M, then ~1.81 at 9.4M. Extra corpus ticks after 7.2M did not close the gap.
+The field learned the dialogue file as text. It did not learn that the reply has to answer the prompt.
 
-Generation, greedy, T=0.3, top-k=2:
+## Role-loss probes, not the tip
 
-- 7.2M: `Bonjour, comment ça va ?` and `Qui es-tu ?` both returned `C'est pas se pas`. English `The cat sat on the` returned `The see`. Zeroing `last_atom_scale` still returned `C'est pas…`. The attractor is in the field, not only the bigram readout.
-- 7.6M, after a prompt-dependence penalty on corpus halves: the two French prompts diverged (`Je ne sais pas de la fait…` / `Je suis pas de la marien.`) and still did not answer.
-- 9.4M day tip: `Bonjour` → `Je suis au monde de.` ; `Qui es-tu ?` → `Je suis au mon de l'anter de la commune de l'au` ; `Il était une fois` → `Je l'ai des pas de la commune de l'autre.` ; `The cat sat on the` → `The see`.
+CE only after `Assistant:`, gold byte reinjected, no free-run, leak frozen.
 
-Prompt loss. Putting the real prompts in the loss works, and overfits. On two gold replies (`Je vais bien.` / `Je suis ATOM.`), CE fell to ~0 by step 150 and the outputs tracked the gold, then looped the last letter (`biennnennence`, `ATOMMMMM`). A 25-pair bank that re-injected the model's own argmax byte collapsed to `eee` by step 100, including a held-out prompt. That run was killed before save. The replacement injects the gold byte, lr 1e-4, no same-output penalty. At step 399, CE 0.09: `Qui es-tu ?` → `Je suis ATOMMOMOMMMOMAMM`, `Forge une page html.` → `Je forge une encete une`, held-out `Ouvre Siren.` → `Je liste le res`. That file is a probe, 6.6 Mo, not the tip. The Space ignores it.
+- `checkpoints/bi_d64_role/atom_native_step_10900000_bi_d64_role.pt` — role mask on the mixed corpus. Mask was drowned by French/English streams with no `Assistant:` tag. At 10.9M every prompt opened with `Je vais par`.
+- `checkpoints/bi_d64_rolefds/atom_native_step_10900000_bi_d64_rolefds.pt` — same mask on the Fractus shard only, resumed from the 10.7M tip, not from the `Je vais par` file. Role CE ~0.56 at 10.9M. A running probe, not a claim of speech.
 
-## Not in the tip
+## Earlier cuts
 
-`thefinalboss/fractus-datasets` has been converted to `Utilisateur:` / `Assistant:` text (2,265,000 pairs, ~7.1 Go) at `data/fractus_ds/train_fractus_pairs.txt` on the train pod. It is not in the 9.5M checkpoint. It gets injected on the next resume, same `.pt`, leak frozen. Not a raw dump into a new brain.
+Learnable leak drifted to ~0.36 and was frozen at 0.05. CE ~2.45 after the cut, ~1.82 at 7.2M. At 7.2M two French prompts returned the same string, `C'est pas se pas`. Zeroing the bigram readout did not remove it. Free-run of the model's own byte collapsed a 25-pair bank to `eee`. Teacher-forced gold replies can be memorized (`Je suis ATOM`) and then loop the last letter.
 
 ## What this is not
 
-Not a product. Not Siren. Not a tool-caller. Weights are operable. Fluency is an engineering gap: teacher-forced CE does not punish ignoring the prompt. A low CE is not a reply.
+Not a product. Not Siren. Not a tool-caller. Weights are operable. Fluency is the gap between teacher-forced next-byte and a reply that depends on the prompt.