Update model card and export table

README.md

@@ -356,9 +356,9 @@ A vLLM serving path for GPU servers is in preparation.
 
 ### Ready-made GGUF
 
-`parallax-8b-lambda-725B-tokens.gguf` at the repository root (2,133,006,592 bytes, 2.13 GB, sha256
-`eca13c0bf0bb4a5fbdf33b1616c7f33823ee84eb40fe7edb8852c566c7f6275f`, also in the `.sha256` file next to it) was
-converted from `exports/725B-tokens_step58814` (725.2B training tokens, step 58814).
+`parallax-8b-lambda-790B-tokens.gguf` at the repository root (2,133,006,592 bytes, 2.13 GB, sha256
+`489cd1d23f6c36ac394dc558fc266b26b14f5fb29b68ef9cbea0bb93b0354ef4`, also in the `.sha256` file next to it) was
+converted from `exports/790B-tokens_step64456` (790.2B training tokens, step 64456).
 It keeps the export's bf16 trunk and stores the routed experts' exact ternary codes and per-row scales;
 no weight is requantized. Each GGUF is a snapshot of one export and is not updated as new exports are added.
 To use a later export, convert it yourself (see `tools/parallax/README.md` on the branch).
@@ -367,13 +367,14 @@ The earlier snapshots remain at the repository root:
 
 | File | Export | sha256 |
 |---|---|---|
+| `parallax-8b-lambda-725B-tokens.gguf` | `exports/725B-tokens_step58814` | `eca13c0bf0bb4a5fbdf33b1616c7f33823ee84eb40fe7edb8852c566c7f6275f` |
 | `parallax-8b-lambda-584B-tokens.gguf` | `exports/584B-tokens_step47029` | `0026f760b29d4098da94d3ec88e6239a5216cc09df6db046d5474ab70d54f200` |
 | `parallax-8b-lambda-525B-tokens.gguf` | `exports/525B-tokens_step42242` | `7a20cb014a3b125e14979871e9adb6bc57a9b194f46d6638325ee9daf8cb5aec` |
 | `parallax-8b-lambda-431B-tokens.gguf` | `exports/431B-tokens_step35140` | `687296380f21e2c133b6d00668e09a70e1fc5488186b27a4e4ca20cdf4ddd694` |
 | `parallax-8b-lambda-381B-tokens.gguf` | `exports/381B-tokens_step30942` | `86312194ffc72b7b8d87f37389fe0fdfe7c510e2f6066f0c424cc0eaf592aedf` |
 
 ```bash
-hf download chutesai/parallax-8b-lambda parallax-8b-lambda-725B-tokens.gguf --local-dir .
+hf download chutesai/parallax-8b-lambda parallax-8b-lambda-790B-tokens.gguf --local-dir .
 hf download NousResearch/Meta-Llama-3-8B tokenizer.json --local-dir .   # Llama 3 tokenizer
 ```
 
@@ -390,12 +391,12 @@ pip install tokenizers
 
 # Apple GPU
 python tools/parallax/run.py --binary build/bin/llama-parallax \
-  --model ../parallax-8b-lambda-725B-tokens.gguf --tokenizer ../tokenizer.json \
+  --model ../parallax-8b-lambda-790B-tokens.gguf --tokenizer ../tokenizer.json \
   --backend metal --experts lut9 --prompt 'The capital of France is' --predict 64
 
 # CPU (x86-64 or ARM64)
 python tools/parallax/run.py --binary build/bin/llama-parallax \
-  --model ../parallax-8b-lambda-725B-tokens.gguf --tokenizer ../tokenizer.json \
+  --model ../parallax-8b-lambda-790B-tokens.gguf --tokenizer ../tokenizer.json \
   --backend cpu --experts lut8 --threads 8 --prompt 'The capital of France is' --predict 64
 ```
 
@@ -421,23 +422,30 @@ same results but is not tuned for speed: on a shared, loaded 64-core AMD EPYC 95
 
 ### Fidelity
 
-The 725B GGUF was checked on CPU with `lut` against the training code's own model loaded from the same
+The 790B GGUF was checked on CPU with `lut` against the training code's own model loaded from the same
 export, with FP32 activations in both, on the same 2,555 held-out positions from public text and code
-passages as before. Mean KL was 1.0e-5 nats/token, the top-1 next token agreed at all 2,555 positions,
+passages as before. Overall mean KL was 3.9e-7 nats/token; the top-1 next token agreed at all 2,555 positions,
 and three 48-token greedy continuations matched exactly.
 
-The remaining differences are confined to two passages. Each begins at a token where the router's 12th-
-and 13th-ranked expert-selection scores were within a few FP32 rounding steps (in one case an exact tie
-in the runtime), so the two implementations picked different experts; the outputs diverge from that point.
-Excluding the positions from those tokens onward in those passages (2,068 positions remain), mean KL was
-1.6e-9 nats/token with top-1 agreement at every position.
+The larger differences are in two passages (maximum logit error 0.63 and 0.24; at most 0.001 in the
+other three). In each passage they begin at a token where the router's 12th- and 13th-ranked
+expert-selection scores were near-tied, so the two implementations picked different experts. In passage A
+the flip occurs in layer 3 and the larger logit differences extend to later positions; in passage B it
+occurs in layer 63, the last layer, and the larger logit difference is confined to that position. Top-1
+predictions agree throughout both passages. In passage A, the score margins were 1.9e-6 in the training
+code and 2.4e-6 in the runtime near -4.67 (FP32 spacing 4.8e-7, about 4-5 rounding steps); in passage B,
+they were 3.1e-6 and 2.6e-6 near -3.35 (FP32 spacing 2.4e-7, about 11-13 rounding steps).
+Forcing the runtime's selection at just that one token/layer in the training code reduced mean KL to
+1.7e-9 and 1.9e-9 in those passages, with top-1 agreement at every position.
+Conservatively excluding every position from those tokens onward in those passages (2,183 positions
+remain), mean KL was 1.7e-9 nats/token with top-1 agreement at every position.
 
 The training code itself runs in bf16; against that bf16 forward the top-1 agreement was 94.9% (mean
 KL 0.0027, measured on the earlier `270B-tokens_step23035` export), which reflects the bf16 rounding
 of the reference, not an error of the runtime.
 
-On the 725B GGUF, Metal and CPU had per-layer relative error within 2.0e-5 at the final position of
-a 512-token text, and identical 48-token greedy continuations on three prompts. `lut8` (CPU)
+On the 790B GGUF, the largest per-layer relative RMS difference between Metal and CPU was about 2.2e-5
+at the final position of a 512-token text, and the two gave identical 48-token greedy continuations on three prompts. `lut8` (CPU)
 quantizes expert lookup tables to 8 bits and is slightly approximate; `lut` and `lut9` are exact.
 
 ### Limitations of the runtime
@@ -462,4 +470,4 @@ alignment or safety tuning, and it is small and far from converged.
 
 MIT.
 
-<sub>Table updated 2026-10-09 06:25 UTC.</sub>
+<sub>Table updated 2026-10-09 06:36 UTC.</sub>