Update model card and export table

README.md

@@ -11,16 +11,15 @@ tags:
 - research-checkpoint
 ---
 
-# Parallax 8B lambda: mid-training checkpoints
+# Parallax 8B lambda: training checkpoints
 
-These are research checkpoints from **lambda**, a training run of Parallax that is still in progress.
+These are research checkpoints from **lambda**, a training run of Parallax. Training was stopped at about 900B tokens, before the planned end of the run. The designated final checkpoint is step 73973 (900.8B tokens), marked in the table below. Exports saved later than the final checkpoint were saved after training stopped and are not the final checkpoint.
 Parallax is a mixture-of-experts language model trained on consumer GPUs spread over several
 countries. The hosts have no direct connections to each other and synchronize over the public internet.
 
 - **Base model.** It is pretrained on a mixture of web, document, math, code, knowledge-focused and book text. It is not instruction-tuned, chat-tuned or
   safety-tuned, and it will continue text rather than follow instructions.
-- **Mid-training.** Each export is a snapshot of a run that has not finished. Later exports are
-  expected to differ, and quality between exports is not monotonic.
+- **Training snapshots.** Every export, the final one included, is a snapshot taken before the planned ~973B tokens. Quality between exports is not monotonic.
 - **Research artifact.** It is published so the training can be followed and inspected. It is
   not intended for production use.
 
@@ -45,28 +44,30 @@ Live training dashboard: [parallax.chutes.ai](https://parallax.chutes.ai).
 
 ## Training
 
-- Data: seven stationary sources totalling about 1.09T tokens: two general pretraining mixtures of web, document, math and code text, an additional web set, two math sets, a knowledge-focused set and a books set. At about 899.6B tokens the data switches to two further sources with no learning-rate event. Planned training totals about 972.9B tokens.
+- Data: seven stationary sources totalling about 1.09T tokens: two general pretraining mixtures of web, document, math and code text, an additional web set, two math sets, a knowledge-focused set and a books set. At about 899.6B tokens the data switched to two further sources while the learning-rate anneal that began at 877.28B continued. Training stopped at about 900B tokens, shortly after this switch. The planned training total was about 972.9B tokens.
 - Optimization: the dense trunk is synchronized with a decoupled DiLoCo scheme over the internet. Routed
   experts are trained through low-rank adapters on the GPUs that use them, and the adapter updates are folded
   into full-precision masters that publish new ternary versions. The tech report describes the details.
 - Learning rate: linear warm-up from 0 to 6e-4 over the first 19.66B tokens, then 6e-4 until 400B tokens,
-  3e-4 until 700B and 1.5e-4 through the end of training. There is no final decay.
+  3e-4 until 700B and 1.5e-4 until 877.28B. A live anneal started at 877.28B tokens on 2026-10-09,
+  with multiplier m = 1 - 0.9 sqrt((T - 877.28B) / (972.90B - 877.28B)) and learning rate m times 1.5e-4.
+  When training stopped, the multiplier had reached 0.5489 (learning rate 8.234e-5). The anneal was partial.
 - Batch: 10 gradient-accumulation micro-batches per step at launch (about 8.5M tokens per fleet step with 26 machines), switched to 20 at about 100B tokens on 2026-10-04 at 22:08 UTC. This gives about 17M tokens per step with all 26 machines; tokens per step vary with fleet size.
-- Fleet: 26 machines and 208 GPUs at launch. Three machines left on 2026-10-05 and their roles were reassigned automatically while training continued; the fleet has since comprised 23 machines and 184 GPUs.
+- Fleet: 26 machines and 208 GPUs at launch. Seven machines were lost over 5-8 October, with roles reassigned automatically while training continued. At the end of the run, 19 machines were assigned, totalling 152 GPUs.
 - Document masking: In training, attention and recurrent state are reset at document boundaries (up to 64 documents per 4096-token sequence; further boundaries are merged into the 64th document).
 - Output logits: the learnable output logit scale is capped at 3.0 in training and inference (see *Files and format*).
 
 ## Exports
 
 One folder per export under `exports/`, named by training tokens (rounded down to whole billions) and
-fleet step. A new export is added about every 30 minutes while the run continues. Published exports are
+fleet step. Exports were added about every 30 minutes while training ran. Training has stopped; the final checkpoint is marked in the table. Published exports are
 never modified or removed.
 
 | Export | Tokens | Step | Size | Exported (UTC) |
 |---|---:|---:|---:|---|
-| [`901B-tokens_step73973_fc88abba`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/901B-tokens_step73973_fc88abba) **(latest)** | 901.5B | 73973 | 2.43 GB | 2026-10-10 01:44 |
+| [`901B-tokens_step73973_fc88abba`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/901B-tokens_step73973_fc88abba) | 901.5B | 73973 | 2.43 GB | 2026-10-10 01:44 |
 | [`901B-tokens_step73973`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/901B-tokens_step73973) | 901.5B | 73973 | 2.43 GB | 2026-10-10 01:17 |
-| [`900B-tokens_step73973`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/900B-tokens_step73973) | 900.8B | 73973 | 2.43 GB | 2026-10-10 00:45 |
+| [`900B-tokens_step73973`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/900B-tokens_step73973) **final** | 900.8B | 73973 | 2.43 GB | 2026-10-10 00:45 |
 | [`898B-tokens_step73878`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/898B-tokens_step73878) | 898.7B | 73878 | 2.43 GB | 2026-10-10 00:18 |
 | [`895B-tokens_step73637`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/895B-tokens_step73637) | 896.0B | 73637 | 2.43 GB | 2026-10-09 23:49 |
 | [`892B-tokens_step73364`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/892B-tokens_step73364) | 892.8B | 73364 | 2.43 GB | 2026-10-09 23:18 |
@@ -341,13 +342,13 @@ never modified or removed.
 
 - This table lists the exports only. Validation loss and benchmark results for this run are shown on the live training dashboard.
 
-## Latest export
+## Final checkpoint
 
-`exports/901B-tokens_step73973_fc88abba`: 901.5B training tokens, step 73973.
+`exports/900B-tokens_step73973`: 900.8B training tokens, step 73973.
 
 ```bash
-hf download chutesai/parallax-8b-lambda --include "exports/901B-tokens_step73973_fc88abba/*" --local-dir parallax-8b-lambda
-cd parallax-8b-lambda/exports/901B-tokens_step73973_fc88abba
+hf download chutesai/parallax-8b-lambda --include "exports/900B-tokens_step73973/*" --local-dir parallax-8b-lambda
+cd parallax-8b-lambda/exports/900B-tokens_step73973
 tar -xf packed_experts.tar
 sha256sum -c --quiet SHA256SUMS   # every file of the original export, byte for byte
 ```
@@ -512,4 +513,4 @@ alignment or safety tuning, and it is small and far from converged.
 
 MIT.
 
-<sub>Table updated 2026-10-10 01:52 UTC.</sub>
+<sub>Table updated 2026-10-10 02:05 UTC.</sub>