File size: 64,664 Bytes
ea0d634 29057a0 ea0d634 29057a0 ea0d634 29057a0 ea0d634 baaa83e ea0d634 baaa83e ea0d634 baaa83e ea0d634 29057a0 ea0d634 29057a0 baaa83e 29057a0 baaa83e ea0d634 29057a0 ea0d634 baaa83e 29057a0 99f061c c8efb24 ad83d37 29057a0 4c304d5 dcb7bc4 171109a 1f5f6fa 87fac18 c03b1ee 2121ee8 bec2acf 205b95a 421feb7 7c75a25 eacd756 06947af d6720a1 a7c1899 9070564 35872ec b4e117a e8db6c4 bf8155f 9651dfc 9dc5bd8 d71625f 7628331 0b93030 fe102ea b936c42 854dcc4 35893c6 5a36a61 1699ede 6a11372 24a611f e368900 184c5b2 5d9d4b5 0010065 959ce47 40a99ad c821c1a 97dcdcf a4b4927 10139b1 ae4cab2 02bdb79 9c2160d 1f13a71 4812eb0 e2ebcba ab10996 a55bd7e 0967dc6 f3bce5c c06d55b b4936ba c3a6f7a 799a6d5 7cd99ef f3705eb 370161c 01723bf c01be36 d488bf9 0c4b320 682bfac 9058426 2e01b3e 46ee74c 8f3a778 2775624 56445c1 a8eeea9 f246331 71d11fe df53ff6 440204d 8fa1e0b a3260bb ff89a1c 4b6fc4d 3ed64f7 9a10e7e 56a1fff 6af7eda 90647d0 cb30bab da22bca 9fa6507 a705dd8 60a2919 324d060 d33d506 41ebf54 632afde 73424c6 cad6b52 740d5c6 95d394e 45be222 cf49168 f16980b fb8345a 08d37b7 e405322 8bfb88b dd0b398 5629fb2 522f335 54083df 41f9b1c d6ebd64 be858d2 f072ee1 a4fbee6 e13d83b d3123a7 d84d4ec 59844b8 917d52d e868bf4 675fd3c fe5afd9 be2cc13 ef49979 c52a782 c062a71 ab7079d 5bd7e23 63616d3 ee593c9 dfc70e7 3950f0a 6637ed3 ea59a77 5dbf5fe 2b1c19c 9f6ef0c f5083d9 c676e5d 02d19e7 e200b24 e567518 4c109c0 b9cecb9 0c95ab7 c4968f4 d4ecee4 c8277ba f57450f 1f9ac48 033dd6a 80910e5 f8d79ff 8209d08 4f0f5d0 99b437e 6bc256e edb3ff4 ec9780b e9e6002 247ea98 dea7e39 7db220f e8b1230 fbc4481 b86d6d4 0929297 87d0773 78f8935 3366b3f 3785e41 cb75a9c 2e69c22 593d8b2 c342739 730e9e7 b70b0f8 d93b60e 416d629 2efebdd 7da42f5 3ac57c8 40271b4 39dd290 5d6ae47 516ddb5 0e0e279 418bb89 60e3d5a 88d68f0 06c1d3c 2206600 cc5ee20 773d94f 36f544b 4496869 79a1b73 53f4efe 6f77519 4cd2031 18cd028 7082467 e513715 47c8885 1b0f3d1 67986a6 1b99290 4ec380a b61274d 5a33a30 d383641 0cfaa91 baaa83e 99f061c ea0d634 29057a0 9a6e1fa 29057a0 9a6e1fa 29057a0 9a6e1fa ea0d634 c48c6f9 9756ded c48c6f9 9756ded 4a3a1ae 9756ded 7da8bb6 9756ded 337766b c89679a 40ccb01 558d72f 7da8bb6 c48c6f9 9756ded c48c6f9 ea0d634 c48c6f9 ea0d634 c48c6f9 9756ded c48c6f9 9756ded c48c6f9 ea0d634 c48c6f9 9756ded c48c6f9 9756ded 4a3a1ae 9756ded 4a3a1ae 9756ded c48c6f9 9756ded ea0d634 99f061c ea0d634 9756ded | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 | ---
license: mit
language:
- en
pipeline_tag: text-generation
tags:
- mixture-of-experts
- ternary
- decentralized-training
- base-model
- research-checkpoint
---
# Parallax 8B lambda: training checkpoints
These are research checkpoints from **lambda**, a training run of Parallax. Training was stopped at about 900B tokens, before the planned end of the run. The designated final checkpoint is step 73973 (900.8B tokens), marked in the table below. Exports saved later than the final checkpoint were saved after training stopped and are not the final checkpoint.
Parallax is a mixture-of-experts language model trained on consumer GPUs spread over several
countries. The hosts have no direct connections to each other and synchronize over the public internet.
- **Base model.** It is pretrained on a mixture of web, document, math, code, knowledge-focused and book text. It is not instruction-tuned, chat-tuned or
safety-tuned, and it will continue text rather than follow instructions.
- **Training snapshots.** Every export, the final one included, is a snapshot taken before the planned ~973B tokens. Quality between exports is not monotonic.
- **Research artifact.** It is published so the training can be followed and inspected. It is
not intended for production use.
Live training dashboard: [parallax.chutes.ai](https://parallax.chutes.ai).
## Model
| | |
|---|---|
| Parameters | ~7.77B total, ~1.20B active per token |
| Layers | 64: 32 mixers (18 EDA recurrent, 8 sliding-window attention, 6 global attention) interleaved with 32 mixture-of-experts layers |
| Recurrent mixers | EDA recurrence (a gated delta rule) with per-key-channel decay bounded below by e^-5 per token; erase rank 16; output normalization |
| Sliding-window attention | First and last mixers use 2048 tokens; the six interior layers share a per-step draw of 128 or 2048 tokens with equal probability throughout training; evaluation and inference use 2048 |
| Global attention | sparse-capable global attention (MSA) with 256-dimensional latent KV, 16 query heads, 4 KV heads and RoPE; dense causal attention throughout this run, with sparse top-k selection inactive |
| Experts | 4096 routed experts (128 per MoE layer), 12 routed + 1 shared expert per token |
| Expert weights | ternary (-1, 0, +1) with per-row scales; at most two nonzero pairs in every group of eight |
| Width | Model width 1152; expert intermediate width 2304, latent width 384; ReLU² expert activation |
| Tokenizer | Llama 3 tokenizer (vocabulary padded to 128,384); tied input/output embeddings |
| Training context | 4096 tokens |
| Document masking | In training, attention and recurrent state are reset at document boundaries (up to 64 documents per 4096-token sequence; further boundaries are merged into the 64th document) |
| Output logit scale | learnable, capped at 3.0 |
## Training
- Data: seven stationary sources totalling about 1.09T tokens: two general pretraining mixtures of web, document, math and code text, an additional web set, two math sets, a knowledge-focused set and a books set. At about 899.6B tokens the data switched to two further sources while the learning-rate anneal that began at 877.28B continued. Training stopped at about 900B tokens, shortly after this switch. The planned training total was about 972.9B tokens.
- Optimization: the dense trunk is synchronized with a decoupled DiLoCo scheme over the internet. Routed
experts are trained through low-rank adapters on the GPUs that use them, and the adapter updates are folded
into full-precision masters that publish new ternary versions. The tech report describes the details.
- Learning rate: linear warm-up from 0 to 6e-4 over the first 19.66B tokens, then 6e-4 until 400B tokens,
3e-4 until 700B and 1.5e-4 until 877.28B. A live anneal started at 877.28B tokens on 2026-10-09,
with multiplier m = 1 - 0.9 sqrt((T - 877.28B) / (972.90B - 877.28B)) and learning rate m times 1.5e-4.
When training stopped, the multiplier had reached 0.5489 (learning rate 8.234e-5). The anneal was partial.
- Batch: 10 gradient-accumulation micro-batches per step at launch (about 8.5M tokens per fleet step with 26 machines), switched to 20 at about 100B tokens on 2026-10-04 at 22:08 UTC. This gives about 17M tokens per step with all 26 machines; tokens per step vary with fleet size.
- Fleet: 26 machines and 208 GPUs at launch. Seven machines were lost over 5-8 October, with roles reassigned automatically while training continued. At the end of the run, 19 machines were assigned, totalling 152 GPUs.
- Document masking: In training, attention and recurrent state are reset at document boundaries (up to 64 documents per 4096-token sequence; further boundaries are merged into the 64th document).
- Output logits: the learnable output logit scale is capped at 3.0 in training and inference (see *Files and format*).
## Exports
One folder per export under `exports/`, named by training tokens (rounded down to whole billions) and
fleet step. Exports were added about every 30 minutes while training ran. Training has stopped; the final checkpoint is marked in the table. Published exports are
never modified or removed.
| Export | Tokens | Step | Size | Exported (UTC) |
|---|---:|---:|---:|---|
| [`901B-tokens_step73973_fc88abba`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/901B-tokens_step73973_fc88abba) | 901.5B | 73973 | 2.43 GB | 2026-10-10 01:44 |
| [`901B-tokens_step73973_6a749265`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/901B-tokens_step73973_6a749265) | 901.5B | 73973 | 2.43 GB | 2026-10-10 02:45 |
| [`901B-tokens_step73973_b08bb757`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/901B-tokens_step73973_b08bb757) | 901.5B | 73973 | 2.43 GB | 2026-10-10 02:18 |
| [`901B-tokens_step73973`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/901B-tokens_step73973) | 901.5B | 73973 | 2.43 GB | 2026-10-10 01:17 |
| [`900B-tokens_step73973`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/900B-tokens_step73973) **final** | 900.8B | 73973 | 2.43 GB | 2026-10-10 00:45 |
| [`898B-tokens_step73878`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/898B-tokens_step73878) | 898.7B | 73878 | 2.43 GB | 2026-10-10 00:18 |
| [`895B-tokens_step73637`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/895B-tokens_step73637) | 896.0B | 73637 | 2.43 GB | 2026-10-09 23:49 |
| [`892B-tokens_step73364`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/892B-tokens_step73364) | 892.8B | 73364 | 2.43 GB | 2026-10-09 23:18 |
| [`890B-tokens_step73127`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/890B-tokens_step73127) | 890.0B | 73127 | 2.43 GB | 2026-10-09 22:48 |
| [`886B-tokens_step72845`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/886B-tokens_step72845) | 886.9B | 72845 | 2.43 GB | 2026-10-09 22:19 |
| [`884B-tokens_step72604`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/884B-tokens_step72604) | 884.2B | 72604 | 2.43 GB | 2026-10-09 21:46 |
| [`881B-tokens_step72330`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/881B-tokens_step72330) | 881.0B | 72330 | 2.43 GB | 2026-10-09 21:17 |
| [`878B-tokens_step72097`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/878B-tokens_step72097) | 878.3B | 72097 | 2.43 GB | 2026-10-09 20:48 |
| [`875B-tokens_step71820`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/875B-tokens_step71820) | 875.2B | 71820 | 2.43 GB | 2026-10-09 20:20 |
| [`872B-tokens_step71588`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/872B-tokens_step71588) | 872.4B | 71588 | 2.43 GB | 2026-10-09 19:48 |
| [`869B-tokens_step71310`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/869B-tokens_step71310) | 869.3B | 71310 | 2.43 GB | 2026-10-09 19:19 |
| [`866B-tokens_step71076`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/866B-tokens_step71076) | 866.5B | 71076 | 2.43 GB | 2026-10-09 18:47 |
| [`863B-tokens_step70798`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/863B-tokens_step70798) | 863.4B | 70798 | 2.43 GB | 2026-10-09 18:16 |
| [`860B-tokens_step70559`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/860B-tokens_step70559) | 860.6B | 70559 | 2.43 GB | 2026-10-09 17:46 |
| [`857B-tokens_step70296`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/857B-tokens_step70296) | 857.5B | 70296 | 2.43 GB | 2026-10-09 17:17 |
| [`854B-tokens_step70065`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/854B-tokens_step70065) | 854.7B | 70065 | 2.43 GB | 2026-10-09 16:50 |
| [`851B-tokens_step69789`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/851B-tokens_step69789) | 851.7B | 69789 | 2.43 GB | 2026-10-09 16:16 |
| [`848B-tokens_step69548`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/848B-tokens_step69548) | 848.9B | 69548 | 2.43 GB | 2026-10-09 15:46 |
| [`845B-tokens_step69283`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/845B-tokens_step69283) | 845.7B | 69283 | 2.43 GB | 2026-10-09 15:15 |
| [`843B-tokens_step69045`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/843B-tokens_step69045) | 843.0B | 69045 | 2.43 GB | 2026-10-09 14:48 |
| [`839B-tokens_step68773`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/839B-tokens_step68773) | 839.9B | 68773 | 2.43 GB | 2026-10-09 14:15 |
| [`837B-tokens_step68544`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/837B-tokens_step68544) | 837.1B | 68544 | 2.43 GB | 2026-10-09 13:48 |
| [`834B-tokens_step68273`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/834B-tokens_step68273) | 834.0B | 68273 | 2.43 GB | 2026-10-09 13:15 |
| [`831B-tokens_step68038`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/831B-tokens_step68038) | 831.3B | 68038 | 2.43 GB | 2026-10-09 12:47 |
| [`828B-tokens_step67772`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/828B-tokens_step67772) | 828.2B | 67772 | 2.43 GB | 2026-10-09 12:16 |
| [`825B-tokens_step67534`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/825B-tokens_step67534) | 825.5B | 67534 | 2.43 GB | 2026-10-09 11:49 |
| [`822B-tokens_step67258`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/822B-tokens_step67258) | 822.3B | 67258 | 2.43 GB | 2026-10-09 11:16 |
| [`819B-tokens_step67027`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/819B-tokens_step67027) | 819.6B | 67027 | 2.43 GB | 2026-10-09 10:47 |
| [`816B-tokens_step66758`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/816B-tokens_step66758) | 816.4B | 66758 | 2.43 GB | 2026-10-09 10:15 |
| [`813B-tokens_step66509`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/813B-tokens_step66509) | 813.7B | 66509 | 2.43 GB | 2026-10-09 09:47 |
| [`810B-tokens_step66235`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/810B-tokens_step66235) | 810.5B | 66235 | 2.43 GB | 2026-10-09 09:16 |
| [`807B-tokens_step66001`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/807B-tokens_step66001) | 807.8B | 66001 | 2.43 GB | 2026-10-09 08:46 |
| [`804B-tokens_step65728`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/804B-tokens_step65728) | 804.7B | 65728 | 2.43 GB | 2026-10-09 08:14 |
| [`801B-tokens_step65490`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/801B-tokens_step65490) | 801.9B | 65490 | 2.43 GB | 2026-10-09 07:46 |
| [`798B-tokens_step65214`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/798B-tokens_step65214) | 798.8B | 65214 | 2.43 GB | 2026-10-09 07:16 |
| [`796B-tokens_step64968`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/796B-tokens_step64968) | 796.0B | 64968 | 2.43 GB | 2026-10-09 06:47 |
| [`792B-tokens_step64700`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/792B-tokens_step64700) | 792.9B | 64700 | 2.43 GB | 2026-10-09 06:14 |
| [`790B-tokens_step64456`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/790B-tokens_step64456) | 790.2B | 64456 | 2.43 GB | 2026-10-09 05:46 |
| [`787B-tokens_step64188`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/787B-tokens_step64188) | 787.0B | 64188 | 2.43 GB | 2026-10-09 05:13 |
| [`784B-tokens_step63948`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/784B-tokens_step63948) | 784.3B | 63948 | 2.43 GB | 2026-10-09 04:49 |
| [`781B-tokens_step63679`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/781B-tokens_step63679) | 781.1B | 63679 | 2.43 GB | 2026-10-09 04:15 |
| [`778B-tokens_step63434`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/778B-tokens_step63434) | 778.4B | 63434 | 2.43 GB | 2026-10-09 03:45 |
| [`775B-tokens_step63160`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/775B-tokens_step63160) | 775.3B | 63160 | 2.43 GB | 2026-10-09 03:15 |
| [`772B-tokens_step62921`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/772B-tokens_step62921) | 772.6B | 62921 | 2.43 GB | 2026-10-09 02:47 |
| [`769B-tokens_step62646`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/769B-tokens_step62646) | 769.4B | 62646 | 2.43 GB | 2026-10-09 02:16 |
| [`766B-tokens_step62413`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/766B-tokens_step62413) | 766.6B | 62413 | 2.43 GB | 2026-10-09 01:46 |
| [`763B-tokens_step62131`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/763B-tokens_step62131) | 763.6B | 62131 | 2.43 GB | 2026-10-09 01:16 |
| [`760B-tokens_step61889`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/760B-tokens_step61889) | 760.7B | 61889 | 2.43 GB | 2026-10-09 00:48 |
| [`757B-tokens_step61611`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/757B-tokens_step61611) | 757.6B | 61611 | 2.43 GB | 2026-10-09 00:18 |
| [`754B-tokens_step61370`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/754B-tokens_step61370) | 754.9B | 61370 | 2.43 GB | 2026-10-08 23:45 |
| [`751B-tokens_step61097`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/751B-tokens_step61097) | 751.8B | 61097 | 2.43 GB | 2026-10-08 23:15 |
| [`748B-tokens_step60864`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/748B-tokens_step60864) | 749.0B | 60864 | 2.43 GB | 2026-10-08 22:46 |
| [`745B-tokens_step60600`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/745B-tokens_step60600) | 745.9B | 60600 | 2.43 GB | 2026-10-08 22:20 |
| [`743B-tokens_step60341`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/743B-tokens_step60341) | 743.1B | 60341 | 2.43 GB | 2026-10-08 21:45 |
| [`740B-tokens_step60075`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/740B-tokens_step60075) | 740.0B | 60075 | 2.43 GB | 2026-10-08 21:16 |
| [`737B-tokens_step59833`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/737B-tokens_step59833) | 737.2B | 59833 | 2.43 GB | 2026-10-08 20:45 |
| [`734B-tokens_step59575`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/734B-tokens_step59575) | 734.2B | 59575 | 2.43 GB | 2026-10-08 20:16 |
| [`731B-tokens_step59330`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/731B-tokens_step59330) | 731.4B | 59330 | 2.43 GB | 2026-10-08 19:46 |
| [`728B-tokens_step59064`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/728B-tokens_step59064) | 728.1B | 59064 | 2.43 GB | 2026-10-08 19:16 |
| [`725B-tokens_step58814`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/725B-tokens_step58814) | 725.2B | 58814 | 2.43 GB | 2026-10-08 18:46 |
| [`721B-tokens_step58547`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/721B-tokens_step58547) | 722.0B | 58547 | 2.43 GB | 2026-10-08 18:18 |
| [`719B-tokens_step58295`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/719B-tokens_step58295) | 719.1B | 58295 | 2.43 GB | 2026-10-08 17:48 |
| [`715B-tokens_step58037`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/715B-tokens_step58037) | 715.9B | 58037 | 2.43 GB | 2026-10-08 17:16 |
| [`712B-tokens_step57790`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/712B-tokens_step57790) | 712.9B | 57790 | 2.43 GB | 2026-10-08 16:46 |
| [`709B-tokens_step57529`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/709B-tokens_step57529) | 709.7B | 57529 | 2.43 GB | 2026-10-08 16:17 |
| [`706B-tokens_step57284`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/706B-tokens_step57284) | 706.8B | 57284 | 2.43 GB | 2026-10-08 15:47 |
| [`703B-tokens_step57012`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/703B-tokens_step57012) | 703.7B | 57012 | 2.43 GB | 2026-10-08 15:16 |
| [`700B-tokens_step56772`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/700B-tokens_step56772) | 700.7B | 56772 | 2.43 GB | 2026-10-08 14:45 |
| [`697B-tokens_step56506`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/697B-tokens_step56506) | 697.5B | 56506 | 2.43 GB | 2026-10-08 14:16 |
| [`694B-tokens_step56272`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/694B-tokens_step56272) | 694.6B | 56272 | 2.43 GB | 2026-10-08 13:46 |
| [`691B-tokens_step55992`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/691B-tokens_step55992) | 691.4B | 55992 | 2.43 GB | 2026-10-08 13:19 |
| [`688B-tokens_step55745`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/688B-tokens_step55745) | 688.4B | 55745 | 2.43 GB | 2026-10-08 12:45 |
| [`685B-tokens_step55474`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/685B-tokens_step55474) | 685.2B | 55474 | 2.43 GB | 2026-10-08 12:15 |
| [`682B-tokens_step55234`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/682B-tokens_step55234) | 682.3B | 55234 | 2.43 GB | 2026-10-08 11:45 |
| [`679B-tokens_step54971`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/679B-tokens_step54971) | 679.1B | 54971 | 2.43 GB | 2026-10-08 11:14 |
| [`676B-tokens_step54731`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/676B-tokens_step54731) | 676.1B | 54731 | 2.43 GB | 2026-10-08 10:48 |
| [`672B-tokens_step54458`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/672B-tokens_step54458) | 672.9B | 54458 | 2.43 GB | 2026-10-08 10:16 |
| [`670B-tokens_step54209`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/670B-tokens_step54209) | 670.0B | 54209 | 2.43 GB | 2026-10-08 09:46 |
| [`666B-tokens_step53941`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/666B-tokens_step53941) | 666.8B | 53941 | 2.43 GB | 2026-10-08 09:14 |
| [`663B-tokens_step53702`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/663B-tokens_step53702) | 663.9B | 53702 | 2.43 GB | 2026-10-08 08:45 |
| [`660B-tokens_step53438`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/660B-tokens_step53438) | 660.7B | 53438 | 2.43 GB | 2026-10-08 08:17 |
| [`657B-tokens_step53196`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/657B-tokens_step53196) | 657.8B | 53196 | 2.43 GB | 2026-10-08 07:48 |
| [`654B-tokens_step52696`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/654B-tokens_step52696) | 654.5B | 52696 | 2.43 GB | 2026-10-08 07:17 |
| [`651B-tokens_step52456`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/651B-tokens_step52456) | 651.5B | 52456 | 2.43 GB | 2026-10-08 06:48 |
| [`648B-tokens_step52190`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/648B-tokens_step52190) | 648.2B | 52190 | 2.43 GB | 2026-10-08 06:16 |
| [`645B-tokens_step51945`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/645B-tokens_step51945) | 645.2B | 51945 | 2.43 GB | 2026-10-08 05:50 |
| [`641B-tokens_step51674`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/641B-tokens_step51674) | 641.9B | 51674 | 2.43 GB | 2026-10-08 05:15 |
| [`638B-tokens_step51438`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/638B-tokens_step51438) | 638.9B | 51438 | 2.43 GB | 2026-10-08 04:47 |
| [`635B-tokens_step51165`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/635B-tokens_step51165) | 635.5B | 51165 | 2.43 GB | 2026-10-08 04:16 |
| [`632B-tokens_step50914`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/632B-tokens_step50914) | 632.5B | 50914 | 2.43 GB | 2026-10-08 03:47 |
| [`629B-tokens_step50646`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/629B-tokens_step50646) | 629.1B | 50646 | 2.43 GB | 2026-10-08 03:17 |
| [`626B-tokens_step50402`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/626B-tokens_step50402) | 626.1B | 50402 | 2.43 GB | 2026-10-08 02:47 |
| [`622B-tokens_step50135`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/622B-tokens_step50135) | 622.7B | 50135 | 2.43 GB | 2026-10-08 02:18 |
| [`619B-tokens_step49880`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/619B-tokens_step49880) | 619.8B | 49880 | 2.43 GB | 2026-10-08 01:47 |
| [`616B-tokens_step49610`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/616B-tokens_step49610) | 616.4B | 49610 | 2.43 GB | 2026-10-08 01:22 |
| [`613B-tokens_step49361`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/613B-tokens_step49361) | 613.3B | 49361 | 2.43 GB | 2026-10-08 00:47 |
| [`610B-tokens_step49091`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/610B-tokens_step49091) | 610.0B | 49091 | 2.43 GB | 2026-10-08 00:17 |
| [`606B-tokens_step48845`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/606B-tokens_step48845) | 607.0B | 48845 | 2.43 GB | 2026-10-07 23:47 |
| [`603B-tokens_step48573`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/603B-tokens_step48573) | 603.6B | 48573 | 2.43 GB | 2026-10-07 23:17 |
| [`600B-tokens_step48324`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/600B-tokens_step48324) | 600.7B | 48324 | 2.43 GB | 2026-10-07 22:47 |
| [`597B-tokens_step48055`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/597B-tokens_step48055) | 597.3B | 48055 | 2.43 GB | 2026-10-07 22:14 |
| [`594B-tokens_step47817`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/594B-tokens_step47817) | 594.4B | 47817 | 2.43 GB | 2026-10-07 21:46 |
| [`590B-tokens_step47547`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/590B-tokens_step47547) | 591.0B | 47547 | 2.43 GB | 2026-10-07 21:18 |
| [`587B-tokens_step47301`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/587B-tokens_step47301) | 588.0B | 47301 | 2.43 GB | 2026-10-07 20:46 |
| [`584B-tokens_step47029`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/584B-tokens_step47029) | 584.6B | 47029 | 2.43 GB | 2026-10-07 20:16 |
| [`581B-tokens_step46782`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/581B-tokens_step46782) | 581.6B | 46782 | 2.43 GB | 2026-10-07 19:46 |
| [`578B-tokens_step46521`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/578B-tokens_step46521) | 578.3B | 46521 | 2.43 GB | 2026-10-07 19:16 |
| [`575B-tokens_step45978`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/575B-tokens_step45978) | 575.3B | 45978 | 2.43 GB | 2026-10-07 18:46 |
| [`571B-tokens_step45785`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/571B-tokens_step45785) | 571.9B | 45785 | 2.43 GB | 2026-10-07 18:16 |
| [`568B-tokens_step45541`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/568B-tokens_step45541) | 568.7B | 45541 | 2.43 GB | 2026-10-07 17:47 |
| [`565B-tokens_step45273`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/565B-tokens_step45273) | 565.2B | 45273 | 2.43 GB | 2026-10-07 17:14 |
| [`562B-tokens_step45031`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/562B-tokens_step45031) | 562.1B | 45031 | 2.43 GB | 2026-10-07 16:46 |
| [`558B-tokens_step44774`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/558B-tokens_step44774) | 558.5B | 44774 | 2.43 GB | 2026-10-07 16:18 |
| [`555B-tokens_step44528`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/555B-tokens_step44528) | 555.4B | 44528 | 2.43 GB | 2026-10-07 15:46 |
| [`551B-tokens_step44262`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/551B-tokens_step44262) | 551.9B | 44262 | 2.43 GB | 2026-10-07 15:17 |
| [`548B-tokens_step44023`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/548B-tokens_step44023) | 548.7B | 44023 | 2.43 GB | 2026-10-07 14:47 |
| [`545B-tokens_step43760`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/545B-tokens_step43760) | 545.2B | 43760 | 2.43 GB | 2026-10-07 14:17 |
| [`542B-tokens_step43518`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/542B-tokens_step43518) | 542.1B | 43518 | 2.43 GB | 2026-10-07 13:46 |
| [`538B-tokens_step43262`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/538B-tokens_step43262) | 538.6B | 43262 | 2.43 GB | 2026-10-07 13:17 |
| [`535B-tokens_step43016`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/535B-tokens_step43016) | 535.4B | 43016 | 2.43 GB | 2026-10-07 12:47 |
| [`531B-tokens_step42753`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/531B-tokens_step42753) | 531.9B | 42753 | 2.43 GB | 2026-10-07 12:20 |
| [`528B-tokens_step42504`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/528B-tokens_step42504) | 528.8B | 42504 | 2.43 GB | 2026-10-07 11:46 |
| [`525B-tokens_step42242`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/525B-tokens_step42242) | 525.2B | 42242 | 2.43 GB | 2026-10-07 11:21 |
| [`522B-tokens_step42001`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/522B-tokens_step42001) | 522.2B | 42001 | 2.43 GB | 2026-10-07 10:48 |
| [`518B-tokens_step41738`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/518B-tokens_step41738) | 518.6B | 41738 | 2.43 GB | 2026-10-07 10:18 |
| [`515B-tokens_step41487`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/515B-tokens_step41487) | 515.5B | 41487 | 2.43 GB | 2026-10-07 09:45 |
| [`511B-tokens_step41224`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/511B-tokens_step41224) | 511.9B | 41224 | 2.43 GB | 2026-10-07 09:15 |
| [`508B-tokens_step40992`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/508B-tokens_step40992) | 508.8B | 40992 | 2.43 GB | 2026-10-07 08:48 |
| [`505B-tokens_step40722`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/505B-tokens_step40722) | 505.3B | 40722 | 2.43 GB | 2026-10-07 08:20 |
| [`502B-tokens_step40480`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/502B-tokens_step40480) | 502.1B | 40480 | 2.43 GB | 2026-10-07 07:47 |
| [`498B-tokens_step40216`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/498B-tokens_step40216) | 498.6B | 40216 | 2.43 GB | 2026-10-07 07:17 |
| [`495B-tokens_step39965`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/495B-tokens_step39965) | 495.4B | 39965 | 2.43 GB | 2026-10-07 06:46 |
| [`491B-tokens_step39706`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/491B-tokens_step39706) | 492.0B | 39706 | 2.43 GB | 2026-10-07 06:17 |
| [`488B-tokens_step39456`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/488B-tokens_step39456) | 488.8B | 39456 | 2.43 GB | 2026-10-07 05:45 |
| [`485B-tokens_step39196`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/485B-tokens_step39196) | 485.2B | 39196 | 2.43 GB | 2026-10-07 05:19 |
| [`482B-tokens_step38951`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/482B-tokens_step38951) | 482.1B | 38951 | 2.43 GB | 2026-10-07 04:44 |
| [`478B-tokens_step38687`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/478B-tokens_step38687) | 478.6B | 38687 | 2.43 GB | 2026-10-07 04:18 |
| [`475B-tokens_step38450`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/475B-tokens_step38450) | 475.4B | 38450 | 2.43 GB | 2026-10-07 03:46 |
| [`471B-tokens_step38181`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/471B-tokens_step38181) | 471.9B | 38181 | 2.43 GB | 2026-10-07 03:16 |
| [`468B-tokens_step37937`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/468B-tokens_step37937) | 468.7B | 37937 | 2.43 GB | 2026-10-07 02:48 |
| [`465B-tokens_step37674`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/465B-tokens_step37674) | 465.2B | 37674 | 2.43 GB | 2026-10-07 02:20 |
| [`462B-tokens_step37429`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/462B-tokens_step37429) | 462.1B | 37429 | 2.43 GB | 2026-10-07 01:45 |
| [`458B-tokens_step37164`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/458B-tokens_step37164) | 458.5B | 37164 | 2.43 GB | 2026-10-07 01:17 |
| [`455B-tokens_step36914`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/455B-tokens_step36914) | 455.4B | 36914 | 2.43 GB | 2026-10-07 00:45 |
| [`451B-tokens_step36659`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/451B-tokens_step36659) | 451.9B | 36659 | 2.43 GB | 2026-10-07 00:19 |
| [`448B-tokens_step36416`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/448B-tokens_step36416) | 448.7B | 36416 | 2.43 GB | 2026-10-06 23:48 |
| [`445B-tokens_step36150`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/445B-tokens_step36150) | 445.2B | 36150 | 2.43 GB | 2026-10-06 23:17 |
| [`442B-tokens_step35905`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/442B-tokens_step35905) | 442.0B | 35905 | 2.43 GB | 2026-10-06 22:45 |
| [`438B-tokens_step35646`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/438B-tokens_step35646) | 438.5B | 35646 | 2.43 GB | 2026-10-06 22:17 |
| [`435B-tokens_step35402`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/435B-tokens_step35402) | 435.3B | 35402 | 2.43 GB | 2026-10-06 21:46 |
| [`431B-tokens_step35140`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/431B-tokens_step35140) | 431.9B | 35140 | 2.43 GB | 2026-10-06 21:18 |
| [`428B-tokens_step34890`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/428B-tokens_step34890) | 428.7B | 34890 | 2.43 GB | 2026-10-06 20:46 |
| [`425B-tokens_step34634`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/425B-tokens_step34634) | 425.2B | 34634 | 2.43 GB | 2026-10-06 20:21 |
| [`422B-tokens_step34390`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/422B-tokens_step34390) | 422.0B | 34390 | 2.43 GB | 2026-10-06 19:46 |
| [`418B-tokens_step34134`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/418B-tokens_step34134) | 418.5B | 34134 | 2.43 GB | 2026-10-06 19:17 |
| [`415B-tokens_step33886`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/415B-tokens_step33886) | 415.2B | 33886 | 2.43 GB | 2026-10-06 18:45 |
| [`411B-tokens_step33618`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/411B-tokens_step33618) | 411.8B | 33618 | 2.43 GB | 2026-10-06 18:20 |
| [`408B-tokens_step33381`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/408B-tokens_step33381) | 408.7B | 33381 | 2.43 GB | 2026-10-06 17:47 |
| [`405B-tokens_step33116`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/405B-tokens_step33116) | 405.1B | 33116 | 2.43 GB | 2026-10-06 17:17 |
| [`402B-tokens_step32861`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/402B-tokens_step32861) | 402.0B | 32861 | 2.43 GB | 2026-10-06 16:45 |
| [`398B-tokens_step32598`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/398B-tokens_step32598) | 398.6B | 32598 | 2.43 GB | 2026-10-06 16:16 |
| [`395B-tokens_step32353`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/395B-tokens_step32353) | 395.3B | 32353 | 2.43 GB | 2026-10-06 15:44 |
| [`391B-tokens_step32098`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/391B-tokens_step32098) | 391.8B | 32098 | 2.43 GB | 2026-10-06 15:18 |
| [`388B-tokens_step31854`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/388B-tokens_step31854) | 388.6B | 31854 | 2.43 GB | 2026-10-06 14:45 |
| [`385B-tokens_step31191`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/385B-tokens_step31191) | 385.1B | 31191 | 2.43 GB | 2026-10-06 14:18 |
| [`381B-tokens_step30942`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/381B-tokens_step30942) | 381.8B | 30942 | 2.43 GB | 2026-10-06 13:45 |
| [`378B-tokens_step30696`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/378B-tokens_step30696) | 378.2B | 30696 | 2.43 GB | 2026-10-06 13:18 |
| [`374B-tokens_step30455`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/374B-tokens_step30455) | 374.8B | 30455 | 2.43 GB | 2026-10-06 12:44 |
| [`371B-tokens_step30200`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/371B-tokens_step30200) | 371.2B | 30200 | 2.43 GB | 2026-10-06 12:14 |
| [`367B-tokens_step29959`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/367B-tokens_step29959) | 367.8B | 29959 | 2.43 GB | 2026-10-06 11:45 |
| [`364B-tokens_step29699`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/364B-tokens_step29699) | 364.3B | 29699 | 2.43 GB | 2026-10-06 11:17 |
| [`360B-tokens_step29461`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/360B-tokens_step29461) | 360.9B | 29461 | 2.43 GB | 2026-10-06 10:47 |
| [`357B-tokens_step29209`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/357B-tokens_step29209) | 357.3B | 29209 | 2.43 GB | 2026-10-06 10:17 |
| [`353B-tokens_step28971`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/353B-tokens_step28971) | 354.0B | 28971 | 2.43 GB | 2026-10-06 09:46 |
| [`350B-tokens_step28721`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/350B-tokens_step28721) | 350.4B | 28721 | 2.43 GB | 2026-10-06 09:19 |
| [`347B-tokens_step28477`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/347B-tokens_step28477) | 347.0B | 28477 | 2.43 GB | 2026-10-06 08:46 |
| [`343B-tokens_step28219`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/343B-tokens_step28219) | 343.4B | 28219 | 2.43 GB | 2026-10-06 08:17 |
| [`340B-tokens_step27983`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/340B-tokens_step27983) | 340.1B | 27983 | 2.43 GB | 2026-10-06 07:47 |
| [`336B-tokens_step27720`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/336B-tokens_step27720) | 336.4B | 27720 | 2.43 GB | 2026-10-06 07:18 |
| [`333B-tokens_step27484`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/333B-tokens_step27484) | 333.0B | 27484 | 2.43 GB | 2026-10-06 06:46 |
| [`329B-tokens_step27229`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/329B-tokens_step27229) | 329.5B | 27229 | 2.43 GB | 2026-10-06 06:20 |
| [`326B-tokens_step26986`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/326B-tokens_step26986) | 326.1B | 26986 | 2.43 GB | 2026-10-06 05:43 |
| [`322B-tokens_step26728`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/322B-tokens_step26728) | 322.5B | 26728 | 2.43 GB | 2026-10-06 05:17 |
| [`319B-tokens_step26493`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/319B-tokens_step26493) | 319.2B | 26493 | 2.43 GB | 2026-10-06 04:44 |
| [`315B-tokens_step26240`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/315B-tokens_step26240) | 315.6B | 26240 | 2.43 GB | 2026-10-06 04:17 |
| [`312B-tokens_step25999`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/312B-tokens_step25999) | 312.2B | 25999 | 2.43 GB | 2026-10-06 03:44 |
| [`308B-tokens_step25748`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/308B-tokens_step25748) | 308.6B | 25748 | 2.43 GB | 2026-10-06 03:18 |
| [`305B-tokens_step25502`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/305B-tokens_step25502) | 305.2B | 25502 | 2.43 GB | 2026-10-06 02:45 |
| [`301B-tokens_step25253`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/301B-tokens_step25253) | 301.7B | 25253 | 2.43 GB | 2026-10-06 02:19 |
| [`298B-tokens_step25009`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/298B-tokens_step25009) | 298.3B | 25009 | 2.43 GB | 2026-10-06 01:43 |
| [`294B-tokens_step24762`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/294B-tokens_step24762) | 294.7B | 24762 | 2.43 GB | 2026-10-06 01:18 |
| [`291B-tokens_step24519`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/291B-tokens_step24519) | 291.3B | 24519 | 2.43 GB | 2026-10-06 00:46 |
| [`287B-tokens_step24258`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/287B-tokens_step24258) | 287.7B | 24258 | 2.43 GB | 2026-10-06 00:15 |
| [`284B-tokens_step24014`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/284B-tokens_step24014) | 284.4B | 24014 | 2.43 GB | 2026-10-05 23:44 |
| [`280B-tokens_step23768`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/280B-tokens_step23768) | 280.7B | 23768 | 2.43 GB | 2026-10-05 23:15 |
| [`277B-tokens_step23530`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/277B-tokens_step23530) | 277.4B | 23530 | 2.43 GB | 2026-10-05 22:46 |
| [`273B-tokens_step23276`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/273B-tokens_step23276) | 273.8B | 23276 | 2.44 GB | 2026-10-05 22:16 |
| [`270B-tokens_step23035`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/270B-tokens_step23035) | 270.4B | 23035 | 2.44 GB | 2026-10-05 21:44 |
| [`266B-tokens_step22784`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/266B-tokens_step22784) | 266.9B | 22784 | 2.44 GB | 2026-10-05 21:19 |
| [`263B-tokens_step22539`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/263B-tokens_step22539) | 263.5B | 22539 | 2.44 GB | 2026-10-05 20:47 |
| [`259B-tokens_step22282`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/259B-tokens_step22282) | 259.9B | 22282 | 2.44 GB | 2026-10-05 20:17 |
| [`256B-tokens_step22052`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/256B-tokens_step22052) | 256.6B | 22052 | 2.44 GB | 2026-10-05 19:47 |
| [`252B-tokens_step21791`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/252B-tokens_step21791) | 252.9B | 21791 | 2.44 GB | 2026-10-05 19:17 |
| [`249B-tokens_step21550`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/249B-tokens_step21550) | 249.6B | 21550 | 2.44 GB | 2026-10-05 18:45 |
| [`245B-tokens_step21250`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/245B-tokens_step21250) | 245.3B | 21250 | 2.45 GB | 2026-10-05 18:09 |
| [`242B-tokens_step21067`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/242B-tokens_step21067) | 242.4B | 21067 | 2.45 GB | 2026-10-05 17:45 |
| [`238B-tokens_step20776`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/238B-tokens_step20776) | 238.0B | 20776 | 2.45 GB | 2026-10-05 17:14 |
| [`235B-tokens_step20575`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/235B-tokens_step20575) | 235.2B | 20575 | 2.46 GB | 2026-10-05 16:46 |
| [`230B-tokens_step20280`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/230B-tokens_step20280) | 230.7B | 20280 | 2.46 GB | 2026-10-05 16:12 |
| [`227B-tokens_step20081`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/227B-tokens_step20081) | 227.9B | 20081 | 2.46 GB | 2026-10-05 15:45 |
| [`223B-tokens_step19786`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/223B-tokens_step19786) | 223.4B | 19786 | 2.45 GB | 2026-10-05 15:11 |
| [`220B-tokens_step19602`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/220B-tokens_step19602) | 220.6B | 19602 | 2.45 GB | 2026-10-05 14:47 |
| [`216B-tokens_step19301`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/216B-tokens_step19301) | 216.2B | 19301 | 2.45 GB | 2026-10-05 14:11 |
| [`213B-tokens_step19100`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/213B-tokens_step19100) | 213.3B | 19100 | 2.45 GB | 2026-10-05 13:45 |
| [`208B-tokens_step18807`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/208B-tokens_step18807) | 208.9B | 18807 | 2.45 GB | 2026-10-05 13:15 |
| [`206B-tokens_step18619`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/206B-tokens_step18619) | 206.0B | 18619 | 2.45 GB | 2026-10-05 12:45 |
| [`201B-tokens_step18313`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/201B-tokens_step18313) | 201.6B | 18313 | 2.45 GB | 2026-10-05 12:12 |
| [`198B-tokens_step18118`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/198B-tokens_step18118) | 198.7B | 18118 | 2.45 GB | 2026-10-05 11:44 |
| [`194B-tokens_step17828`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/194B-tokens_step17828) | 194.3B | 17828 | 2.45 GB | 2026-10-05 11:11 |
| [`191B-tokens_step17631`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/191B-tokens_step17631) | 191.5B | 17631 | 2.45 GB | 2026-10-05 10:44 |
| [`187B-tokens_step17334`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/187B-tokens_step17334) | 187.0B | 17334 | 2.45 GB | 2026-10-05 10:12 |
| [`184B-tokens_step17145`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/184B-tokens_step17145) | 184.2B | 17145 | 2.45 GB | 2026-10-05 09:45 |
| [`179B-tokens_step16851`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/179B-tokens_step16851) | 179.8B | 16851 | 2.45 GB | 2026-10-05 09:11 |
| [`176B-tokens_step16645`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/176B-tokens_step16645) | 176.9B | 16645 | 2.45 GB | 2026-10-05 08:45 |
| [`172B-tokens_step16359`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/172B-tokens_step16359) | 172.6B | 16359 | 2.45 GB | 2026-10-05 08:12 |
| [`169B-tokens_step16164`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/169B-tokens_step16164) | 169.7B | 16164 | 2.45 GB | 2026-10-05 07:45 |
| [`165B-tokens_step15861`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/165B-tokens_step15861) | 165.2B | 15861 | 2.45 GB | 2026-10-05 07:10 |
| [`162B-tokens_step15667`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/162B-tokens_step15667) | 162.4B | 15667 | 2.46 GB | 2026-10-05 06:45 |
| [`157B-tokens_step15384`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/157B-tokens_step15384) | 157.9B | 15384 | 2.46 GB | 2026-10-05 06:13 |
| [`155B-tokens_step15179`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/155B-tokens_step15179) | 155.1B | 15179 | 2.46 GB | 2026-10-05 05:46 |
| [`150B-tokens_step14880`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/150B-tokens_step14880) | 150.6B | 14880 | 2.46 GB | 2026-10-05 05:12 |
| [`147B-tokens_step14693`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/147B-tokens_step14693) | 147.8B | 14693 | 2.46 GB | 2026-10-05 04:45 |
| [`143B-tokens_step14118`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/143B-tokens_step14118) | 143.4B | 14118 | 2.47 GB | 2026-10-05 04:13 |
| [`140B-tokens_step13940`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/140B-tokens_step13940) | 140.5B | 13940 | 2.47 GB | 2026-10-05 03:44 |
| [`135B-tokens_step13714`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/135B-tokens_step13714) | 135.8B | 13714 | 2.47 GB | 2026-10-05 03:14 |
| [`132B-tokens_step13524`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/132B-tokens_step13524) | 132.5B | 13524 | 2.47 GB | 2026-10-05 02:48 |
| [`127B-tokens_step13242`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/127B-tokens_step13242) | 127.7B | 13242 | 2.48 GB | 2026-10-05 02:12 |
| [`123B-tokens_step13016`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/123B-tokens_step13016) | 123.6B | 13016 | 2.48 GB | 2026-10-05 01:43 |
| [`119B-tokens_step12790`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/119B-tokens_step12790) | 119.5B | 12790 | 2.48 GB | 2026-10-05 01:13 |
| [`115B-tokens_step12532`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/115B-tokens_step12532) | 115.4B | 12532 | 2.48 GB | 2026-10-05 00:38 |
| [`111B-tokens_step12302`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/111B-tokens_step12302) | 111.4B | 12302 | 2.48 GB | 2026-10-05 00:10 |
| [`107B-tokens_step12081`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/107B-tokens_step12081) | 107.2B | 12081 | 2.48 GB | 2026-10-04 23:41 |
| [`103B-tokens_step11842`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/103B-tokens_step11842) | 103.2B | 11842 | 2.48 GB | 2026-10-04 23:14 |
| [`99B-tokens_step11533`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/99B-tokens_step11533) | 99.2B | 11533 | 2.48 GB | 2026-10-04 22:39 |
| [`95B-tokens_step11160`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/95B-tokens_step11160) | 96.0B | 11160 | 2.47 GB | 2026-10-04 22:12 |
| [`92B-tokens_step10766`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/92B-tokens_step10766) | 92.6B | 10766 | 2.47 GB | 2026-10-04 21:43 |
| [`89B-tokens_step10388`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/89B-tokens_step10388) | 89.3B | 10388 | 2.47 GB | 2026-10-04 21:12 |
| [`85B-tokens_step9986`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/85B-tokens_step9986) | 86.0B | 9986 | 2.47 GB | 2026-10-04 20:41 |
| [`82B-tokens_step9591`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/82B-tokens_step9591) | 82.7B | 9591 | 2.47 GB | 2026-10-04 20:09 |
| [`79B-tokens_step9195`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/79B-tokens_step9195) | 79.3B | 9195 | 2.47 GB | 2026-10-04 19:39 |
| [`76B-tokens_step8835`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/76B-tokens_step8835) | 76.1B | 8835 | 2.48 GB | 2026-10-04 19:13 |
| [`72B-tokens_step8443`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/72B-tokens_step8443) | 72.8B | 8443 | 2.48 GB | 2026-10-04 18:40 |
| [`69B-tokens_step8039`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/69B-tokens_step8039) | 69.5B | 8039 | 2.48 GB | 2026-10-04 18:11 |
| [`66B-tokens_step7652`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/66B-tokens_step7652) | 66.1B | 7652 | 2.48 GB | 2026-10-04 17:42 |
| [`62B-tokens_step7275`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/62B-tokens_step7275) | 62.9B | 7275 | 2.48 GB | 2026-10-04 17:10 |
| [`59B-tokens_step6711`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/59B-tokens_step6711) | 59.6B | 6711 | 2.48 GB | 2026-10-04 16:40 |
| [`56B-tokens_step6444`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/56B-tokens_step6444) | 56.3B | 6444 | 2.48 GB | 2026-10-04 16:09 |
| [`53B-tokens_step6096`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/53B-tokens_step6096) | 53.0B | 6096 | 2.48 GB | 2026-10-04 15:40 |
| [`49B-tokens_step5723`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/49B-tokens_step5723) | 49.8B | 5723 | 2.48 GB | 2026-10-04 15:12 |
| [`46B-tokens_step5320`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/46B-tokens_step5320) | 46.4B | 5320 | 2.48 GB | 2026-10-04 14:40 |
| [`43B-tokens_step4946`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/43B-tokens_step4946) | 43.1B | 4946 | 2.48 GB | 2026-10-04 14:10 |
| [`39B-tokens_step4549`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/39B-tokens_step4549) | 39.8B | 4549 | 2.48 GB | 2026-10-04 13:38 |
| [`36B-tokens_step4166`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/36B-tokens_step4166) | 36.5B | 4166 | 2.48 GB | 2026-10-04 13:15 |
| [`33B-tokens_step3768`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/33B-tokens_step3768) | 33.2B | 3768 | 2.48 GB | 2026-10-04 12:40 |
| [`29B-tokens_step3391`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/29B-tokens_step3391) | 29.8B | 3391 | 2.49 GB | 2026-10-04 12:16 |
| [`26B-tokens_step3002`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/26B-tokens_step3002) | 26.5B | 3002 | 2.49 GB | 2026-10-04 11:43 |
| [`23B-tokens_step2604`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/23B-tokens_step2604) | 23.1B | 2604 | 2.50 GB | 2026-10-04 11:15 |
| [`19B-tokens_step2198`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/19B-tokens_step2198) | 19.9B | 2198 | 2.50 GB | 2026-10-04 10:40 |
| [`16B-tokens_step1817`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/16B-tokens_step1817) | 16.7B | 1817 | 2.50 GB | 2026-10-04 10:11 |
| [`13B-tokens_step1419`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/13B-tokens_step1419) | 13.3B | 1419 | 2.50 GB | 2026-10-04 09:42 |
| [`6B-tokens_step635`](https://huggingface.co/chutesai/parallax-8b-lambda/tree/main/exports/6B-tokens_step635) | 6.6B | 635 | 2.50 GB | 2026-10-04 08:41 |
- This table lists the exports only. Final benchmarks of the final checkpoint, in percent, from the same lm-eval harness used for every model in the comparison: 35.4 on MMLU (cloze, 5-shot), 46.5 on ARC-Challenge (5-shot), 59.8 on HellaSwag (0-shot), and, on the full generative test splits, 14.3 on GSM8K (5-shot), 15.4 on MBPP (3-shot) and 12.2 on HumanEval (pass@1). No statistically significant aggregate difference was detected among the three late exports (paired 95% confidence intervals). The tech report has the full comparison, including other public base models.
## Final checkpoint
`exports/900B-tokens_step73973`: 900.8B training tokens, step 73973.
```bash
hf download chutesai/parallax-8b-lambda --include "exports/900B-tokens_step73973/*" --local-dir parallax-8b-lambda
cd parallax-8b-lambda/exports/900B-tokens_step73973
tar -xf packed_experts.tar
sha256sum -c --quiet SHA256SUMS # every file of the original export, byte for byte
```
## Files and format
The files are in Parallax's **native compact export format** (inference only, no optimizer state). They
are byte-identical to the export the training system produced:
| File | Contents |
|---|---|
| `manifest.json` | export manifest: tensor inventory, per-file sha256 digests, token clock |
| `model_config.json` | model configuration |
| `coverage.json`, `layouts.json` | tensor coverage and expert frame layouts |
| `indexer.bundle` | sparse-attention indexer weights |
| `relay_pack/` | trunk (non-expert) weights in bf16, with their own manifest |
| `packed_experts.tar` | the 4096 routed experts (`packed_experts/*.t24p`, packed ternary codes and scales) |
| `SHA256SUMS` | sha256 of every file of the original export |
| `export_info.json` | step, tokens, time, sizes and digests of this export |
The only change from the original export is packaging. The 4096 expert files are stored in one
uncompressed tar to keep the repository's file count manageable. Extract it and check `SHA256SUMS`
as shown above. Every upload was checked against the training system's own digests before and
after it was published.
**Logit-scale bound.** The model's output logit scale is bounded: the forward pass uses
`exp(min(logit_scale_log, log 3.0))`. The stored trunk tensor is the raw training parameter (it can sit
slightly above the bound, e.g. from bf16 rounding), and each export records the bound in `manifest.json`
(`logit_scale`: `max`, `raw`, `effective`) and in `coverage.json` (`inference_policy.logit_scale_max`).
A loader must apply the recorded bound; the files themselves are left byte-identical.
## Running it
This is a **base model only**. It is not chat- or instruction-tuned, so it does plain text
completion: give it the start of a text and it continues it. It will not follow instructions or hold a
conversation.
Standard `transformers` cannot load this format. Use the `parallax-lambda` branch of our llama.cpp fork,
[https://github.com/chutesai/llama.cpp](https://github.com/chutesai/llama.cpp/tree/parallax-lambda). It adds a
dedicated runtime, `llama-parallax`, for CPU (x86-64, ARM64) and Apple GPUs (Metal). Lambda's recurrent
mixers (EDA) are implemented only on this branch. The fork's `master` branch runs the earlier kappa
checkpoints and cannot run lambda.
A vLLM serving path for GPU servers is in preparation.
### The finisher continuation
After lambda stopped, its final checkpoint (`exports/900B-tokens_step73973`, 900.8B tokens) was continued on a
single 8-GPU server for a further 66.8B tokens, without the fleet. The mid-training mixture replays part of
lambda's data and adds higher-quality web and reference text, books and other creative writing, scientific
papers, code, math, instruction-style and question-answering text, synthetic long-document retrieval tasks and
reasoning traces. The training context was raised in stages from 4,096 to 8,192, 16,384, 32,768 and 65,536
tokens (23.8B tokens over the first three lengths with dense global attention; then 6B at 16,384 tokens, during
which, after about 1B tokens, the sparse global-attention selection was switched on; then 10B at 32,768 and 12B
at 65,536 tokens with it on), followed by a closing 15B-token learning-rate anneal over a mix of 4,096- to
65,536-token sequences. The longest trained
context is 65,536 tokens; a planned 131,072-token stage was skipped. The architecture, tokenizer and expert
format are unchanged; the RoPE base of the global-attention layers was raised from 1e6 to 1.67e7 for the longer
contexts. The finisher's final export is `exports/finisher-967B-tokens_step31888` (967.6B tokens in total,
finisher step 31888); like the lambda exports it is a base model with no instruction, chat or safety tuning.
The *Model* and *Training* sections above describe the original lambda run.
Benchmarks of the finisher's final export, in percent, from the same lm-eval harness as the lambda numbers
above; the lambda column repeats the final checkpoint's earlier scores on that harness for comparison:
| Benchmark | Finisher final | Lambda final (900.8B) |
|---|---:|---:|
| MMLU (continuation, 5-shot) | 34.6 | 35.4 |
| ARC-Challenge (25-shot) | 45.8 | 47.4 |
| HellaSwag (0-shot) | 59.0 | 59.8 |
| GSM8K (5-shot) | 14.7 | 14.3 |
| HumanEval (pass@1) | 12.2 | 12.2 |
| MBPP (3-shot) | 16.0 | 15.4 |
Long context, finisher final only: needle-in-a-haystack retrieval 100, 100 and 98 at 16k, 32k and 64k tokens,
and 63 at 128k (past the trained context), in percent over 100 items each; the RULER seven-task mean was 79.3, 72.4, 60.5,
49.7, 33.5 and 26.3 at 2k, 4k, 8k, 16k, 32k and 64k. The needle test uses the finisher's own synthetic
long-context format, so it is in-distribution and measures retrieval the model was trained for; RULER is the
external benchmark. These long-context numbers were measured with the training code with sparse global attention,
not with the GGUF runtime below.
### Ready-made GGUF
`parallax-8b-lambda-finisher-967B-tokens.gguf` at the repository root (2,133,006,592 bytes, 2.13 GB, sha256
`4a3ed3cf2d4ca959a7a93f0dac9e4cd52b45e39355e4aa63244080b56916cd74`, also in the `.sha256` file next to it) was
converted from `exports/finisher-967B-tokens_step31888`, the finisher's final export (967.6B training tokens in
total, finisher step 31888). It keeps the export's bf16 trunk and stores the routed experts' exact ternary codes and
per-row scales; no weight is requantized. Its global-attention layers run with sparse block selection and the
finisher's RoPE base, as trained; the lambda GGUFs below run dense global attention. Each GGUF is a snapshot of one
export. To use another export, convert it yourself (see `tools/parallax/README.md` on the branch).
The earlier snapshots, converted from lambda exports, remain at the repository root:
| File | Export | sha256 |
|---|---|---|
| `parallax-8b-lambda-878B-tokens.gguf` | `exports/878B-tokens_step72097` | `ad38864d341fe0ab70fab02d91c3320e6d36424171e20076ebba022737091a98` |
| `parallax-8b-lambda-860B-tokens.gguf` | `exports/860B-tokens_step70559` | `a65ecadcb38cb8ed34bd73ecc8fedf564393609ac2bb87f1f9f48b6149a70ba7` |
| `parallax-8b-lambda-790B-tokens.gguf` | `exports/790B-tokens_step64456` | `489cd1d23f6c36ac394dc558fc266b26b14f5fb29b68ef9cbea0bb93b0354ef4` |
| `parallax-8b-lambda-725B-tokens.gguf` | `exports/725B-tokens_step58814` | `eca13c0bf0bb4a5fbdf33b1616c7f33823ee84eb40fe7edb8852c566c7f6275f` |
| `parallax-8b-lambda-584B-tokens.gguf` | `exports/584B-tokens_step47029` | `0026f760b29d4098da94d3ec88e6239a5216cc09df6db046d5474ab70d54f200` |
| `parallax-8b-lambda-525B-tokens.gguf` | `exports/525B-tokens_step42242` | `7a20cb014a3b125e14979871e9adb6bc57a9b194f46d6638325ee9daf8cb5aec` |
| `parallax-8b-lambda-431B-tokens.gguf` | `exports/431B-tokens_step35140` | `687296380f21e2c133b6d00668e09a70e1fc5488186b27a4e4ca20cdf4ddd694` |
| `parallax-8b-lambda-381B-tokens.gguf` | `exports/381B-tokens_step30942` | `86312194ffc72b7b8d87f37389fe0fdfe7c510e2f6066f0c424cc0eaf592aedf` |
```bash
hf download chutesai/parallax-8b-lambda parallax-8b-lambda-finisher-967B-tokens.gguf --local-dir .
hf download NousResearch/Meta-Llama-3-8B tokenizer.json --local-dir . # Llama 3 tokenizer
```
### Build and run
Requires CMake, a C++17 compiler, and Python 3 with `tokenizers` for the text front end.
```bash
git clone -b parallax-lambda https://github.com/chutesai/llama.cpp && cd llama.cpp
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON # -DGGML_METAL=OFF for CPU only
cmake --build build --target llama-parallax -j 8
build/bin/llama-parallax --backend metal --self-test # or --backend cpu
pip install tokenizers
# Apple GPU
python tools/parallax/run.py --binary build/bin/llama-parallax \
--model ../parallax-8b-lambda-finisher-967B-tokens.gguf --tokenizer ../tokenizer.json \
--backend metal --experts lut9 --prompt 'The capital of France is' --predict 64
# CPU (x86-64 or ARM64)
python tools/parallax/run.py --binary build/bin/llama-parallax \
--model ../parallax-8b-lambda-finisher-967B-tokens.gguf --tokenizer ../tokenizer.json \
--backend cpu --experts lut8 --threads 8 --prompt 'The capital of France is' --predict 64
```
Generation is greedy by default. `build/bin/llama-parallax --help` lists sampling, context size (`-c`, default 4096),
prompt scoring and benchmark options.
### Speed and memory
Measured on an Apple M5 Max on AC power (batch 1, 128 generated tokens) with the bf16-trunk GGUF of the earlier
`270B-tokens_step23035` export, which has the same architecture, using the same runtime code; speed does not
depend on the weight values. The finisher GGUF additionally runs the sparse-attention indexer and block selection in its global-attention layers; it was not re-timed. Prompt times used `--prefill-batch 128`:
| Backend | Experts | Tokens/s, short context | Tokens/s after 4,096 tokens | 2,048-token prompt |
|---|---|---:|---:|---:|
| Metal | `lut9` | 192 | 172 | 2.2 s |
| CPU, 12 threads | `lut8` | 157 | 112 | 4.5 s |
| CPU, 12 threads | `lut` (exact) | 90 | 74 | |
Peak resident memory was about 2.8 GB on Metal and 3.4 GB on CPU at the default 4,096-token context. Other
Apple and ARM hardware will differ. ARM64 CPUs use NEON kernels. x86-64 CPUs use portable C++ that gives the
same results but is not tuned for speed: on a shared, loaded 64-core AMD EPYC 9534 server it generated about
22 tokens/s with 16 threads.
### Fidelity
The finisher GGUF was checked on CPU with `lut` against the training code's own model loaded from the same
export, with FP32 activations in both, on the same 2,555 held-out positions from public text and code
passages as the earlier releases, with sparse global attention in both. Overall mean KL was 1.7e-9 nats/token
(per passage 1.5e-9 to 1.8e-9); the top-1 next token agreed at all 2,555 positions; the largest logit
difference at any position was 0.0010; and three 48-token greedy continuations matched exactly. No passage
showed the divergence seen with the lambda GGUFs, where a router near-tie made the two implementations pick
different experts from one token onward.
The training code itself runs in bf16; against that bf16 forward the top-1 agreement was 94.9% (mean
KL 0.0027, measured on the earlier `270B-tokens_step23035` export), which reflects the bf16 rounding
of the reference, not an error of the runtime.
On the finisher GGUF, the largest per-layer relative RMS difference between Metal and CPU was about 2.7e-5
(logits 6.8e-6) at the final position of a 512-token text, and the two gave identical 48-token greedy continuations
on three prompts. `lut8` (CPU) quantizes expert lookup tables to 8 bits and is slightly approximate; `lut` and
`lut9` are exact.
### Limitations of the runtime
- One sequence per process, greedy or simple sampled generation. No chat template, batching or server.
- Not integrated into `llama-cli`, `llama-server` or the standard llama.cpp model loader.
- Context is limited by `-c`. Inference uses a 2,048-token window for all sliding-window layers. With the
finisher GGUF the global-attention layers attend to up to 16 selected blocks of 128 tokens per query, always
including the current block (so inputs up to 2,048 tokens are attended in full); with the lambda GGUFs they are
dense. The finisher was trained on sequences
up to 65,536 tokens and lambda on 4,096, but the runtime was checked only on 512-token passages and 48-token
continuations; longer inputs run (raise `-c`) but were not qualified in the runtime, and the long-context
numbers above come from the training code.
## Tech report
The Parallax tech report: [https://parallax.chutes.ai/tech-report.pdf](https://parallax.chutes.ai/tech-report.pdf). It is AI-generated from the project's
measurements and logs, and it is a living document that changes as the run progresses. It describes lambda, including the end of training and the final benchmark comparison.
## Limitations
This is an early base model. It can produce incorrect, biased or nonsensical output. It has no
alignment or safety tuning, and it is small and far from converged.
## License
MIT.
<sub>Table updated 2026-10-21 23:47 UTC.</sub>
|