Publish v1.0.0 model card and technical report

Adds the audited technical report, minimal llama.cpp/Pi demo, scientific figures, publication evidence, citation metadata, and paper license. Replaces superseded presentation assets.

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 gguf/zero-125m-f16.gguf filter=lfs diff=lfs merge=lfs -text
 gguf/zero-125m-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+lm-from-zero-technical-report.pdf filter=lfs diff=lfs merge=lfs -text

CITATION.cff

@@ -0,0 +1,15 @@
+cff-version: 1.2.0
+message: "If you use this work, please cite it as below."
+title: "lm-from-zero: A Local-First, Reproducible Language-Model Pipeline from Tokenization to Serving"
+type: software
+authors:
+  - name: tarpous
+version: 1.0.0
+date-released: 2026-08-10
+license: Apache-2.0
+repository-code: "https://github.com/tarpous/lm-from-zero"
+url: "https://huggingface.co/tarpous/zero-125m"
+abstract: >-
+  A local-first language-model pipeline covering deterministic byte-level BPE,
+  checked data shards, dense OLMo2, Mamba-2, masked diffusion, pretraining,
+  post-training, evaluation, export, quantization, and local serving.

LICENSE

@@ -180,7 +180,7 @@
       To apply the Apache License to your work, attach the following
       boilerplate notice, with the fields enclosed by brackets "[]"
       replaced with your own identifying information. (Don't include
-      the brackets!)  The text should be enclosed in the appropriate
+      the brackets!) The text should be enclosed in the appropriate
       comment syntax for the file format. We also recommend that a
       file or class name and description of purpose be included on the
       same "printed page" as the copyright notice for easier

LICENSE-PAPER

@@ -0,0 +1,17 @@
+Creative Commons Attribution 4.0 International
+
+Copyright 2026 tarpous
+
+The technical report and publication figures in this repository are licensed
+under the Creative Commons Attribution 4.0 International Public License.
+
+You are free to share and adapt this material for any purpose, including
+commercially, provided that you give appropriate credit, provide a link to the
+license, and indicate whether changes were made. No additional restrictions
+may be applied that legally restrict others from doing anything the license
+permits.
+
+The complete legal code is available at:
+https://creativecommons.org/licenses/by/4.0/legalcode
+
+SPDX-License-Identifier: CC-BY-4.0

README.md

@@ -1,117 +1,158 @@
 ---
 license: apache-2.0
-language:
-- en
 library_name: transformers
 pipeline_tag: text-generation
+datasets:
+  - HuggingFaceFW/fineweb-edu
+language:
+  - en
 tags:
-- lm-from-zero
-- olmo2
-- causal-lm
-- byte-level-bpe
-- gguf
-- llama.cpp
+  - olmo2
+  - base-model
+  - gguf
+  - from-scratch
+  - reproducible-training
 ---
 
 # zero-125m
 
-`zero-125m` is a compact decoder-only language model trained from scratch in
-the [`lm-from-zero`](https://github.com/tarpous/lm-from-zero) project. It is
-released in two interoperable forms:
-
-- a standard Hugging Face `Olmo2ForCausalLM` package for Transformers and
-  vLLM; and
-- F16 and Q8_0 GGUF files under [`gguf/`](gguf) for llama.cpp.
-
-This repository is the immutable artifact home. Source code, training
-configuration, test coverage, generated report, and local reproduction
-instructions remain in the linked GitHub project.
-
-## Model details
+`zero-125m` is a 124,680,960-parameter OLMo2-compatible base language model
+trained from scratch on 2,500,001,792 FineWeb-Edu tokens. It is the released
+dense model from the
+[`lm-from-zero`](https://github.com/tarpous/lm-from-zero) project, which covers
+the complete path from byte-level BPE tokenization to local serving.
+
+This repository includes the native Transformers model and tokenizer, F16 and
+Q8_0 GGUF derivatives, evaluation evidence, and the full technical report.
+
+![Local base-model continuation in Pi using the Q8_0 GGUF through llama.cpp](assets/zero-125m-serving-demo.gif)
+
+[Technical report](lm-from-zero-technical-report.pdf) |
+[Source and reproducibility](https://github.com/tarpous/lm-from-zero) |
+[Capture evidence](assets/pi-demo-evidence.json)
+
+## Intended use
+
+The model is intended for research, education, systems experiments, conversion
+tests, and local inference demonstrations. It is a pretrained base model. It
+has not been aligned for assistant use and should not be treated as a reliable
+chatbot, factual reference, or safety-critical system.
+
+## Training
+
+- Architecture: OLMo2-compatible causal decoder
+- Parameters: 124,680,960
+- Layers: 12
+- Hidden width: 768
+- Query heads: 12
+- Key/value heads: 4
+- Context length: 2,048 tokens
+- Vocabulary: 32,000 byte-level BPE tokens
+- Training tokens: 2,500,001,792
+- Data: FineWeb-Edu `sample-10BT`, immutable revision
+  `87f09149ef4734204d70ed1d046ddc9ca3f2b8f9`
+- Optimizer: AdamW with linear warmup and cosine decay
+- Numerical format: bfloat16 training with fp32 optimizer state
+- Hardware: one NVIDIA RTX 4080 SUPER
+
+The tokenizer, shard manifest, model configuration, checkpoint, evaluation,
+and converted package are connected by recorded hashes. The native
+Transformers export has an exact maximum fp32 logit error of 0.0 against the
+project checkpoint in the recorded parity evaluation.
 
-- **Architecture:** dense OLMo2-compatible causal decoder with RoPE, grouped
-  query attention, Q/K normalization, RMSNorm, SwiGLU, and untied output head.
-- **Tokenizer:** project-trained 32K byte-level BPE; the tokenizer files in
-  this repository are required for Transformers use.
-- **Training data:** a deterministic shard build from
-  [`HuggingFaceFW/fineweb-edu`](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu),
-  `sample-10BT` revision `87f09149ef4734204d70ed1d046ddc9ca3f2b8f9`
-  (ODC-By). Raw training data is not redistributed here.
-- **Training lineage:** see [`export_manifest.json`](export_manifest.json) for
-  the source checkpoint, parameter count, tokenizer hash, model hash, and
-  fp32 export-parity result.
-
-## Files and verification
+## Evaluation
 
-| Path | Purpose |
-|---|---|
-| `model.safetensors` and configuration/tokenizer files | Standard Transformers package |
-| `gguf/zero-125m-f16.gguf` | F16 GGUF conversion |
-| `gguf/zero-125m-q8_0.gguf` | Q8_0 GGUF for local llama.cpp serving |
-| `gguf/gguf-manifest.json` | Checksums, converter revision, tokenizer parity, and CUDA smoke evidence |
-| `evaluation/` | Fixed-window validation/test evidence and generated summary |
-| `figures/` | Generated FineWeb training and evaluation figures |
+Evaluation uses fixed, non-wrapping FineWeb-Edu windows.
 
-Verify any downloaded file against the SHA-256 values in the export or GGUF
-manifest before use.
+| Split | Mean loss | Perplexity |
+|---|---:|---:|
+| Validation | 3.324985 | 27.799 |
+| Test | 3.400198 | 29.970 |
 
-## Loading
+These measurements describe next-token likelihood under this tokenizer and
+window protocol. They are not instruction-following, factuality, or safety
+scores.
 
-Transformers:
+## Transformers usage
 
 ```python
+import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
 model_id = "tarpous/zero-125m"
 tokenizer = AutoTokenizer.from_pretrained(model_id)
-model = AutoModelForCausalLM.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    dtype=torch.bfloat16,
+    device_map="auto",
+)
+
+inputs = tokenizer("Once upon a time,", return_tensors="pt").to(model.device)
+with torch.inference_mode():
+    output = model.generate(
+        **inputs,
+        max_new_tokens=64,
+        do_sample=False,
+    )
+
+print(tokenizer.decode(output[0], skip_special_tokens=True))
 ```
 
-llama.cpp Q8_0 server:
-
-```bash
-llama-server \
-  -m gguf/zero-125m-q8_0.gguf \
-  --n-gpu-layers 99 --ctx-size 2048 \
-  --host 127.0.0.1 --port 8091
+This is a base model, so continuations may be repetitive, abrupt, or
+inaccurate. A prompt formatted as a conversation does not turn it into an
+instruction-following model.
+
+## GGUF files
+
+| File | Size | SHA-256 |
+|---|---:|---|
+| `zero-125m-f16.gguf` | 250,567,744 bytes | `4863f80ea315fc0713da86750f71cddc8dfbec09c0d3f8a853cee650220027ab` |
+| `zero-125m-q8_0.gguf` | 133,708,864 bytes | `401866c0a19e803eea19b00e056915a37b05ebe1146430b618355071680b8692` |
+
+The Q8_0 artifact produced 76.5 decoded tokens per second in the recorded
+bounded CUDA llama.cpp smoke. Throughput is hardware and build dependent.
+Conversion details and the pre-tokenizer compatibility patch are documented in
+the project repository.
+
+## Broader project evidence
+
+The project also trained a 20M dense FineWeb model and separate TinyStories
+models near 20M parameters using dense attention, Mamba-2, and masked discrete
+diffusion. It includes dense ablations, supervised fine-tuning, DPO, and local
+judge calibration. Those results describe the research project and are not
+additional training stages of this released base checkpoint.
+
+The full report is available in this model repository as
+`lm-from-zero-technical-report.pdf` and in the GitHub release.
+
+## Limitations
+
+- The model is small and was trained for only 2.5 billion tokens.
+- It is a base model without safety or instruction tuning.
+- FineWeb-Edu may contain errors, bias, personal information, and other
+  undesirable web content despite dataset-level filtering.
+- The model can generate false, biased, offensive, repetitive, or incoherent
+  text.
+- Evaluation covers held-out language-model loss, not downstream capability or
+  responsible-deployment criteria.
+- The planned 125M FineWeb Mamba-2 and masked-diffusion runs were not completed.
+- The planned 500-prompt cross-judge calibration was not completed.
+
+Users are responsible for evaluating the model in their own setting and for
+adding safeguards appropriate to any application.
+
+## Citation
+
+```bibtex
+@software{tarpous2026lmfromzero,
+  author = {tarpous},
+  title = {lm-from-zero: A Local-First, Reproducible Language-Model Pipeline from Tokenization to Serving},
+  year = {2026},
+  url = {https://github.com/tarpous/lm-from-zero}
+}
 ```
 
-The local validation used llama.cpp `b10069` for the Q8_0 CUDA smoke. The
-runtime conversion details and the exact pre-tokenizer compatibility mapping
-are documented in the [source repository](https://github.com/tarpous/lm-from-zero/blob/b27bb9ce063a26c8606bd35207a2f19060e1cbc8/docs/gguf-conversion.md).
-
-## Evaluation
-
-Evaluation uses fixed, non-wrapping held-out FineWeb token windows. The raw
-machine-readable records are in
-[`evaluation/zero-125m-validation.jsonl`](evaluation/zero-125m-validation.jsonl)
-and [`evaluation/zero-125m-test.jsonl`](evaluation/zero-125m-test.jsonl).
-The corresponding generated table is
-[`evaluation/presentation-summary.md`](evaluation/presentation-summary.md).
-
-These files are the canonical evaluation evidence. They bind metrics to the
-exact tokenizer and shard manifests; do not compare them directly with results
-that use a different data split, tokenizer, token budget, or evaluation
-protocol.
-
-## Intended use and limitations
-
-This is an educational, compact base language model intended for local
-experimentation, reproducibility work, and inference-stack testing. It is not
-instruction-tuned and should not be treated as a reliable assistant.
-
-Outputs can be incorrect, biased, unsafe, or unsuitable for a particular
-context. The project does not claim broad safety evaluation or benchmark
-competitiveness. Do not use this model for high-stakes decisions without
-independent validation and appropriate safeguards.
-
-## License and provenance
-
-The model weights, tokenizer, configurations, GGUF conversions, and files in
-this repository are released under [Apache-2.0](LICENSE). Dataset terms remain
-those of their respective providers; the FineWeb-Edu source used here is
-recorded as ODC-By in the project provenance.
+## License
 
-For the complete project report, implementation, training/evaluation lineage,
-and reproducible local commands, see
-[`tarpous/lm-from-zero`](https://github.com/tarpous/lm-from-zero).
+The model weights, tokenizer, GGUF files, and code are released under
+Apache-2.0. The technical report and its figures are released under CC BY 4.0.

assets/pi-demo-evidence.json

@@ -0,0 +1,33 @@
+{
+  "format": "lm-from-zero-pi-demo",
+  "format_version": 1,
+  "label": "Local base-model text continuation - Q8_0 GGUF via llama.cpp and Pi",
+  "model": {
+    "filename": "zero-125m-q8_0.gguf",
+    "sha256": "401866c0a19e803eea19b00e056915a37b05ebe1146430b618355071680b8692"
+  },
+  "prompt": "Once upon a time,",
+  "seed": 0,
+  "sampling": "greedy",
+  "server_context_length": 2048,
+  "pi_declared_context_window": 8192,
+  "maximum_new_tokens": 256,
+  "bridge": "Final user text only; newline stop",
+  "interface": {
+    "pi": "0.83.0",
+    "vhs": "vhs version v0.11.0 (c6af91a)",
+    "ttyd": "ttyd version 1.7.7-e2819f2",
+    "ffmpeg": "ffmpeg version 7.1.3-0+deb13u1 Copyright (c) 2000-2025 the FFmpeg developers",
+    "llama_cpp": "b10069"
+  },
+  "gif": {
+    "path": "docs/assets/zero-125m-serving-demo.gif",
+    "bytes": 61703,
+    "sha256": "4a8be28346e7f7b889ec7c6c5dc53b638d82c4726a8a19acbde3b882c9001cf6"
+  },
+  "poster": {
+    "path": "docs/assets/zero-125m-serving-demo.png",
+    "bytes": 50588,
+    "sha256": "9afc98818204dcb0ca579fd02be785c631f4c3a131580f930ca4d58207f0fa6b"
+  }
+}

evaluation/presentation-summary.md

@@ -1,23 +0,0 @@
-# Generated result summary
-
-This file is generated by `tools/build_presentation_assets.py` from canonical local artifacts. Do not edit its values by hand.
-
-## FineWeb dense evaluation
-
-| Model | Validation loss | Validation PPL | Test loss | Test PPL |
-|---|---:|---:|---:|---:|
-| 20M | 4.218200 | 67.911 | 4.264027 | 71.096 |
-| 125M | 3.324985 | 27.799 | 3.400198 | 29.970 |
-
-## 20M architecture study
-
-- Dense causal PPL: 6.249
-- Mamba-2 causal PPL: 6.852
-- Diffusion masked reconstruction loss: 3.740
-- Diffusion variational upper bound: 3.045
-
-## Local judge and serving
-
-- 100-prompt cross-judge agreement: 83.0%; Cohen's kappa 0.616.
-- Qwen primary 500-prompt run: 1,000 calls; 0 parse failures; 19.8% order flips.
-- `zero-125m` vLLM completion latency: 373 ms for 8 generated tokens.

evaluation/publication-results.json

@@ -0,0 +1,281 @@
+{
+  "acceleration": {
+    "cells": 78,
+    "maximum_loss_absolute_delta": 4.38690185546875e-05,
+    "promoted_architecture": "diffusion",
+    "promoted_cell": "fused-adamw",
+    "speedup": 1.136088420878017
+  },
+  "architecture_study": {
+    "dense": {
+      "compute_seconds": 1310.775,
+      "loss": 1.832368791103363,
+      "model": {
+        "context_length": 1024,
+        "hidden_size": 384,
+        "layers": 5,
+        "parameter_count": 20159104,
+        "vocabulary_size": 16000
+      },
+      "perplexity": 6.24867093730173,
+      "predicted_tokens": 196416,
+      "sequences": 192,
+      "tokens_per_second": 364043.5415766547,
+      "tokens_trained": 500006912
+    },
+    "diffusion": {
+      "causal_perplexity_applicable": false,
+      "compute_seconds": 1313.311,
+      "masked_reconstruction_loss": 3.740404522177884,
+      "masked_tokens": 97655,
+      "model": {
+        "context_length": 1024,
+        "hidden_size": 384,
+        "layers": 4,
+        "parameter_count": 19959168,
+        "vocabulary_size": 16000
+      },
+      "tokens_trained": 528990208,
+      "variational_upper_bound": 3.0454040666421256
+    },
+    "mamba2": {
+      "compute_seconds": 3630.512,
+      "loss": 1.924551824728648,
+      "model": {
+        "context_length": 1024,
+        "hidden_size": 384,
+        "layers": 7,
+        "parameter_count": 19943164,
+        "vocabulary_size": 16000
+      },
+      "perplexity": 6.852077046149102,
+      "predicted_tokens": 196416,
+      "sequences": 192,
+      "tokens_per_second": 156774.6946154588,
+      "tokens_trained": 605544448
+    },
+    "seed": 1337
+  },
+  "datasets": {
+    "fineweb_edu": {
+      "dataset_id": "HuggingFaceFW/fineweb-edu",
+      "documents": 2450915,
+      "license": "odc-by",
+      "revision": "87f09149ef4734204d70ed1d046ddc9ca3f2b8f9",
+      "test_tokens": 25260908,
+      "train_tokens": 2500000871,
+      "validation_tokens": 25661975
+    },
+    "tinystories": {
+      "dataset_id": "roneneldan/TinyStories",
+      "documents": 112206,
+      "license": "cdla-sharing-1.0",
+      "revision": "f54c09fd23315a6f9c86f9dc80f725de7d8f9c64",
+      "test_tokens": 237469,
+      "train_tokens": 23323923,
+      "validation_tokens": 245778
+    }
+  },
+  "dense_ablations": {
+    "screening_tokens_per_seed": 100007936,
+    "seeds": [
+      1337,
+      2027,
+      3407
+    ],
+    "variants": [
+      {
+        "change": "swiglu_to_gelu",
+        "downstream_validation_perplexity": null,
+        "mean_terminal_loss": 1.6645395755767822,
+        "mean_tokens_per_second": 402805.640803283,
+        "name": "gelu",
+        "terminal_loss_std": 0.13158630932487156
+      },
+      {
+        "change": "adamw_to_hybrid_muon",
+        "downstream_validation_perplexity": 5.713338699142928,
+        "mean_terminal_loss": 1.522047758102417,
+        "mean_tokens_per_second": 307086.7898872683,
+        "name": "hybrid_muon",
+        "terminal_loss_std": 0.13786254833334086
+      },
+      {
+        "change": "rmsnorm_to_layernorm",
+        "downstream_validation_perplexity": 5.90013935655289,
+        "mean_terminal_loss": 1.618891954421997,
+        "mean_tokens_per_second": 389267.12581697945,
+        "name": "layer_norm",
+        "terminal_loss_std": 0.13529636072177245
+      },
+      {
+        "change": "rope_to_learned_absolute_positions",
+        "downstream_validation_perplexity": null,
+        "mean_terminal_loss": 1.7279485861460369,
+        "mean_tokens_per_second": 351116.52991555206,
+        "name": "learned_absolute_positions",
+        "terminal_loss_std": 0.1359720722424022
+      },
+      {
+        "change": "gqa_to_mha",
+        "downstream_validation_perplexity": 5.84947328452131,
+        "mean_terminal_loss": 1.6075799067815144,
+        "mean_tokens_per_second": 382954.5419660864,
+        "name": "mha",
+        "terminal_loss_std": 0.13742111225327125
+      },
+      {
+        "change": "tie_input_output_embeddings",
+        "downstream_validation_perplexity": 5.826874126198687,
+        "mean_terminal_loss": 1.6208405097325642,
+        "mean_tokens_per_second": 435082.9346366357,
+        "name": "tied_embeddings",
+        "terminal_loss_std": 0.14300388423302463
+      },
+      {
+        "change": "remove_qk_normalization",
+        "downstream_validation_perplexity": null,
+        "mean_terminal_loss": 1.8348172108332317,
+        "mean_tokens_per_second": 398739.79417508264,
+        "name": "without_qk_norm",
+        "terminal_loss_std": 0.13095864590886672
+      }
+    ]
+  },
+  "fineweb_dense": {
+    "125m": {
+      "model": {
+        "attention_heads": 12,
+        "context_length": 2048,
+        "hidden_size": 768,
+        "intermediate_size": 2048,
+        "key_value_heads": 4,
+        "layers": 12,
+        "model_config_sha256": "c886f7c13c6c027276c3f2ddcede72425fac511e2d1b2551b5d7eadd121f2bf4",
+        "optimizer_steps": 152588,
+        "parameter_count": 124680960,
+        "tokens_trained": 2500001792,
+        "vocabulary_size": 32000
+      },
+      "test": {
+        "loss": 3.4001976028084755,
+        "perplexity": 29.970021622760513,
+        "predicted_tokens": 65504,
+        "sequences": 32,
+        "tokens_per_second": 73350.9216982716
+      },
+      "validation": {
+        "loss": 3.3249846026301384,
+        "perplexity": 27.798570614501042,
+        "predicted_tokens": 65504,
+        "sequences": 32,
+        "tokens_per_second": 86392.51131817144
+      }
+    },
+    "20m": {
+      "model": {
+        "attention_heads": 8,
+        "context_length": 1024,
+        "hidden_size": 256,
+        "intermediate_size": 704,
+        "key_value_heads": 2,
+        "layers": 5,
+        "model_config_sha256": "d421a006c0f4ea1ae8a6a7a9a1396d93ea0634567ddabd37bc2f70c8632bcc63",
+        "optimizer_steps": 61036,
+        "parameter_count": 19910976,
+        "tokens_trained": 500006912,
+        "vocabulary_size": 32000
+      },
+      "test": {
+        "loss": 4.264026649296284,
+        "perplexity": 71.09568524875908,
+        "predicted_tokens": 261888,
+        "sequences": 256,
+        "tokens_per_second": 332064.2247825187
+      },
+      "validation": {
+        "loss": 4.21819968521595,
+        "perplexity": 67.91111278850597,
+        "predicted_tokens": 261888,
+        "sequences": 256,
+        "tokens_per_second": 318733.37056578987
+      }
+    }
+  },
+  "format": "lm-from-zero-publication-results",
+  "format_version": 1,
+  "judge": {
+    "cohen_kappa": 0.616274476609672,
+    "cross_judge_prompts": 100,
+    "cross_judge_tasks": 200,
+    "gemma_order_flip_rate": 0.21,
+    "primary_calls": 1000,
+    "primary_order_flip_rate": 0.198,
+    "primary_parse_failures": 0,
+    "primary_prompts": 500,
+    "qwen_order_flip_rate": 0.26,
+    "raw_agreement": 0.83
+  },
+  "post_training": {
+    "dpo": {
+      "beta": 0.1,
+      "dpo_objective_preference_accuracy": 0.6443925233644859,
+      "dpo_sequence_preference_accuracy": 0.4331775700934579,
+      "final_loss": 0.38742566108703613,
+      "first_loss": 0.6931471824645996,
+      "heldout_pairs": 10700,
+      "mean_forward_kl": 0.023786345937711253,
+      "mean_heldout_dpo_loss": 0.7430345315044359,
+      "mean_last_100_loss": 0.7263714300002903,
+      "mean_reward_margin": 8.639506955102226,
+      "sft_sequence_preference_accuracy": 0.4321495327102804,
+      "steps": 25000,
+      "training_pairs": 50000
+    },
+    "sft": {
+      "examples": 100000,
+      "final_loss": 1.4705926179885864,
+      "first_loss": 6.417908191680908,
+      "mean_last_100_loss": 1.4433360010385514,
+      "steps": 12500,
+      "tokens": 62684063
+    }
+  },
+  "release": {
+    "gguf": {
+      "F16": {
+        "bytes": 250567744,
+        "filename": "zero-125m-f16.gguf",
+        "sha256": "4863f80ea315fc0713da86750f71cddc8dfbec09c0d3f8a853cee650220027ab"
+      },
+      "Q8_0": {
+        "bytes": 133708864,
+        "filename": "zero-125m-q8_0.gguf",
+        "sha256": "401866c0a19e803eea19b00e056915a37b05ebe1146430b618355071680b8692"
+      }
+    },
+    "hf_fp32_max_abs_error": 0.0,
+    "hf_parameter_count": 124680960,
+    "llama_cpp_tokens_per_second": 76.50524060898172,
+    "model_safetensors_bytes": 498738888,
+    "model_safetensors_sha256": "4828a7f18d40cba9504d593fbcc1b4ad67c52f2a6387b4de8c7ab682c9c31e1d",
+    "vllm_latency_ms": 373.0
+  },
+  "tokenizers": {
+    "fineweb_edu": {
+      "merge_count": 31736,
+      "sha256": "e16b820d0d8c7ad8e89904714ef6d89eb573a4e651dd337bde56fd8f418476dd",
+      "vocabulary_size": 32000
+    },
+    "tinystories": {
+      "merge_count": 15736,
+      "sha256": "f65920522fb584143cfee0f0e9d7581286bbde9fdc5c068a34d5ce0441695da9",
+      "vocabulary_size": 16000
+    }
+  },
+  "unfinished": [
+    "125M FineWeb training for Mamba-2",
+    "125M FineWeb training for masked diffusion",
+    "500-prompt cross-judge calibration"
+  ]
+}

evaluation/publication-summary.md

@@ -0,0 +1,12 @@
+# Generated publication summary
+
+This file is generated from canonical experiment artifacts. Do not edit its values by hand.
+
+| Result | Validation | Test |
+|---|---:|---:|
+| FineWeb-Edu dense 20M perplexity | 67.911 | 71.096 |
+| FineWeb-Edu dense 125M perplexity | 27.799 | 29.970 |
+
+The seed-1337 full-budget TinyStories study measured causal perplexity 6.249 for Dense and 6.852 for Mamba-2. Masked diffusion recorded reconstruction loss 3.740 nats and a variational upper bound of 3.045 nats; causal perplexity does not apply to that objective.
+
+The 100-prompt cross-judge calibration reached 83.0% raw agreement and Cohen's kappa 0.616. The separate 500-prompt primary-judge run completed 1,000 order-swapped calls with 0 parse failures after repair and a 19.8% flip rate.

lm-from-zero-technical-report.pdf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:998f3f312a0957bc6da07fa048c9a0165eba321cf3c3031775f6df040a6f74d7
+size 537873