Add pretraining training-loss and held-out validation figure

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+pretraining-training-loss.png filter=lfs diff=lfs merge=lfs -text

PACKAGING_VERIFICATION.json

@@ -29,5 +29,8 @@
     "value_and_stop_matches": 40,
     "correct_against_reference": 29
   },
-  "scope": "Package entry-point validation; no new blind benchmark or full-test cross-runtime parity claim."
+  "scope": "Package entry-point validation; no new blind benchmark or full-test cross-runtime parity claim.",
+  "documentation_update": "Owner-provided pretraining loss/validation image added; native model, tokenizer, adapter and weight bytes are unchanged.",
+  "native_bundle_sha256": "5df3ada361b6fe7226170a73cd8d9cf08a70dd3cdee3ea7c78cb751e7922c94c",
+  "source_commit": "c5198ca0fbbb6f1ac10b3685846602226facb21e"
 }

README.md

@@ -108,6 +108,14 @@ These are intended processed-token sampling shares, not unique-document counts.
 
 Nominal pretraining peak LR was 3e-4, but the adaptive controller capped the effective LR at **3.221225472e-5** at the selected checkpoint. AdamW used betas (0.9, 0.95), epsilon 1e-8, weight decay 0.1 and gradient clipping 1.0. Exact warmup, batch-transition history, source revisions, filters and controller state are recorded in the technical report.
 
+### Training run
+
+The 175.6M-parameter base was trained from random initialization using next-token prediction across the educational web, Wikipedia, Cosmopedia and Python mixture. Five Apple Silicon Macs carried out distributed pretraining over Thunderbolt/RDMA. Validation was measured every 50M tokens. The curves show a steep early loss reduction followed by slower improvement; the published base was selected using held-out validation.
+
+![Stage A pretraining training loss, EMA and held-out validation loss](pretraining-training-loss.png)
+
+*Training loss and held-out validation during pretraining. Left: raw training loss and exponential moving average. Right: held-out validation loss.*
+
 ### 2. SFT-065: answering from supplied evidence
 
 This stage starts directly from Stage A/132005. The prepared corpus has **25,000 grounded QA examples**: 12,000 natural QA, 5,000 contrasting questions, 3,000 answerability/calibration examples, 3,000 multipart questions and 2,000 corrections. Published inputs/answers come from selected SQuAD 2.0 and Dolly closed QA; controlled fictional examples use programmatically checked fact oracles.

SHA256SUMS

@@ -2,18 +2,19 @@ d0b49f29bcb68dbb1d9967dc026b972ef9fa4a22eb67c9fea88539a90a786d03  CITATION.cff
 82b9908c9fd21fd8c9d10ad809e7c48446485ee98a56bfd9d64bf9fb353761c2  INFERENCE_VERIFICATION.json
 d29cf5952a82db444a6df56446806b1f4e2ccbee1b4aa6b93cb7eeeabcab8536  JSONLITE_RESULTS.json
 c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4  LICENSE
-8179f766c423a10e719ccd5efeb097b60e460323507bf2644b9626edc0a441c6  PACKAGING_VERIFICATION.json
-58b9318ac486b4d70a7a9f805aef385a51dba74380f2a276481690c4034c0a17  README.md
+d93ce1dcf8bf09dbb15b92b3da659a89a335122f0f151b547534ef05590fb2fd  PACKAGING_VERIFICATION.json
+ba3fab6986fee521149d0ec02e1d37b7a26ceba9b8ff2bd93554d1c0ca8c1ed9  README.md
 994a1798d7b961e34b581bf5337dd5dd14f533e9ebfb4fe1be10cd7b17edf0ec  RUN_GUIDE.md
 5bc419e245a6d8f074186210d9cba59df3e2cc82dfd04452c625268108a6796d  SOURCE_NOTICE.md
-5edeb60b77046aebf671ebc2c670d0637658f1929c621f66b9de89b445a3c623  TECHNICAL_REPORT.md
+478e1b7ec484be803d3bbfb1fa3b202d89b47889f412bed5c839202b7238ba79  TECHNICAL_REPORT.md
 423f0d1e0e2690505c24e6d31308db728fba60657c5166e177bb269d6a7198d0  THIRD_PARTY_NOTICES.md
 6f8de0b591c8a07885c8adcab359b8ca1cdc81be5f90638cc0ce286f3899c6c5  checkpoint_provenance.json
 fcc46a0f69fc4afb8d6e25361a013e7656ac6758ae5a49cc6b629f95bf728c1a  complete-request-exact.svg
 2f918a15b7b06a792e48d476fa9884dfc7fb46d5ccd0f756bd3890a8ad259d96  config.json
-5f66ff6f09a6ce26603220defe3aa2bc0231f7144785bc33865683d0097a2c34  inference.py
+01d820f55a4dd08b6c9cc940ef84b45d2f80a733ae919876ac97e90ffec0c360  inference.py
 d552c1380c71c7c0569e9d692ec9c46c5c94bb87a7b024e56df0f0e36df4d798  model.safetensors
-459b495441931bf520ea14274401509bc6441cd701bbbf2a1d162fcae5eacbc4  native_bundle.zip
+5df3ada361b6fe7226170a73cd8d9cf08a70dd3cdee3ea7c78cb751e7922c94c  native_bundle.zip
+641fe860d6fd24547fe4d15bd852190f00ddec31bd74eb7ca9eeae74031c2f0c  pretraining-training-loss.png
 c7b1a3c0efb09ef3b5416a24072e48b25e2cc1f122fb5ed638c0f95b72594af9  requirements.txt
 e095f0413a11fa91fbb58f37d00e4c6bcda59beb89b28f6f680db1436a7e8891  tokenizer.json
 619f500553367eb76f5be00a98625130ec857b9c367561334984096c029ba2a9  tokenizer_config.json

TECHNICAL_REPORT.md

@@ -218,6 +218,14 @@ Validation and qualitative samples occur at 50M-token boundaries. The fixed vali
 
 Regular saves occur every 100M tokens, plus best-validation and clean-stop saves. Exact checkpoint metadata is authoritative. The initial October 2 technical report's early observations, 20B target and old schedule describe history; they are superseded here for this selected artifact. Contemporary qualitative logs often showed roughly 41k tokens/s, but no reproducible end-to-end cluster performance/energy benchmark is established by this report. The separate 42.7k profiling candidate is not claimed as the configuration that trained this checkpoint.
 
+### Training run
+
+The 175.6M-parameter base was trained from random initialization using next-token prediction across the educational web, Wikipedia, Cosmopedia and Python mixture. Five Apple Silicon Macs carried out distributed pretraining over Thunderbolt/RDMA. Validation was measured every 50M tokens. The curves show a steep early loss reduction followed by slower improvement; the published base was selected using held-out validation.
+
+![Stage A pretraining training loss, EMA and held-out validation loss](pretraining-training-loss.png)
+
+*Training loss and held-out validation during pretraining. Left: raw training loss and exponential moving average. Right: held-out validation loss.*
+
 ## 7. Common supervised objective and precision
 
 All inherited SFT stages run locally on Mac1 and update all 175,611,648 parameters. No LoRA/adapters, added span head, frozen-layer adaptation, reward model, DPO, PPO or GRPO is part of this lineage.

inference.py

@@ -12,7 +12,7 @@ import tempfile
 import zipfile
 
 ROOT = Path(__file__).resolve().parent
-BUNDLE_SHA256 = "459b495441931bf520ea14274401509bc6441cd701bbbf2a1d162fcae5eacbc4"
+BUNDLE_SHA256 = "5df3ada361b6fe7226170a73cd8d9cf08a70dd3cdee3ea7c78cb751e7922c94c"
 
 def main():
     archive = ROOT / "native_bundle.zip"

native_bundle.zip

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:459b495441931bf520ea14274401509bc6441cd701bbbf2a1d162fcae5eacbc4
-size 1685929
+oid sha256:5df3ada361b6fe7226170a73cd8d9cf08a70dd3cdee3ea7c78cb751e7922c94c
+size 1980238