Phase 5: keep the raw publish/Gate 5 stdout with the docs, so the private scratch repo is redundant

docs/phase5/dryrun_card.md

@@ -0,0 +1,89 @@
+ckpt latest.json -> {'step': 3814, 'path_in_repo': 'final'}
+tensors 200 params 106,194,240 dtypes ['F32']
+card: 106,194,240 params from the safetensors header; 6 files hashed
+---
+license: cc-by-sa-4.0
+base_model: Cion-lab/ounce100m-v1
+language: en
+library_name: transformers
+pipeline_tag: text-generation
+tags: [ounce100m, from-scratch, base, decoder-only, huggingfacetb]
+---
+
+# ounce100m-v1
+
+A **106,194,240-parameter decoder-only transformer trained from scratch on
+999,817,216 tokens** of `Cion-lab/ounce100m-mix-v1`, in 3814 optimiser steps on
+**2×Nvidia Tesla T4** inside Kaggle notebook sessions. It is the deliverable of a project whose whole point
+was to find out what a ~100 M-parameter model reaches from ~1 B tokens of a curated public mix, and to
+report it honestly, including where that is near chance.
+
+## Measured facts
+
+| | |
+|---|---|
+| Parameters | **106,194,240** (recomputed from `model.safetensors`'s own header, not asserted) |
+| Architecture | 22 layers, hidden 576, GQA 9 Q / 3 KV heads, SwiGLU 1536, RMSNorm 1e-05, RoPE θ=10000.0, **tied** embeddings, vocab 49,152 |
+| Sequence length | 1024 — which is why ARC's 25-shot prompts are truncated from the left by the eval harness |
+| Tokens consumed | **999,817,216** (99.98 % of the 1.0 B target) |
+| Batch | micro 4 × accum 32 × 2 ranks = 262,144 tokens/step |
+| Optimiser | `adamw_torch`, LR 0.0006, warmup fraction 0.02, trapezoid decay over the final 20.0 % of the schedule |
+| Precision | fp16 autocast + fp32 master weights + GradScaler — asserted at the start of every session, and each session's log line `precision: fp16=True bf16=False scaler=GradScaler` is the evidence |
+| Throughput | 12,150 tok/s in the session that produced this export |
+| Attention | `eager` |
+| Gradient checkpointing | `False` |
+| Randomness | model seed 20260919, data seed 20260919 |
+| Wall clock | 4.59 h in the session that produced this export. The run is split across sessions at checkpoint boundaries, so this is **not** the total training time |
+| Loss | 4.43187255859375 -> 4.264701843261719 |
+| Validation PPL | **75.98** on Cion-lab/ounce100m-mix-v1's held-out `val/` shards (22,934,043 tokens, never trained on) |
+
+## Data
+
+`Cion-lab/ounce100m-mix-v1`: 1,109,714,831 training tokens in 139 shards + 22,934,043
+held-out validation tokens, cut with `HuggingFaceTB/SmolLM2-135M:tokenizer.json` (49,152 vocab,
+Apache-2.0), from 15 public sources at ≥15
+per shard. Format and loading code are in that dataset's card.
+
+**Contamination.** Overlap was measured mechanically against benchmark **train/validation/dev** material of
+the eight evaluation tasks (13-token windows, counts only — no benchmark item text was ever read, and test
+splits stayed untouched until evaluation). `audit.json` in the dataset repo records
+`overlap_total = 0` over 1,073,047 documents and
+21,898 held-out documents, `tasks_covered` = 8/8.
+Building it required dropping 5878 documents
+(17,366,967 tokens)
+that matched reference material; the exact exclusion masks ship in `filter/` so the mix is reproducible.
+
+## Evaluate it
+
+```bash
+pip install "lm-eval==0.4.13"
+lm_eval --model hf --model_args pretrained=Cion-lab/ounce100m-v1,dtype=float16,trust_remote_code=False \
+  --tasks arc_challenge,arc_easy,hellaswag,mmlu,piqa,truthfulqa_mc1,truthfulqa_mc2,winogrande,gsm8k \
+  --batch_size 8 --seed 42 --log_samples
+```
+
+The run's own results, with shot counts and chance levels beside every number, are in `eval/` here. Four of
+the eight tasks (Winogrande, HellaSwag, PIQA, TruthfulQA) are scored by the harness on *validation* — the
+splits this project was allowed to read — and the other four on test.
+
+## Files
+
+| file | sha256 | bytes |
+|---|---|---|
+| `config.json` | be63e8d2dc708f8b… | 723 |
+| `cursor.json` | c5ea3ba9293f3278… | 205 |
+| `generation_config.json` | 1dac20576af54bdd… | 219 |
+| `model.safetensors` | 669ac130a8853cc2… | 424,799,272 |
+| `run_summary.json` | d7789a4e70e296c3… | 968 |
+| `tokenizer.json` | 9ca9acddb6525a19… | 2,104,556 |
+
+`cursor.json` is the data position this checkpoint corresponds to: samples consumed, step, seed, the
+corpus fingerprint and a hash of the visit-order permutation, so the resume can be checked rather than
+trusted.
+
+## Not in this model
+
+No instruction tuning, no RLHF, no chat template — it is a base model, deliberately. It has ~1 B tokens of
+training against the ~300 B of the comparable published reference models, so expect scores at or near
+chance on several tasks; that is the experiment's result, not a defect to be engineered around.
+

docs/phase5/gate5_status.json

@@ -0,0 +1,5 @@
+{
+ "err": null,
+ "repo": "Cion-lab/ounce100m-v1",
+ "rev": "55f23ef5d0e8e6a60adad23bf8a2ffffc6bc7580"
+}

docs/phase5/gate5_stdout.txt

@@ -0,0 +1,5 @@
+clean-room download (anonymous) -> /tmp/cleanroom_ounce100m-v1
+VERIFY tensors 200 params 106,194,240 dtypes ['F32']
+GENERATED: The city of Paris is the of people all the - are a of of world. is part the, and the of world is - and the
+GEN_SHA 8e51e7aeee1ed518
+VERDICT GATE5 load_ok=True generate_ok=True params_ok=True tied_ok=True vocab=49152

docs/phase5/publish_status.json

@@ -0,0 +1,17 @@
+{
+ "err": null,
+ "files": [
+  ".gitattributes",
+  "README.md",
+  "config.json",
+  "cursor.json",
+  "generation_config.json",
+  "model.safetensors",
+  "run_summary.json",
+  "special_tokens_map.json",
+  "tokenizer.json",
+  "tokenizer_config.json",
+  "training_args.bin"
+ ],
+ "rev": "55f23ef5d0e8e6a60adad23bf8a2ffffc6bc7580"
+}

docs/phase5/publish_stdout.txt

@@ -0,0 +1,30 @@
+ckpt latest.json -> {'step': 3814, 'path_in_repo': 'final'}
+tokenizer source for config files: HuggingFaceTB/SmolLM2-135M
+tokenizer config tokenizer_config.json 3658 bytes sha256 4bb9af56a342753d
+tokenizer config special_tokens_map.json 831 bytes sha256 e786b595b9a23148
+tensors 200 params 106,194,240 dtypes ['F32']
+card: 106,194,240 params from the safetensors header; 8 files hashed
+uploading 8 of 10 files to Cion-lab/ounce100m-v1
+PUBLISH_JSON_BEGIN
+{
+ "repo": "Cion-lab/ounce100m-v1",
+ "params": 106194240,
+ "tensors": 200,
+ "files": 11,
+ "missing": [],
+ "anon_readback": {
+  "README.md": true,
+  "config.json": true,
+  "cursor.json": true,
+  "generation_config.json": true,
+  "model.safetensors": true,
+  "run_summary.json": true,
+  "special_tokens_map.json": true,
+  "tokenizer.json": true,
+  "tokenizer_config.json": true,
+  "training_args.bin": true
+ },
+ "published_verified": true
+}
+PUBLISH_JSON_END
+NEXT (Gate 5, on a fresh instance with an empty cache): python build/publish_model.py --repo Cion-lab/ounce100m-v1 --verify-only

docs/phase5/status.json

@@ -0,0 +1,11 @@
+{
+ "err": null,
+ "argv": [
+  "publish_model.py",
+  "--repo",
+  "Cion-lab/ounce100m-v1",
+  "--src",
+  "/kaggle/working/mixroot",
+  "--dry-run"
+ ]
+}