Refresh to step-24000: weights (Q4/f32), eval report, README, showcase assets (PPL 147.97)

README.md

@@ -18,9 +18,7 @@ pipeline_tag: text-generation
 
 <!--
   NeuralAI — Nae1
-  Model card maintained by De'Andrew Preston Harris (@Subject-Emu-5259)
-  Last updated: 2026-10-08
--->
+  Model card maintained by De'Andrew Preston Harris (@Subject-Emu-5259)  Last updated: 2026-10-08 (step-24000 refresh)-->
 
 <img src="neuralai-nae1-banner.png" alt="NeuralAI Nae1 banner" width="100%" />
 
@@ -34,8 +32,8 @@ pipeline_tag: text-generation
 <p align="center">
   <a href="https://huggingface.co/Subject-Emu-5259/NeuralAI-Nae1"><img src="https://img.shields.io/badge/🤗%20Hub-NeuralAI%20Nae1-blue" alt="Nae1"></a>
   <img src="https://img.shields.io/badge/params-100.1M-22d3ee" alt="100.1M params">
-  <img src="https://img.shields.io/badge/held--out%20PPL-148.84-34d399" alt="PPL 148.84">
-  <img src="https://img.shields.io/badge/steps-20,000-a78bfa" alt="20000 steps">
+  <img src="https://img.shields.io/badge/held--out%20PPL-147.97-34d399" alt="PPL 147.97">
+  <img src="https://img.shields.io/badge/steps-24,000-a78bfa" alt="24000 steps">
   <img src="https://img.shields.io/badge/license-Apache%202.0-orange" alt="Apache 2.0">
 </p>
 
@@ -51,9 +49,9 @@ pipeline_tag: text-generation
 | **Vocabulary** | 439 tokens (ByteLevel BPE, sliced from a 32,000-target config) |
 | **Context** | 2,048 trained · evaluated on `n_ctx=512` |
 | **Training data** | 768M tokens — FineWeb CC-MAIN-2013-20, 400k documents, 8 shards |
-| **Training schedule** | 20,000 steps — five cosine cycles (0→4k→8k→12k→16k→20k) · Kaggle T4 |
-| **Final training loss** | 24.15 @ step 20,000 (22.96 @ step 16,000 → 24.15 @ step 20,000; final-step loss is batch-noisy — held-out PPL decides) |
-| **Held-out perplexity** | **148.84** (Q4_K_M, 992 docs / 1,027,766 tokens, teacher-forced) |
+| **Training schedule** | 24,000 steps — six cosine cycles (0→4k→8k→12k→16k→20k→24k) · Kaggle T4 |
+| **Final training loss** | 24.17 @ step 24,000 (best in cycle 6: 21.81 @ step 21,250; final-step loss is batch-noisy — held-out PPL decides) |
+| **Held-out perplexity** | **147.97** (Q4_K_M, 992 docs / 1,027,766 tokens, teacher-forced) |
 | **Formats in this repo** | `f32` GGUF (305 MB) · `Q4_K_M` GGUF (45 MB) · config + tokenizer |
 | **License** | Apache 2.0 |
 
@@ -62,13 +60,13 @@ pipeline_tag: text-generation
 ## 📈 Training
 
 <p align="center">
-  <img src="neuralai-nae1-training.png" alt="Nae1 training loss step 4000 to 8000" width="92%" />
+  <img src="neuralai-nae1-training.png" alt="Nae1 training loss step 20000 to 24000" width="92%" />
 </p>
 
-Nae1 is pretrained from random initialization in five cosine cycles: 0 → 4,000 · 4,000 → 8,000 · 8,000 → 12,000 · 12,000 → 16,000 · 16,000 → 20,000, each resumed from the previous checkpoint. The chart below shows cycle two — loss falls from ~25.9 to a best of **22.88** (step 7,050), finishing at **24.90** at step 8,000 as the learning rate anneals to 1.2e-11; cycles three and four carry training loss to **22.96** at step 16,000, and cycle five finishes at **24.15** at step 20,000 (batch-noisy final step; held-out PPL keeps improving: 151.45 → 148.84).
+Nae1 is pretrained from random initialization in six cosine cycles: 0 → 4,000 · 4,000 → 8,000 · 8,000 → 12,000 · 12,000 → 16,000 · 16,000 → 20,000 · 20,000 → 24,000, each resumed from the previous checkpoint. The chart shows cycle six — loss drops from ~25.4 to a best of **21.81** (step 21,250), the lowest training loss of the run, finishing at **24.17** at step 24,000 as the learning rate anneals to 1.3e-12 (batch-noisy final step; held-out PPL keeps improving: 151.45 → 148.84 → **147.97**).
 
 - **Data pipeline**: FineWeb parquet → chunked into 513-token windows served as zero-copy views (peak-RAM-safe at 768M tokens)
-- **Checkpoints**: every 250 steps; this release ships the final step-20000 checkpoint
+- **Checkpoints**: every 250 steps; this release ships the final step-24000 checkpoint
 
 ---
 
@@ -88,9 +86,10 @@ Teacher-forced perplexity on a held-out FineWeb slice (992 documents / 1,027,766
 | step-8000 | 165.92 |
 | step-12000 | 154.25 |
 | step-16000 | 151.45 |
-| **step-20000 — this release** | **148.84** |
+| step-20000 | 148.84 |
+| **step-24000 — this release** | **147.97** |
 
-Raw report: [`eval_step20000_q4_n512.json`](eval_step20000_q4_n512.json) — mean NLL 5.0029 nats/token (prior releases kept alongside: [`eval_step16000_q4_n512.json`](eval_step16000_q4_n512.json), [`eval_step8000_q4_n512.json`](eval_step8000_q4_n512.json)). Fixed probe ("The quick brown fox", 10 tokens): 169.63 — the 10-token probe is noisy; held-out decides.
+Raw report: [`eval_step24000_q4_n512.json`](eval_step24000_q4_n512.json) — mean NLL 4.9970 nats/token (prior releases kept alongside: [`eval_step20000_q4_n512.json`](eval_step20000_q4_n512.json), [`eval_step16000_q4_n512.json`](eval_step16000_q4_n512.json), [`eval_step8000_q4_n512.json`](eval_step8000_q4_n512.json)). Fixed probe ("The quick brown fox", 10 tokens): 171.33 — the 10-token probe is noisy; held-out decides.
 
 > ⚠️ Perplexity is the trust signal here, not raw capability. At 100M params over 768M tokens, expect exploratory, often garbled text — this is a research-lineage model, not a chat assistant.
 
@@ -131,7 +130,8 @@ print(out["choices"][0]["text"])
 | `nae1-llama-f32.gguf` | Full-precision GGUF, 305 MB — for requantization / research |
 | `config.json` | Architecture config as trained |
 | `tokenizer/` | Native 439-token vocab + merges + manifest |
-| `eval_step20000_q4_n512.json` | The exact evaluation report cited above |
+| `eval_step24000_q4_n512.json` | The exact evaluation report cited above |
+| `eval_step20000_q4_n512.json` | Prior release (step-20000), kept for lineage |
 | `eval_step16000_q4_n512.json` | Prior release (step-16000), kept for lineage |
 | `eval_step8000_q4_n512.json` | Prior release (step-8000), kept for lineage |
 
@@ -173,7 +173,7 @@ NeuralAI was built from resilience, fatherhood, and the belief that personal com
   title        = {NeuralAI — Nae1},
   year         = {2026},
   url          = {https://huggingface.co/Subject-Emu-5259/NeuralAI-Nae1},
-  version      = {step-20000},
+  version      = {step-24000},
   description  = {A 100M-parameter language model pretrained from scratch on 768M FineWeb tokens}
 }
 ```

eval_step24000_q4_n512.json

@@ -0,0 +1,18 @@
+{
+  "model": "/Users/deandrewharris/Projects/NuralAI-Project-is-stored-on-zo-computer/artifacts/nae1_gguf/nae1-llama-Q4_K_M-step24000.gguf",
+  "context_size": 512,
+  "scoring": "teacher forcing: one llm.eval(chunk) + row j scores token j+1 (first token excluded)",
+  "probe": {
+    "text": "The quick brown fox",
+    "tokens": 10,
+    "total_nll_nats": 51.4357048364431,
+    "perplexity": 171.3263956486749
+  },
+  "heldout": {
+    "documents": 992,
+    "tokens": 1027766,
+    "total_nll_nats": 5135780.6794782,
+    "mean_nll_nats_per_token": 4.997033059546823,
+    "perplexity": 147.97347867241118
+  }
+}

nae1-llama-Q4_K_M.gguf

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:faa24bbbd0757e6fe676bd5b687241f89328b381214cf34e4d721693f3c44b81
+oid sha256:49006a2a05b822a5568474db9b11533e6d20d5de0723a73e06e64d2af7e0c980
 size 45762464

nae1-llama-f32.gguf

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f4aa8e13f8a66931e1f2986392fda1c2cb02b7a8d0b468ac0e2d4824622b695a
+oid sha256:f2d677d6eadf5beee328f3c7946e8811e31847b89c954f4aa4332a1b505b87a1
 size 304778816