Sync docs/PROGRESS.md with massive Mythology and Bollywood Cinema corpora expansion

Sync docs/PROGRESS.md with massive Mythology and Bollywood Cinema corpora expansion

Sync docs/PROGRESS.md with massive Mythology and Bollywood Cinema corpora expansion

Sync docs/PROGRESS.md with massive Mythology and Bollywood Cinema corpora expansion

Sync docs/PROGRESS.md with massive Mythology and Bollywood Cinema corpora expansion

Sync docs/PROGRESS.md with adult erotica stories priority integration

Sync docs/PROGRESS.md with adult erotica stories priority integration

feat: step-1 priority queue integration for real Indian history & cricket shards

feat: 100% zero-exclusion streaming architecture (2,081 shards) and updated docs

feat: 100% zero-exclusion streaming architecture (2,081 shards) and updated docs

feat: 100% zero-exclusion streaming architecture (2,081 shards) and updated docs

Lock Golden Development Protocol in PROGRESS.md

Document Step 1 completion in PROGRESS.md

Document RTX 5090 Step 1120 milestone and Viu-1.5B-MoE 42-layer architecture in PROGRESS.md

docs: record RTX 5090 live pretraining milestone

docs: record tokenizer LFS fix

docs: record hardware pricing ROI and rental notebook runners

feat: add RTX 5090 & 4090 configs (docs/PROGRESS.md)

docs: update PROGRESS.md with smart resume and 1k step interval

docs: update PROGRESS.md with .gitignore checkpoint fix

docs: add hf_split bug resolution and DDP warning explanation

docs: update PROGRESS.md with async checkpoints and HF auth warning fix

feat: non-blocking asynchronous checkpoint upload and local disk pruning

Sync PROGRESS.md: Scale to Global Batch Size 128

Sync PROGRESS.md: Unblock toxic and uncensored data (hf_safety: full)

Sync PROGRESS.md: Native PyArrow streaming engine

Sync PROGRESS.md: Streamlined industry logging update

Sync PROGRESS.md: Permanent CastError resolution + direct parquet loader + step 1-20 logging

Sync PROGRESS.md: CastError permanent resolution + step <= 10 logging

Sync PROGRESS.md: Log full 105B data steps, 10k ckpt, no_sync DDP speedup, and live expert logging

Sync PROGRESS.md: train.py CLI args fix

Sync docs/PROGRESS.md: Romance literature and casual chat corpus boost

feat: wipe-proof HF persistence (full ckpt + live log + newest-2 cleanup)

feat: full-ckpt HF push every 10k steps (resume safety net) + docs

feat: hf_round corpus-slice rotation (Option A full-data: file rotate + row skip + per-round seed)

docs: DDP verify RUN#3 FULL PASS — ready for 50k pretrain

fix: exclude non-5col schema paths from HF stream (wikipedia/grammar/gsm8k CastError on shuffle)

docs: update PROGRESS.md with Sanatan Dharma, Epics, Gita, Ramayana, and Reasoning boost

fix: probe OOM reserve 4GB + expandable_segments + natural-mode stream shuffle (Hindi-only trap)

fix: DDP auto_tune probe-before-wrap + broadcast micro/accum, R0-only logs, empty-shard guard, safety import path

v1 tokenizer removal notes + v2 download instruction in train config

docs: sync PROGRESS.md with full HF audit status and v2 tokenizer benchmarks

sync local: enforce_mix sampler, tokenizer v2 (mask fix + 35/30/35), mix optional docs, repair scripts

refactor: clean repo, fix cross-platform paths, update progress

docs: add official comprehensive data audit report (145.93GB/105.1B tokens - 100% PASS)

docs & scripts: add dedicated grammar stream milestone (145.93GB/105.1B tokens)

docs & scripts: add full-scale 5-domain milestone (145.93GB/105B tokens)

docs & scripts: add dedicated science stream milestone (128.75GB/92B tokens)

docs & scripts: add dictionary, expanded stories, and massive uncensored datasets milestone (117GB/83B tokens)

feat/docs: sync stories & uncensored scripts & docs for docs/PROGRESS.md