ViuAI commited on
Commit
82b45f8
·
verified ·
1 Parent(s): 39c9a3c

Sync PROGRESS.md: Scale to Global Batch Size 128

Browse files
Files changed (1) hide show
  1. docs/PROGRESS.md +7 -0
docs/PROGRESS.md CHANGED
@@ -433,3 +433,10 @@ Kaggle par manual file upload se version mismatch hota hai (v1 tokenizer wala ca
433
  - Pretraining now ingests 100% of raw internet data including uncensored dialogues, Hinglish street banter, and slang without filtering.
434
  - Added --hf_safety CLI argument to train.py for runtime override.
435
  - Updated train_config.yaml (commit e7638d93) and train.py (commit a4854271) on ViuAI/ViuMini-MoE-242M.
 
 
 
 
 
 
 
 
433
  - Pretraining now ingests 100% of raw internet data including uncensored dialogues, Hinglish street banter, and slang without filtering.
434
  - Added --hf_safety CLI argument to train.py for runtime override.
435
  - Updated train_config.yaml (commit e7638d93) and train.py (commit a4854271) on ViuAI/ViuMini-MoE-242M.
436
+
437
+ - [x] 2026-09-24 SCALE TO GLOBAL BATCH SIZE 128 (131K TOKENS/STEP, 800K MAX STEPS):
438
+ - Upgraded target effective batch to 64 per GPU (grad_accum: 16, micro: 4) = 128 global sequences across 2x T4 GPUs.
439
+ - Model now processes 131,072 tokens per optimizer step (~131K tokens/step).
440
+ - Scaled total training steps from 6,400,000 to 800,000 steps (~105B tokens / 131,072).
441
+ - Checkpoint sync interval set to 2,000 steps (~6.8 hours / 262M tokens) for Kaggle 12-hour session safety.
442
+ - Verified and pushed live to ViuAI/ViuMini-MoE-242M (commit 39c9a3c2).