-
feat: update RTX 5090 notebook runner and optimized config
-
full ckpt step_1000 (resume)
-
config: set Stage 1 micro 32 accum 4 for peak Tensor Core speed
-
config: set micro 16 accum 8 with grad_ckpt true for safe high-throughput pretraining
-
fix: restore grad_ckpt and safe micro 8 accum 8 for zero-OOM guarantee on T4
-
perf: disable grad_ckpt on T4 and enlarge loss chunk_size to 2048 for 10k+ tok/s
-
config: update Stage 1 batch_size to 16 for micro=16 accum=8 per GPU
-
refactor: consolidate pure 32L Dense architecture, purge MoE files, audit hardening
-
Update model/configs/train_kaggle_t4.yaml: log_every: 10 and real-time step streaming
-
Remove dead config model/configs/model_config.yaml
-
Remove dead config model/configs/viu_1_5b_moe_config.yaml
-
Remove dead config model/configs/viu1_moe_config.yaml
-
Archive model_config.yaml to deprecated folder
-
Archive viu_1_5b_moe_config.yaml to deprecated folder
-
Archive viu1_moe_config.yaml to deprecated folder
-
Fix model/scripts/train.py: batch math comments and find_unused_parameters=False
-
Fix model/configs/train_config.yaml: batch math comments and find_unused_parameters=False
-
Audit fix: Update model/scripts/train.py for ViuMini-Dense-360M
-
Audit fix: Update model/configs/viu_1_5b_moe_config.yaml for ViuMini-Dense-360M
-
Audit fix: Update model/configs/viu1_moe_config.yaml for ViuMini-Dense-360M
-
Audit fix: Update model/configs/model_config.yaml for ViuMini-Dense-360M
-
Audit fix: Update model/configs/train_dense_baseline.yaml for ViuMini-Dense-360M
-
Audit fix: Update model/configs/train_h100_h200.yaml for ViuMini-Dense-360M
-
Audit fix: Update model/configs/train_rtx5090.yaml for ViuMini-Dense-360M
-
Audit fix: Update model/configs/train_rtx4090.yaml for ViuMini-Dense-360M
-
Audit fix: Update model/configs/train_rtx3090.yaml for ViuMini-Dense-360M
-
Audit fix: Update model/configs/config.json for ViuMini-Dense-360M
-
Audit fix: Update model/configs/viu1_dense_config.yaml for ViuMini-Dense-360M
-
Audit fix: Update model/NOTES.md for ViuMini-Dense-360M
-
Update model/scripts/push_to_hf.py for ViuMini-Dense-360M
-
Update model/scripts/train.py for ViuMini-Dense-360M
-
Update model/configs/train_rtx5090.yaml for ViuMini-Dense-360M
-
Update model/configs/train_rtx4090.yaml for ViuMini-Dense-360M
-
Update model/configs/train_rtx3090.yaml for ViuMini-Dense-360M
-
Update model/configs/train_dense_baseline.yaml for ViuMini-Dense-360M
-
Update model/configs/train_config.yaml for ViuMini-Dense-360M
-
Update model/configs/train_kaggle_t4.yaml for ViuMini-Dense-360M
-
Update config.json for 32 layers, dim 960
-
Set Option A: 32 layers, dim 960 (366.6M params, SmolLM-style)
-
Upgrade architecture to ViuMini-Dense-350M (100% Dense, 360M active)
-
Upgrade architecture to ViuMini-Dense-350M (100% Dense, 360M active)
-
Upgrade architecture to ViuMini-Dense-350M (100% Dense, 360M active)
-
Upgrade architecture to ViuMini-Dense-350M (100% Dense, 360M active)
-
Set checkpoint save, eval, sample, and HF push intervals to 1000 steps
-
Remove diverged model/checkpoints/step_3000.pt for clean AdamW start
-
Remove diverged model/checkpoints/step_2500.pt for clean AdamW start
-
Remove diverged model/checkpoints/step_2000.pt for clean AdamW start
-
Switch to PagedAdamW8bit (lr=2.5e-4, warmup=1000) for 100% stable convergence
-
prune old ckpt model/checkpoints/step_1500.pt
-
full ckpt step_3000 (resume)