-
fix: restore grad_ckpt and safe micro 8 accum 8 for zero-OOM guarantee on T4
-
perf: disable grad_ckpt on T4 and enlarge loss chunk_size to 2048 for 10k+ tok/s
-
config: update Stage 1 batch_size to 16 for micro=16 accum=8 per GPU
-
refactor: consolidate pure 32L Dense architecture, purge MoE files, audit hardening
-
Fix model/scripts/train.py: batch math comments and find_unused_parameters=False
-
Audit fix: Update model/scripts/train.py for ViuMini-Dense-360M
-
Update model/scripts/push_to_hf.py for ViuMini-Dense-360M
-
Update model/scripts/train.py for ViuMini-Dense-360M
-
Upgrade architecture to ViuMini-Dense-350M (100% Dense, 360M active)
-
Restore find_unused_parameters=True: prevent MoE DDP reducer freeze
-
Speed optimization: set DDP find_unused_parameters=False
-
Add UTF-8 stdout support to train.py
-
P0 fix: ensure train.py targets viu1_moe 242M with updated defaults
-
P0 fix: gradient checkpointing, chunked cross-entropy, and vectorized MoE dispatch
-
Vectorize MoE dispatch (argsort+bincount): eliminate 2,688 CUDA kernel launches per pass
-
Add live elapsed time counter (fmt_duration) and step pace (s/step, step/h) to console logs
-
Optimize chunk_size to 2048 on 24GB+ GPUs (cuts head projections from 64 to 8 per micro-batch)
-
Fix MoE Muon lag: route 4032 sparse expert weights to PagedAdamW8bit (eliminates 68,000 matmuls per step, 15x faster)
-
RTX 3090 Max Speed: Zero-sync GPU loss accum, pinned-memory batch slicing, Muon in VRAM, 262k tok/step curriculum
-
RTX 3090 Max Speed: Zero-sync GPU loss accum, pinned-memory batch slicing, Muon in VRAM, 262k tok/step curriculum
-
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm
-
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm
-
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm
-
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm
-
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm
-
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm
-
Sync model/scripts/viu_moe.py with 3-stage sequence length curriculum (512->1024->2048)
-
Sync model/scripts/train.py with 3-stage sequence length curriculum (512->1024->2048)
-
Sync model/scripts/train.py with adult erotica stories priority integration
-
Sync model/scripts/train.py with adult erotica stories priority integration
-
feat: step-1 priority queue integration for real Indian history & cricket shards
-
feat: 100% zero-exclusion streaming architecture (2,081 shards) and updated docs
-
Add 42-layer Viu-1.5B-MoE verification test suite
-
perf: free logits immediately in forward pass to release 3.1GB VRAM for mt_loss
-
feat: add CLI overrides and manual batch lock support
-
fix: auto-download tokenizer.json if LFS pointer detected
-
fix: mute DDP barrier and find_unused_parameters stderr warnings
-
feat: add RTX 5090 & 4090 configs (model/scripts/train.py)
-
feat: smart auto-download Hugging Face checkpoint resume
-
fix: add --hf_split argument and remove invalid attribute access in streaming logger
-
fix: pass token to list_repo_files and HfFileSystem to eliminate unauthenticated warnings
-
feat: non-blocking asynchronous checkpoint upload and local disk pruning
-
Add --hf_safety argument and default full
-
Replace load_dataset with direct PyArrow streaming generator: Eliminate CastError permanently
-
Clean pretraining logging: Streamlined single-line logs (loss, lr, speed, ETA) with zero console clutter
-
Fix CastError permanently: Direct parquet loader via hf:// paths + clean step 1-20 logging with pbar.write
-
Fix CastError permanently: exclude code/ prefix + print every step for first 10 steps
-
Speed optimizations: no_sync DDP communication + cuDNN/TF32 + 10k ckpt sync
-
Enhance live training logs: print at step 1 and show all 8 MoE expert percentages
-
Update model/scripts/train.py: Exclude distilled/ from streaming + set log_every to 10