29 days ago
-
Optimize train_kaggle_t4.yaml: micro-batch 8 / accum 16 (8 passes/GPU on Dual T4), log_every 1
-
Optimize train_h100_h200.yaml: batch 64/accum 2 (65k tok/step), grad_ckpt false (~2-3s per step)
-
Optimize train_rtx5090.yaml: micro-batch 32 / grad_accum 4 (~5-7s per step, 9k-12k tok/s)
30 days ago
-
Optimize train_rtx4090.yaml: micro-batch 32 / grad_accum 4 (~10-12s per step, 5.5k-6.5k tok/s)
-
Upgrade RTX 3090 profile: micro-batch 32 / grad_accum 4 (~25s per step, 2.6k-3k tok/s)
-
Vectorize MoE dispatch (argsort+bincount): eliminate 2,688 CUDA kernel launches per pass
-
Add live elapsed time counter (fmt_duration) and step pace (s/step, step/h) to console logs
-
Optimize train_rtx3090.yaml: grad_accum 8 (80s/step), log_every 1, fast 5k refill
-
Optimize chunk_size to 2048 on 24GB+ GPUs (cuts head projections from 64 to 8 per micro-batch)
-
Fix MoE Muon lag: route 4032 sparse expert weights to PagedAdamW8bit (eliminates 68,000 matmuls per step, 15x faster)
-
RTX 3090: Upgrade Stage 1 micro-batch to 16, grad_accum to 32 (2x GEMM saturation, 32 passes/step)
-
RTX 3090 Max Speed: Zero-sync GPU loss accum, pinned-memory batch slicing, Muon in VRAM, 262k tok/step curriculum
-
RTX 3090 Max Speed: Zero-sync GPU loss accum, pinned-memory batch slicing, Muon in VRAM, 262k tok/step curriculum
-
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm
-
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm
-
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm
-
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm
-
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm
-
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm
-
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm
-
Sync model/configs/train_kaggle_t4.yaml after full codebase audit, retry hardening, and token security fixes
-
Sync model/configs/train_rtx5090.yaml after full codebase audit, retry hardening, and token security fixes
-
Sync model/configs/train_rtx4090.yaml after full codebase audit, retry hardening, and token security fixes
-
Sync model/scripts/train.py after full codebase audit, retry hardening, and token security fixes
-
Sync model/configs/train_rtx5090.yaml after full codebase audit, retry hardening, and token security fixes
-
Sync model/configs/train_rtx4090.yaml after full codebase audit, retry hardening, and token security fixes
-
Sync model/scripts/train.py after full codebase audit, retry hardening, and token security fixes
-
Sync model/configs/train_rtx5090.yaml after full codebase audit, retry hardening, and token security fixes
-
Sync model/configs/train_rtx4090.yaml after full codebase audit, retry hardening, and token security fixes
-
Sync model/scripts/viu_moe.py after full codebase audit, retry hardening, and token security fixes
-
Sync model/configs/model_config.yaml after full codebase audit, retry hardening, and token security fixes
-
Sync model/scripts/viu_moe.py after full codebase audit, retry hardening, and token security fixes
-
Sync model/configs/model_config.yaml after full codebase audit, retry hardening, and token security fixes
-
Sync model/scripts/viu_moe.py after full codebase audit, retry hardening, and token security fixes
-
Sync model/scripts/train.py after full codebase audit, retry hardening, and token security fixes
-
Sync model/scripts/verify_arch.py after full codebase audit, retry hardening, and token security fixes
-
Sync model/scripts/viu_moe.py after full codebase audit, retry hardening, and token security fixes
-
Sync model/scripts/train.py after full codebase audit, retry hardening, and token security fixes
-
Sync model/scripts/train.py with adult erotica stories priority integration
-
Sync model/scripts/train.py with adult erotica stories priority integration
-
feat: step-1 priority queue integration for real Indian history & cricket shards
-
feat: 100% zero-exclusion streaming architecture (2,081 shards) and updated docs
-
feat: include toxicity & hate_speech tweets via native tweet adapter in pretraining
-
feat: unlock 8.55M vortex instruction rows and add corrupted text filter
-
feat: dynamic streaming buffer refill for continuous 105B pretraining & frozen val benchmark
-
Sync train.py with language tag normalization & category round-robin interleaving
-
Unlock all 1,874 Parquet files including all 33 legal shards (model/scripts/train.py)
-
Unlock 1,841 valid Parquet files in streaming engine (model/scripts/train.py)
-
Unlock 26M Samanantar parallel translation stream in train.py (model/scripts/train.py)
-
Sync full-scale uncapped pretraining dataset updates (model/scripts/train.py)
