Commit History

  1. fix: restore grad_ckpt and safe micro 8 accum 8 for zero-OOM guarantee on T4

    Verified signature
    96be75f
  2. perf: disable grad_ckpt on T4 and enlarge loss chunk_size to 2048 for 10k+ tok/s

    Verified signature
    7d4c3f5
  3. config: update Stage 1 batch_size to 16 for micro=16 accum=8 per GPU

    Verified signature
    20ebd23
  4. refactor: consolidate pure 32L Dense architecture, purge MoE files, audit hardening

    Verified signature
    3a5d3e5
  5. Fix model/scripts/train.py: batch math comments and find_unused_parameters=False

    Verified signature
    ef65129
  6. Audit fix: Update model/scripts/train.py for ViuMini-Dense-360M

    Verified signature
    e52f2b4
  7. Update model/scripts/push_to_hf.py for ViuMini-Dense-360M

    Verified signature
    d9702de
  8. Update model/scripts/train.py for ViuMini-Dense-360M

    Verified signature
    dedfdc7
  9. Upgrade architecture to ViuMini-Dense-350M (100% Dense, 360M active)

    Verified signature
    94036c7

  1. Restore find_unused_parameters=True: prevent MoE DDP reducer freeze

    Verified signature
    97157cf
  2. Speed optimization: set DDP find_unused_parameters=False

    Verified signature
    fb9d88c
  3. Add UTF-8 stdout support to train.py

    Verified signature
    660f01e
  4. P0 fix: ensure train.py targets viu1_moe 242M with updated defaults

    Verified signature
    f83bc48
  5. P0 fix: gradient checkpointing, chunked cross-entropy, and vectorized MoE dispatch

    Verified signature
    3c7f390

  1. Vectorize MoE dispatch (argsort+bincount): eliminate 2,688 CUDA kernel launches per pass

    Verified signature
    4fa70d5
  2. Add live elapsed time counter (fmt_duration) and step pace (s/step, step/h) to console logs

    Verified signature
    da6a53b
  3. Optimize chunk_size to 2048 on 24GB+ GPUs (cuts head projections from 64 to 8 per micro-batch)

    Verified signature
    a8f7b02
  4. Fix MoE Muon lag: route 4032 sparse expert weights to PagedAdamW8bit (eliminates 68,000 matmuls per step, 15x faster)

    Verified signature
    10f435d
  5. RTX 3090 Max Speed: Zero-sync GPU loss accum, pinned-memory batch slicing, Muon in VRAM, 262k tok/step curriculum

    Verified signature
    fe763fa
  6. RTX 3090 Max Speed: Zero-sync GPU loss accum, pinned-memory batch slicing, Muon in VRAM, 262k tok/step curriculum

    Verified signature
    063b0de
  7. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    Verified signature
    f3fe7af
  8. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    Verified signature
    e1a12dc
  9. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    Verified signature
    f5cccb7
  10. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    Verified signature
    61b414e
  11. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    Verified signature
    a30bdca
  12. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    Verified signature
    9d0f9ca
  13. Sync model/scripts/viu_moe.py with 3-stage sequence length curriculum (512->1024->2048)

    Verified signature
    7e89caa
  14. Sync model/scripts/train.py with 3-stage sequence length curriculum (512->1024->2048)

    Verified signature
    e100ee7

  1. Sync model/scripts/train.py with adult erotica stories priority integration

    Verified signature
    35480b2
  2. Sync model/scripts/train.py with adult erotica stories priority integration

    Verified signature
    9d0207c
  3. feat: step-1 priority queue integration for real Indian history & cricket shards

    Verified signature
    26fdf05
  4. feat: 100% zero-exclusion streaming architecture (2,081 shards) and updated docs

    Verified signature
    a8f8457

  1. Add 42-layer Viu-1.5B-MoE verification test suite

    Verified signature
    fa717d7
  2. perf: free logits immediately in forward pass to release 3.1GB VRAM for mt_loss

    Verified signature
    79836a1
  3. feat: add CLI overrides and manual batch lock support

    Verified signature
    f16d6ac
  4. fix: auto-download tokenizer.json if LFS pointer detected

    Verified signature
    b8da5cc
  5. fix: mute DDP barrier and find_unused_parameters stderr warnings

    Verified signature
    e2b2d3d
  6. feat: add RTX 5090 & 4090 configs (model/scripts/train.py)

    Verified signature
    8fadd89
  7. feat: smart auto-download Hugging Face checkpoint resume

    Verified signature
    d83067a

  1. fix: add --hf_split argument and remove invalid attribute access in streaming logger

    Verified signature
    b690d1d
  2. fix: pass token to list_repo_files and HfFileSystem to eliminate unauthenticated warnings

    Verified signature
    d3005b5
  3. feat: non-blocking asynchronous checkpoint upload and local disk pruning

    Verified signature
    46b3eaf
  4. Add --hf_safety argument and default full

    Verified signature
    a485427
  5. Replace load_dataset with direct PyArrow streaming generator: Eliminate CastError permanently

    Verified signature
    f1beed2
  6. Clean pretraining logging: Streamlined single-line logs (loss, lr, speed, ETA) with zero console clutter

    Verified signature
    848dc17
  7. Fix CastError permanently: Direct parquet loader via hf:// paths + clean step 1-20 logging with pbar.write

    Verified signature
    dc649d2
  8. Fix CastError permanently: exclude code/ prefix + print every step for first 10 steps

    Verified signature
    eee7156
  9. Speed optimizations: no_sync DDP communication + cuDNN/TF32 + 10k ckpt sync

    Verified signature
    58dee8c
  10. Enhance live training logs: print at step 1 and show all 8 MoE expert percentages

    Verified signature
    88658cf
  11. Update model/scripts/train.py: Exclude distilled/ from streaming + set log_every to 10

    Verified signature
    d75aebc