Hugging Face's logo

ViuAI
/
Viu-1.5B-MoE

Text Generation
PyTorch
Hindi
English
Mixture of Experts
mixture-of-experts
indic
hindi
hinglish
mla
deepseek-v3
gemma-2
Model card Files
xet
 Community
Viu-1.5B-MoE

Commit History

30 days ago

  1. Optimize train_kaggle_t4.yaml: micro-batch 8 / accum 16 (8 passes/GPU on Dual T4), log_every 1

    ViuAI
    Verified signature
    0914e9d

  2. Optimize train_h100_h200.yaml: batch 64/accum 2 (65k tok/step), grad_ckpt false (~2-3s per step)

    ViuAI
    Verified signature
    053fcb6

  3. Optimize train_rtx5090.yaml: micro-batch 32 / grad_accum 4 (~5-7s per step, 9k-12k tok/s)

    ViuAI
    Verified signature
    1e7d75d

  1. Optimize train_rtx4090.yaml: micro-batch 32 / grad_accum 4 (~10-12s per step, 5.5k-6.5k tok/s)

    ViuAI
    Verified signature
    fbc4414

  2. Upgrade RTX 3090 profile: micro-batch 32 / grad_accum 4 (~25s per step, 2.6k-3k tok/s)

    ViuAI
    Verified signature
    55fd252

  3. Vectorize MoE dispatch (argsort+bincount): eliminate 2,688 CUDA kernel launches per pass

    ViuAI
    Verified signature
    43847d6

  4. Add live elapsed time counter (fmt_duration) and step pace (s/step, step/h) to console logs

    ViuAI
    Verified signature
    d695257

  5. Optimize train_rtx3090.yaml: grad_accum 8 (80s/step), log_every 1, fast 5k refill

    ViuAI
    Verified signature
    e7de3c4

  6. Optimize chunk_size to 2048 on 24GB+ GPUs (cuts head projections from 64 to 8 per micro-batch)

    ViuAI
    Verified signature
    5b09a76

  7. Auto-inject HF write token into RUN_ON_CLOUD.ipynb & record Section 19 directive in NOTES.md

    ViuAI
    Verified signature
    cd23ed0

  8. Update NOTES.md: Section 18 RTX 3090 Max Speed, MoE Muon-AdamW Hybrid & 16-Batch 32-Accum

    ViuAI
    Verified signature
    bb2bafd

  9. Fix MoE Muon lag: route 4032 sparse expert weights to PagedAdamW8bit (eliminates 68,000 matmuls per step, 15x faster)

    ViuAI
    Verified signature
    2790f5f

  10. RTX 3090: Upgrade Stage 1 micro-batch to 16, grad_accum to 32 (2x GEMM saturation, 32 passes/step)

    ViuAI
    Verified signature
    bd90cfc

  11. RTX 3090 Max Speed: Zero-sync GPU loss accum, pinned-memory batch slicing, Muon in VRAM, 262k tok/step curriculum

    ViuAI
    Verified signature
    e130703

  12. RTX 3090 Max Speed: Zero-sync GPU loss accum, pinned-memory batch slicing, Muon in VRAM, 262k tok/step curriculum

    ViuAI
    Verified signature
    129fae5

  13. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    ViuAI
    Verified signature
    c50d39e

  14. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    ViuAI
    Verified signature
    006553b

  15. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    ViuAI
    Verified signature
    2ec913e

  16. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    ViuAI
    Verified signature
    1c68440

  17. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    ViuAI
    Verified signature
    c99cb28

  18. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    ViuAI
    Verified signature
    e50906a

  19. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    ViuAI
    Verified signature
    efb1c64

  20. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    ViuAI
    Verified signature
    41a0763

  21. Sync model/configs/train_kaggle_t4.yaml after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    4594117

  22. Sync model/configs/train_rtx5090.yaml after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    8bb4dc0

  23. Sync model/configs/train_rtx4090.yaml after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    a5bfa2e

  24. Sync model/scripts/train.py after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    7437570

  25. Sync docs/PROGRESS.md after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    9966752

  26. Sync NOTES.md after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    67f6bed

  27. Sync model/configs/train_rtx5090.yaml after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    662becf

  28. Sync model/configs/train_rtx4090.yaml after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    d5f3bb1

  29. Sync model/scripts/train.py after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    eba5797

  30. Sync model/configs/train_rtx5090.yaml after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    0fd7bb8

  31. Sync model/configs/train_rtx4090.yaml after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    06317af

  32. Sync model/scripts/viu_moe.py after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    3099578

  33. Sync docs/PROGRESS.md after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    3356389

  34. Sync NOTES.md after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    6898af7

  35. Sync model/configs/model_config.yaml after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    12ccb8d

  36. Sync model/scripts/viu_moe.py after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    59d275d

  37. Sync docs/PROGRESS.md after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    f2d9d4a

  38. Sync NOTES.md after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    8da8a9c

  39. Sync model/configs/model_config.yaml after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    36f3761

  40. Sync model/scripts/viu_moe.py after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    a19de2c

  41. Sync model/scripts/train.py after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    90f2273

  1. Sync docs/PROGRESS.md after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    a6f7c7c

  2. Sync NOTES.md after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    70430d3

  3. Sync data/scripts/ingest_real_cricket_corpus.py after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    fff1f7a

  4. Sync data/scripts/ingest_real_bharat_history_pib.py after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    ea9f543

  5. Sync data/scripts/ingest_cinema_mythology_finance.py after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    af87603

  6. Sync model/scripts/verify_arch.py after full codebase audit, retry hardening, and token security fixes

    ViuAI
    Verified signature
    da7f1bf