Hugging Face's logo

ViuAI
/
ViuMini-Dense-360M

Text Generation
PyTorch
Hindi
English
dense
transformer
indic
hindi
hinglish
english
gqa
gemma-4
Model card Files
xet
 Community
ViuMini-Dense-360M / model /scripts /viu_moe.py

Commit History

29 days ago

  1. Vectorize MoE dispatch (argsort+bincount): eliminate 2,688 CUDA kernel launches per pass

    ViuAI
    Verified signature
    4fa70d5

  2. Optimize chunk_size to 2048 on 24GB+ GPUs (cuts head projections from 64 to 8 per micro-batch)

    ViuAI
    Verified signature
    a8f7b02

  3. RTX 3090 Max Speed: Zero-sync GPU loss accum, pinned-memory batch slicing, Muon in VRAM, 262k tok/step curriculum

    ViuAI
    Verified signature
    063b0de

  4. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    ViuAI
    Verified signature
    f5cccb7

  5. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    ViuAI
    Verified signature
    a30bdca

  6. Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

    ViuAI
    Verified signature
    9d0f9ca

  7. Sync model/scripts/viu_moe.py with 3-stage sequence length curriculum (512->1024->2048)

    ViuAI
    Verified signature
    7e89caa