ViuAI
/

ViuMini-MoE-242M

Text Generation

Mixture of Experts

mixture-of-experts

Model card Files Files and versions

ViuMini-MoE-242M / model

2.9 GB

Ctrl+K

Ctrl+K

2 contributors

History: 76 commits

ViuAI's picture

Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

a30bdca verified 21 minutes ago

checkpoints
full ckpt step_2000 (resume) 7 days ago
configs
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm 33 minutes ago
scripts
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm 21 minutes ago
NOTES.md

2.07 kB
feat: SDPA FlashAttention, language tags conditioning, probe safety, slim checkpoints, and streaming clean 14 days ago