ViuAI
/

ViuMini-MoE-242M

Text Generation

Mixture of Experts

mixture-of-experts

Model card Files Files and versions

ViuMini-MoE-242M

2.96 GB

Ctrl+K

Ctrl+K

2 contributors

History: 216 commits

ViuAI's picture

Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

1ad4183 verified 4 minutes ago

data
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm 15 minutes ago
docs
Sync docs/PROGRESS.md with massive Mythology and Bollywood Cinema corpora expansion about 3 hours ago
logs
sync log step_2000 7 days ago
model
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm 9 minutes ago
tokenizer
fix: DDP auto_tune probe-before-wrap + broadcast micro/accum, R0-only logs, empty-shard guard, safety import path 8 days ago
.gitattributes

290 Bytes
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm 9 minutes ago
.gitignore

91 Bytes
fix: remove model/checkpoints/*.pt from .gitignore so Hugging Face commits checkpoints 7 days ago
NOTES.md

38 kB
Sync NOTES.md with massive Mythology and Bollywood Cinema corpora expansion about 3 hours ago
README.md

2.9 kB
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm 15 minutes ago
RUN_ON_CLOUD.ipynb

11.9 kB
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm 4 minutes ago
basic_arithmetic_bodmas_tables.parquet

678 kB
xet

docs & scripts: add full-scale 5-domain milestone (145.93GB/105B tokens) 10 days ago
requirements.txt

245 Bytes
init: 28L MoE 241M audit-fixed, smoke pass 16 days ago