ViuAI
/

ViuMini-MoE-242M

Text Generation

Mixture of Experts

mixture-of-experts

Model card Files Files and versions

ViuMini-MoE-242M

2.96 GB

Ctrl+K

Ctrl+K

2 contributors

History: 222 commits

ViuAI's picture

RTX 3090 Max Speed: Zero-sync GPU loss accum, pinned-memory batch slicing, Muon in VRAM, 262k tok/step curriculum

063b0de verified about 2 hours ago

data
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm about 5 hours ago
docs
Sync docs/PROGRESS.md with massive Mythology and Bollywood Cinema corpora expansion about 7 hours ago
logs
sync log step_2000 7 days ago
model
RTX 3090 Max Speed: Zero-sync GPU loss accum, pinned-memory batch slicing, Muon in VRAM, 262k tok/step curriculum about 2 hours ago
tokenizer
fix: DDP auto_tune probe-before-wrap + broadcast micro/accum, R0-only logs, empty-shard guard, safety import path 8 days ago
.gitattributes

290 Bytes
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm about 5 hours ago
.gitignore

91 Bytes
fix: remove model/checkpoints/*.pt from .gitignore so Hugging Face commits checkpoints 7 days ago
NOTES.md

38 kB
Sync NOTES.md with massive Mythology and Bollywood Cinema corpora expansion about 7 hours ago
README.md

2.9 kB
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm about 5 hours ago
RUN_ON_CLOUD.ipynb

11.9 kB
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm about 4 hours ago
basic_arithmetic_bodmas_tables.parquet

678 kB
xet

docs & scripts: add full-scale 5-domain milestone (145.93GB/105B tokens) 10 days ago
requirements.txt

245 Bytes
init: 28L MoE 241M audit-fixed, smoke pass 16 days ago