ViuAI
/

ViuMini-MoE-242M

Text Generation

Mixture of Experts

mixture-of-experts

Model card Files Files and versions

ViuMini-MoE-242M / data

53.2 MB

Ctrl+K

Ctrl+K

2 contributors

History: 33 commits

ViuAI's picture

Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm

9d0f9ca verified 39 minutes ago

eval
feat: SDPA FlashAttention, language tags conditioning, probe safety, slim checkpoints, and streaming clean 14 days ago
processed
sync local: enforce_mix sampler, tokenizer v2 (mask fix + 35/30/35), mix optional docs, repair scripts 8 days ago
processed_romance_chat
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm 39 minutes ago
processed_spiritual
fix: exclude non-5col schema paths from HF stream (wikipedia/grammar/gsm8k CastError on shuffle) 8 days ago
raw
sync local: enforce_mix sampler, tokenizer v2 (mask fix + 35/30/35), mix optional docs, repair scripts 8 days ago
scripts
Fix CUDA OOM on T4: cast model to amp_dtype directly, batch_size=2, safe RMSNorm 39 minutes ago
NOTES.md

1.83 kB
sync local: enforce_mix sampler, tokenizer v2 (mask fix + 35/30/35), mix optional docs, repair scripts 8 days ago
README.md

3.64 kB
sync local: enforce_mix sampler, tokenizer v2 (mask fix + 35/30/35), mix optional docs, repair scripts 8 days ago
run_auto_tyohar_boost_colab.ipynb

9.93 kB
sync local: enforce_mix sampler, tokenizer v2 (mask fix + 35/30/35), mix optional docs, repair scripts 8 days ago