SODA models trained on Yodas+Emilia+Nemotron for 500B tokens from scratch
AI & ML interests
multimodal, audio, speech, llms
Recent Activity
Papers
View all PapersIsoFLOP Models trained on Yodas+Emilia+Nemotron from budgets of 3e18 to 3e20
-
soda-research/discrete-audio-isoflop-3e20-4.24B-d2816-L27-B64-b829f0
4B • Updated • 6 -
soda-research/discrete-audio-isoflop-3e20-1.93B-d2048-L20-B128-ca249f
2B • Updated • 7 -
soda-research/discrete-audio-isoflop-3e20-2.62B-d2304-L23-B128-29fe40
3B • Updated • 5 -
soda-research/discrete-audio-isoflop-3e20-1.68B-d1920-L19-B128-a41e32
2B • Updated • 7
Pretraining Discrete Audio Data -- Interleaved Sequence using Mimi (8 codebooks)
SODA models trained on Yodas+Emilia+Nemotron for 500B tokens from scratch
Models from our "Flattened vs. Hierarchical" investigation https://soda-audio.github.io/hierarchical.html
IsoFLOP Models trained on Yodas+Emilia+Nemotron from budgets of 3e18 to 3e20
-
soda-research/discrete-audio-isoflop-3e20-4.24B-d2816-L27-B64-b829f0
4B • Updated • 6 -
soda-research/discrete-audio-isoflop-3e20-1.93B-d2048-L20-B128-ca249f
2B • Updated • 7 -
soda-research/discrete-audio-isoflop-3e20-2.62B-d2304-L23-B128-29fe40
3B • Updated • 5 -
soda-research/discrete-audio-isoflop-3e20-1.68B-d1920-L19-B128-a41e32
2B • Updated • 7
SODA models trained on Yodas+Emilia+Nemotron for 500B tokens with a Qwen3 initialization
Pretraining Discrete Audio Data -- Interleaved Sequence using Mimi (8 codebooks)