SomaliDatasets/maay-maxaa-translation
Updated • 1
🔄 In a Training Loop
ayoub adan abdi
Ayoubadanabdi
AI & ML interests
AI and ML
Recent Activity
liked a dataset 37 minutes ago
SomaliDatasets/maay-maxaa-translation updated a dataset 44 minutes ago
SomaliDatasets/maay-maxaa-translation published a dataset about 1 hour ago
SomaliDatasets/maay-maxaa-translationOrganizations
liked a
dataset 37 minutes ago
updated a
dataset 44 minutes ago
published a
dataset about 1 hour ago
updated a
dataset 8 days ago
liked a
dataset 8 days ago
published a
dataset 8 days ago
liked a
Space 29 days ago
updated a
Space 30 days ago
published a
Space 30 days ago
published a
Space about 1 month ago
updated a
model about 2 months ago
published a
model about 2 months ago
updated a
model about 2 months ago
published a
dataset about 2 months ago
published a
model about 2 months ago
liked a
dataset 3 months ago
reacted to omarkamali's post with ❤️ 4 months ago
Post
4603
We got Qwen 3.5 to count Rs in Strawberry correctly! 🚨
Building on Sawtone, we’ve been testing a different way to feed language into an LLM to build the next generation of multilingual AI.
The usual setup gives the model tokenized text and asks it to perform various linguistic tasks. That works surprisingly well, until it doesn’t. Accents disappear. Words get mangled. Internal structure gets blurred away. And the cost of that gets higher once you move into multilingual and lower-resource settings.
So we tried adding a second path.
In addition to the normal text input, the model also receives Sawtone: a byte-level word representation that preserves how a word is written, how it sounds, and how it is structured.
Same LLM. Better interface.
In this proof of concept with Qwen 3.5 0.8B, that pushed our eval from 64% to 88%. The gains showed up exactly where tokenized models usually get shaky: diacritics, character order, exact spelling, and other form-sensitive behavior.
Sawtone itself is tokenizer-free, byte-level, and pre-trained across 507 languages.
Still early, but promising!
Building on Sawtone, we’ve been testing a different way to feed language into an LLM to build the next generation of multilingual AI.
The usual setup gives the model tokenized text and asks it to perform various linguistic tasks. That works surprisingly well, until it doesn’t. Accents disappear. Words get mangled. Internal structure gets blurred away. And the cost of that gets higher once you move into multilingual and lower-resource settings.
So we tried adding a second path.
In addition to the normal text input, the model also receives Sawtone: a byte-level word representation that preserves how a word is written, how it sounds, and how it is structured.
Same LLM. Better interface.
In this proof of concept with Qwen 3.5 0.8B, that pushed our eval from 64% to 88%. The gains showed up exactly where tokenized models usually get shaky: diacritics, character order, exact spelling, and other form-sensitive behavior.
Sawtone itself is tokenizer-free, byte-level, and pre-trained across 507 languages.
Still early, but promising!
- 5 replies