Org card: Sanskrit-first language models, built in the open; sansar-700m v0.2.0; Sansar OCR
Browse files
README.md
CHANGED
|
@@ -9,15 +9,15 @@ pinned: false
|
|
| 9 |
|
| 10 |
# Muse Mesh
|
| 11 |
|
| 12 |
-
We build
|
| 13 |
|
| 14 |
-
## Sansar: Sanskrit-
|
| 15 |
|
| 16 |
-
**Sansar** is a family of
|
| 17 |
|
| 18 |
| Model | Parameters | Bits per byte (held-out, ex-Gītā) ↓ |
|
| 19 |
|---|---|---|
|
| 20 |
-
| [sansar-700m](https://huggingface.co/MuseMesh/sansar-700m) | 704M | **0.
|
| 21 |
| [sansar-350m](https://huggingface.co/MuseMesh/sansar-350m) | 318M | 0.5547 |
|
| 22 |
| [sansar-125m](https://huggingface.co/MuseMesh/sansar-125m) | 97M | 0.6039 |
|
| 23 |
| [sansar-60m](https://huggingface.co/MuseMesh/sansar-60m) | 63M | 0.6434 |
|
|
@@ -35,6 +35,10 @@ ids = tok("संस्कृतं नाम दैवी वाक्", return
|
|
| 35 |
print(tok.decode(model.generate(**ids, max_new_tokens=60, do_sample=True, top_k=100)[0]))
|
| 36 |
```
|
| 37 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
## English and Math models
|
| 39 |
|
| 40 |
Small models trained from scratch with the same recipe (Muon, rotary positions, QK-norm, squared ReLU) and a shared 32k tokenizer, released with their evaluation sets and the exact list of training documents. [Collection](https://huggingface.co/collections/MuseMesh/mume-english-and-math-language-models-6ac4f187f33abd6c9a9cdde6).
|
|
|
|
| 9 |
|
| 10 |
# Muse Mesh
|
| 11 |
|
| 12 |
+
We build **Sanskrit-first language models, in the open**: corpora, tokenizers and models trained from scratch, starting with Sanskrit. Every training run we do, finished or live, with its score and cost, is published at **[saansar.com](https://saansar.com)**.
|
| 13 |
|
| 14 |
+
## Sansar: Sanskrit-first language models
|
| 15 |
|
| 16 |
+
**Sansar** is a family of causal language models trained from scratch, Sanskrit first. The models released so far are trained on Sanskrit text only, with a Sanskrit tokenizer of their own; none is a fine-tune of an English model. Other languages come later on the same recipe.
|
| 17 |
|
| 18 |
| Model | Parameters | Bits per byte (held-out, ex-Gītā) ↓ |
|
| 19 |
|---|---|---|
|
| 20 |
+
| [sansar-700m](https://huggingface.co/MuseMesh/sansar-700m) (v0.2.0) | 704M | **0.5262** |
|
| 21 |
| [sansar-350m](https://huggingface.co/MuseMesh/sansar-350m) | 318M | 0.5547 |
|
| 22 |
| [sansar-125m](https://huggingface.co/MuseMesh/sansar-125m) | 97M | 0.6039 |
|
| 23 |
| [sansar-60m](https://huggingface.co/MuseMesh/sansar-60m) | 63M | 0.6434 |
|
|
|
|
| 35 |
print(tok.decode(model.generate(**ids, max_new_tokens=60, do_sample=True, top_k=100)[0]))
|
| 36 |
```
|
| 37 |
|
| 38 |
+
## Sansar OCR
|
| 39 |
+
|
| 40 |
+
**[sansar-ocr-2b](https://huggingface.co/MuseMesh/sansar-ocr-2b)** reads printed Sanskrit (Devanagari) pages: a 2B vision-language model fine-tuned from Qwen3.5-2B on clean labels only. Median letter error on 54 held-out pages: **1.05%**, against 1.69% for Google Cloud Vision and 9.03% for archive.org's OCR. It runs on a single consumer GPU.
|
| 41 |
+
|
| 42 |
## English and Math models
|
| 43 |
|
| 44 |
Small models trained from scratch with the same recipe (Muon, rotary positions, QK-norm, squared ReLU) and a shared 32k tokenizer, released with their evaluation sets and the exact list of training documents. [Collection](https://huggingface.co/collections/MuseMesh/mume-english-and-math-language-models-6ac4f187f33abd6c9a9cdde6).
|