ks2812 commited on
Commit
ec507fd
·
verified ·
1 Parent(s): cc6a6d1

Org card: Sanskrit-first language models, built in the open; sansar-700m v0.2.0; Sansar OCR

Browse files
Files changed (1) hide show
  1. README.md +8 -4
README.md CHANGED
@@ -9,15 +9,15 @@ pinned: false
9
 
10
  # Muse Mesh
11
 
12
- We build language technology for **Sanskrit** and other Indic languages: corpora, tokenizers and language models trained from scratch. Every training run we do, finished or live, is published at the **[Sansar Lab](https://saansar.com)**.
13
 
14
- ## Sansar: Sanskrit-only language models
15
 
16
- **Sansar** is a family of small causal language models trained only on Sanskrit text, with a Sanskrit tokenizer of its own. Each model is trained from scratch; none is a fine-tune of an English model.
17
 
18
  | Model | Parameters | Bits per byte (held-out, ex-Gītā) ↓ |
19
  |---|---|---|
20
- | [sansar-700m](https://huggingface.co/MuseMesh/sansar-700m) | 704M | **0.5307** |
21
  | [sansar-350m](https://huggingface.co/MuseMesh/sansar-350m) | 318M | 0.5547 |
22
  | [sansar-125m](https://huggingface.co/MuseMesh/sansar-125m) | 97M | 0.6039 |
23
  | [sansar-60m](https://huggingface.co/MuseMesh/sansar-60m) | 63M | 0.6434 |
@@ -35,6 +35,10 @@ ids = tok("संस्कृतं नाम दैवी वाक्", return
35
  print(tok.decode(model.generate(**ids, max_new_tokens=60, do_sample=True, top_k=100)[0]))
36
  ```
37
 
 
 
 
 
38
  ## English and Math models
39
 
40
  Small models trained from scratch with the same recipe (Muon, rotary positions, QK-norm, squared ReLU) and a shared 32k tokenizer, released with their evaluation sets and the exact list of training documents. [Collection](https://huggingface.co/collections/MuseMesh/mume-english-and-math-language-models-6ac4f187f33abd6c9a9cdde6).
 
9
 
10
  # Muse Mesh
11
 
12
+ We build **Sanskrit-first language models, in the open**: corpora, tokenizers and models trained from scratch, starting with Sanskrit. Every training run we do, finished or live, with its score and cost, is published at **[saansar.com](https://saansar.com)**.
13
 
14
+ ## Sansar: Sanskrit-first language models
15
 
16
+ **Sansar** is a family of causal language models trained from scratch, Sanskrit first. The models released so far are trained on Sanskrit text only, with a Sanskrit tokenizer of their own; none is a fine-tune of an English model. Other languages come later on the same recipe.
17
 
18
  | Model | Parameters | Bits per byte (held-out, ex-Gītā) ↓ |
19
  |---|---|---|
20
+ | [sansar-700m](https://huggingface.co/MuseMesh/sansar-700m) (v0.2.0) | 704M | **0.5262** |
21
  | [sansar-350m](https://huggingface.co/MuseMesh/sansar-350m) | 318M | 0.5547 |
22
  | [sansar-125m](https://huggingface.co/MuseMesh/sansar-125m) | 97M | 0.6039 |
23
  | [sansar-60m](https://huggingface.co/MuseMesh/sansar-60m) | 63M | 0.6434 |
 
35
  print(tok.decode(model.generate(**ids, max_new_tokens=60, do_sample=True, top_k=100)[0]))
36
  ```
37
 
38
+ ## Sansar OCR
39
+
40
+ **[sansar-ocr-2b](https://huggingface.co/MuseMesh/sansar-ocr-2b)** reads printed Sanskrit (Devanagari) pages: a 2B vision-language model fine-tuned from Qwen3.5-2B on clean labels only. Median letter error on 54 held-out pages: **1.05%**, against 1.69% for Google Cloud Vision and 9.03% for archive.org's OCR. It runs on a single consumer GPU.
41
+
42
  ## English and Math models
43
 
44
  Small models trained from scratch with the same recipe (Muon, rotary positions, QK-norm, squared ReLU) and a shared 32k tokenizer, released with their evaluation sets and the exact list of training documents. [Collection](https://huggingface.co/collections/MuseMesh/mume-english-and-math-language-models-6ac4f187f33abd6c9a9cdde6).