Text Generation
Transformers
Safetensors
Sanskrit
sansar
sanskrit
devanagari
slp1
causal-lm
base-model
custom_code
Eval Results (legacy)
Instructions to use MuseMesh/sansar-20m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MuseMesh/sansar-20m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="MuseMesh/sansar-20m", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("MuseMesh/sansar-20m", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use MuseMesh/sansar-20m with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "MuseMesh/sansar-20m" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MuseMesh/sansar-20m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/MuseMesh/sansar-20m
- SGLang
How to use MuseMesh/sansar-20m with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "MuseMesh/sansar-20m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MuseMesh/sansar-20m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "MuseMesh/sansar-20m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MuseMesh/sansar-20m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use MuseMesh/sansar-20m with Docker Model Runner:
docker model run hf.co/MuseMesh/sansar-20m
Model card: add Sansar 700M to the comparison table
Browse files
README.md
CHANGED
|
@@ -12,7 +12,6 @@ tags:
|
|
| 12 |
- base-model
|
| 13 |
- sansar
|
| 14 |
datasets:
|
| 15 |
-
- MuseMesh/sansar-sanskrit-corpus
|
| 16 |
- ai4bharat/sangraha
|
| 17 |
- chronbmm/sanskrit-monolingual-pretraining
|
| 18 |
model-index:
|
|
@@ -22,31 +21,30 @@ model-index:
|
|
| 22 |
type: text-generation
|
| 23 |
name: Sanskrit language modelling
|
| 24 |
dataset:
|
| 25 |
-
name: Sansar E0 held-out Sanskrit sets
|
| 26 |
type: sansar-e0-heldout
|
|
|
|
| 27 |
metrics:
|
| 28 |
- type: bits_per_byte
|
|
|
|
| 29 |
value: 0.7177
|
| 30 |
-
name: bits per Devanagari byte, pooled excluding the Bhagavad-gītā (E0 held-out
|
| 31 |
-
sets)
|
| 32 |
- type: bits_per_byte
|
|
|
|
| 33 |
value: 0.7102
|
| 34 |
-
name: bits per Devanagari byte, pooled over all five E0 sets
|
| 35 |
- type: bits_per_byte
|
|
|
|
| 36 |
value: 0.7065
|
| 37 |
-
name: bits per Devanagari byte, dcs_gold
|
| 38 |
- type: bits_per_byte
|
|
|
|
| 39 |
value: 0.6887
|
| 40 |
-
name: bits per Devanagari byte, prose
|
| 41 |
- type: bits_per_byte
|
|
|
|
| 42 |
value: 0.7225
|
| 43 |
-
name: bits per Devanagari byte, ood
|
| 44 |
- type: bits_per_byte
|
|
|
|
| 45 |
value: 0.9685
|
| 46 |
-
name: bits per Devanagari byte, vedic
|
| 47 |
- type: bits_per_byte
|
|
|
|
| 48 |
value: 0.5129
|
| 49 |
-
name: bits per Devanagari byte, gita
|
| 50 |
---
|
| 51 |
|
| 52 |
# Sansar 20M
|
|
@@ -162,6 +160,7 @@ Same metric and sets. External models were scored zero-shot with their own token
|
|
| 162 |
| [Sansar 125M](https://huggingface.co/MuseMesh/sansar-125m) v0.1.0 | 97.2M | 0.6039 | 0.6254 | F6-clean-2x, 2.66B tokens |
|
| 163 |
| [Sansar 350M](https://huggingface.co/MuseMesh/sansar-350m) v0.1.0 | 318.4M | 0.5720 | 0.5937 | F7, 2.66B tokens |
|
| 164 |
| [Sansar 350M](https://huggingface.co/MuseMesh/sansar-350m) v0.2.0 | 318.4M | 0.5547 | 0.5773 | F9, 5.09B tokens |
|
|
|
|
| 165 |
| Krutrim-2-instruct (zero-shot, nf4) | 12B | 0.512 | n/a | general model, 2,048-token window |
|
| 166 |
| Sarvam-1 (zero-shot, fp16) | 2.5B | 0.750 | n/a | general model, 2,048-token window |
|
| 167 |
| Gemma 4 E2B (zero-shot, fp16) | 4B | 0.892 | n/a | general model, 2,048-token window |
|
|
|
|
| 12 |
- base-model
|
| 13 |
- sansar
|
| 14 |
datasets:
|
|
|
|
| 15 |
- ai4bharat/sangraha
|
| 16 |
- chronbmm/sanskrit-monolingual-pretraining
|
| 17 |
model-index:
|
|
|
|
| 21 |
type: text-generation
|
| 22 |
name: Sanskrit language modelling
|
| 23 |
dataset:
|
|
|
|
| 24 |
type: sansar-e0-heldout
|
| 25 |
+
name: Sansar E0 held-out Sanskrit sets
|
| 26 |
metrics:
|
| 27 |
- type: bits_per_byte
|
| 28 |
+
name: "bits per Devanagari byte, pooled excluding the Bhagavad-gītā (E0 held-out sets)"
|
| 29 |
value: 0.7177
|
|
|
|
|
|
|
| 30 |
- type: bits_per_byte
|
| 31 |
+
name: "bits per Devanagari byte, pooled over all five E0 sets"
|
| 32 |
value: 0.7102
|
|
|
|
| 33 |
- type: bits_per_byte
|
| 34 |
+
name: "bits per Devanagari byte, dcs_gold"
|
| 35 |
value: 0.7065
|
|
|
|
| 36 |
- type: bits_per_byte
|
| 37 |
+
name: "bits per Devanagari byte, prose"
|
| 38 |
value: 0.6887
|
|
|
|
| 39 |
- type: bits_per_byte
|
| 40 |
+
name: "bits per Devanagari byte, ood"
|
| 41 |
value: 0.7225
|
|
|
|
| 42 |
- type: bits_per_byte
|
| 43 |
+
name: "bits per Devanagari byte, vedic"
|
| 44 |
value: 0.9685
|
|
|
|
| 45 |
- type: bits_per_byte
|
| 46 |
+
name: "bits per Devanagari byte, gita"
|
| 47 |
value: 0.5129
|
|
|
|
| 48 |
---
|
| 49 |
|
| 50 |
# Sansar 20M
|
|
|
|
| 160 |
| [Sansar 125M](https://huggingface.co/MuseMesh/sansar-125m) v0.1.0 | 97.2M | 0.6039 | 0.6254 | F6-clean-2x, 2.66B tokens |
|
| 161 |
| [Sansar 350M](https://huggingface.co/MuseMesh/sansar-350m) v0.1.0 | 318.4M | 0.5720 | 0.5937 | F7, 2.66B tokens |
|
| 162 |
| [Sansar 350M](https://huggingface.co/MuseMesh/sansar-350m) v0.2.0 | 318.4M | 0.5547 | 0.5773 | F9, 5.09B tokens |
|
| 163 |
+
| [Sansar 700M](https://huggingface.co/MuseMesh/sansar-700m) v0.1.0 | 704.1M | 0.5307 | 0.5541 | F10, 4.76B tokens |
|
| 164 |
| Krutrim-2-instruct (zero-shot, nf4) | 12B | 0.512 | n/a | general model, 2,048-token window |
|
| 165 |
| Sarvam-1 (zero-shot, fp16) | 2.5B | 0.750 | n/a | general model, 2,048-token window |
|
| 166 |
| Gemma 4 E2B (zero-shot, fp16) | 4B | 0.892 | n/a | general model, 2,048-token window |
|