Instructions to use arunjmisra2273/OdiaSLM-100M-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use arunjmisra2273/OdiaSLM-100M-Base with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="arunjmisra2273/OdiaSLM-100M-Base")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("arunjmisra2273/OdiaSLM-100M-Base") model = AutoModelForCausalLM.from_pretrained("arunjmisra2273/OdiaSLM-100M-Base", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use arunjmisra2273/OdiaSLM-100M-Base with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "arunjmisra2273/OdiaSLM-100M-Base" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "arunjmisra2273/OdiaSLM-100M-Base", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/arunjmisra2273/OdiaSLM-100M-Base
- SGLang
How to use arunjmisra2273/OdiaSLM-100M-Base with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "arunjmisra2273/OdiaSLM-100M-Base" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "arunjmisra2273/OdiaSLM-100M-Base", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "arunjmisra2273/OdiaSLM-100M-Base" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "arunjmisra2273/OdiaSLM-100M-Base", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use arunjmisra2273/OdiaSLM-100M-Base with Docker Model Runner:
docker model run hf.co/arunjmisra2273/OdiaSLM-100M-Base
- OdiaSLM-100M-Base: Native Odia Small Language Model (100,685,568 (100.69M) Parameters)
- 1. Model Architecture Specification
- 2. Training Telemetry & Convergence Figures
- 3. Intrinsic Evaluation: Domain-Stratified Perplexity & Bits-Per-Byte (
BPB) - 4. Zero-Shot Downstream Evaluation (
saidutta69/odia-eval-benchmark) - 5. Quickstart: Python (
transformers+sentencepiece) - 6. Dual License (
CC-BY-NC-4.0+ Custom Commercial License) - 7. Citation (BibTeX)
- 1. Model Architecture Specification
OdiaSLM-100M-Base: Native Odia Small Language Model (100,685,568 (100.69M) Parameters)
OdiaSLM-100M-Base is a from-scratch decoder-only Small Language Model (SLM) pre-trained specifically for the Odia language (or, ଓଡ଼ିଆ) on 1.57 Billion tokens drawn from a strictly deduplicated and benchmark-decontaminated corpus of 9,619,050 clean documents (1.247 Billion words).
Built on a modern Llama-style architecture (Grouped-Query Attention, SwiGLU, Half-Split RoPE, Pre-Norm FP32-Variance RMSNorm, Tied Input/Output Embeddings, and Auxiliary Z-Loss), OdiaSLM-100M-Base loads natively in Hugging Face transformers (LlamaForCausalLM), vLLM, TGI, and llama.cpp with trust_remote_code=False.
Official Research Paper (DOI): 10.5281/zenodo.24167433 | Download Full PDF Paper (OdiaSLM_Preprint_Misra_2026.pdf) | GitHub Repository
1. Model Architecture Specification
| Hyperparameter | Value |
|---|---|
| Model Variant | OdiaSLM-100M-Base |
| Architecture Class | LlamaForCausalLM (trust_remote_code=False) |
| Total Trainable Parameters | 100,685,568 (100.69M) (Tied Embeddings) |
| Transformer Layers ($L$) | 14 |
| Hidden Dimension ($d_{model}$) | 768 |
| Attention Heads ($H_Q$ / $H_{KV}$) | 12 Query / 4 KV (GQA) |
| Head Dimension ($d_{head}$) | 64 (Half-Split RoPE, $\theta = 10,000$) |
| SwiGLU Intermediate Size ($d_{ff}$) | 2048 |
| Vocabulary Size ($V$) | 16,384 (Odia SentencePiece BPE + Byte Fallback) |
| Tokenizer Fertility (Odia) | 1.4809 tokens/word (11.5727 UTF-8 bytes/token, 0 <unk>) |
| Context Window | 1,024 tokens (~690 Odia words) |
| Pretraining Tokens Processed | 1,572,864,000 (1.57B) |
| Best Held-Out Validation Loss (CE) | 2.8417 |
2. Training Telemetry & Convergence Figures
3. Intrinsic Evaluation: Domain-Stratified Perplexity & Bits-Per-Byte (BPB)
| Validation Domain | Cross-Entropy (CE) | Perplexity (PPL) | Bits-Per-Byte (BPB) | Top-1 Token Acc | Top-5 Token Acc |
|---|---|---|---|---|---|
sangraha:verified_ori |
3.0754 | 21.66 | 0.3750 | 40.46% | 62.87% |
v2:odia_web_corpus_v5 |
2.9469 | 19.05 | 0.3531 | 42.42% | 64.94% |
v2:varta |
2.8242 | 16.85 | 0.3238 | 43.86% | 66.74% |
v2:indiccorp_v2 |
3.0930 | 22.04 | 0.3841 | 40.94% | 62.43% |
v2:indic_nlp_corp |
2.9978 | 20.04 | 0.3916 | 43.17% | 63.34% |
anchor:fineweb_edu_en |
3.0579 | 21.28 | 1.3085 | 41.20% | 62.13% |
4. Zero-Shot Downstream Evaluation (saidutta69/odia-eval-benchmark)
- Pure-Odia 18-Task Macro Average:
60.83%(PMI = 59.86%,+34.44%lift over26.39%random baseline) - All 27 Sub-Benchmarks Macro Average:
56.83%(+30.01%lift over26.82%random baseline)
| Benchmark Task | Script Mode | Random Chance | Raw Acc | Byte-Norm Acc | PMI Calibrated Acc | Lift Over Chance |
|---|---|---|---|---|---|---|
indic_xparaphrase (classification) |
pure_odia |
25.0% | 93.8% | 95.0% | 100.0% | +75.0% |
flores_plus (translation) |
cross_lingual_en_choices |
25.0% | 36.2% | 37.5% | 96.2% | +71.2% |
indic_headline_gen (generation) |
pure_odia |
25.0% | 53.8% | 57.5% | 93.8% | +68.8% |
indic_qa (qa) |
pure_odia |
25.0% | 73.8% | 55.0% | 93.8% | +68.8% |
odia_gsm8k (math) |
pure_odia |
25.0% | 63.8% | 32.5% | 93.8% | +68.8% |
indic_quest_odia (qa) |
pure_odia |
25.0% | 27.5% | 38.8% | 87.5% | +62.5% |
indic_sentence_summarization (generation) |
pure_odia |
25.0% | 51.2% | 51.2% | 86.2% | +61.2% |
odia_truthfulqa (qa) |
pure_odia |
25.0% | 46.2% | 62.5% | 86.2% | +61.2% |
indic_question_generation (generation) |
pure_odia |
25.0% | 62.5% | 57.5% | 83.8% | +58.8% |
bhram_il (qa) |
cross_lingual_en_choices |
25.0% | 32.5% | 38.8% | 71.2% | +46.2% |
sarvam_gsm8k_indic (math) |
cross_lingual_en_choices |
25.0% | 36.2% | 33.8% | 68.8% | +43.8% |
indic_squad_odia (qa) |
pure_odia |
25.0% | 27.5% | 28.8% | 53.8% | +28.8% |
indic_glue_wstp (multiple_choice) |
pure_odia |
25.0% | 43.8% | 28.8% | 46.2% | +21.2% |
odia_truthfulqa_mc (multiple_choice) |
cross_lingual_en_choices |
24.1% | 31.2% | 36.2% | 30.0% | +12.1% |
indic_copa (multiple_choice) |
pure_odia |
50.0% | 57.5% | 58.8% | 46.2% | +8.8% |
5. Quickstart: Python (transformers + sentencepiece)
import sentencepiece as spm
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoModelForCausalLM
repo_id = "arunjmisra2273/OdiaSLM-100M-Base"
tok_path = hf_hub_download(repo_id=repo_id, filename="tokenizer.model")
sp = spm.SentencePieceProcessor(model_file=tok_path)
model = AutoModelForCausalLM.from_pretrained(
repo_id,
dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
trust_remote_code=False,
)
model.eval()
prompt = "ଓଡ଼ିଶାର ରାଜଧାନୀ ଭୁବନେଶ୍ୱର ଏକ"
input_ids = torch.tensor([[1] + sp.encode(prompt, out_type=int)], dtype=torch.long)
with torch.no_grad():
out_ids = model.generate(
input_ids,
max_new_tokens=64,
do_sample=True,
temperature=0.7,
top_p=0.9,
eos_token_id=2,
pad_token_id=0,
)
print(sp.decode([int(t) for t in out_ids[0].tolist() if int(t) not in (0, 1, 2)]))
6. Dual License (CC-BY-NC-4.0 + Custom Commercial License)
- Code: Licensed under Apache License 2.0.
- Model Weights & Tokenizer: Licensed under CC-BY-NC-4.0 for academic, research, and non-commercial use. Commercial deployments require a separate Commercial License from the author (
arunjmisra2273). SeeLICENSE.mdfor details.
7. Citation (BibTeX)
@article{misra2026odiaslm,
title={OdiaSLM: From-Scratch Pretraining, Morphology-Aware Tokenization, and Calibrated Evaluation of 60M and 100M Small Language Models for Odia},
author={Misra, Arun},
year={2026},
month={November},
publisher={Zenodo},
doi={10.5281/zenodo.24167433},
url={https://doi.org/10.5281/zenodo.24167433}
}
- Downloads last month
- 59










