shubham105/bhojpuri-sarvam-v1

Bhojpuri Sarvam v1

First open-source fine-tuned LLM for Bhojpuri language.

Bhojpuri is spoken by 50+ million people across UP, Bihar, and global diaspora (Mauritius, Fiji, Trinidad) — yet has near-zero AI representation. This model is the first step toward fixing that.

What it does

• Answers questions in Bhojpuri and Hinglish
• Covers government schemes (PM Kisan, Ration Card, Aadhar)
• Handles romanized Bhojpuri input (how people type on phones)
• Fine-tuned on manually curated UP/Bihar domain data

Training Details

• Base model: sarvamai/sarvam-1 (7B)
• Method: QLoRA fine-tuning (LoRA r=16, alpha=32)
• Trainable params: 24M / 1.43B (1.68%)
• Dataset: 100 manually curated Bhojpuri instruction pairs
• Hardware: Kaggle T4 x2 GPUs (free tier)
• Training loss: 3.61 → 1.55

How to use

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="shubham105/bhojpuri-sarvam-v1",
    max_seq_length=2048,
    load_in_4bit=True,
)

FastLanguageModel.for_inference(model)

messages = [
    {"role": "system", 
     "content": "Tu ek helpful Bhojpuri assistant ba. Hamesha Bhojpuri mein jawab de."},
    {"role": "user", 
     "content": "hamra ration card nahi bana kya kari?"}
]

inputs = tokenizer.apply_chat_template(
    messages, tokenize=True, 
    add_generation_prompt=True, return_tensors="pt"
).to("cuda")

outputs = model.generate(
    input_ids=inputs, max_new_tokens=200,
    temperature=0.7, do_sample=True
)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

Roadmap

• Scale dataset to 500+ examples
• Build first Bhojpuri evaluation benchmark
• Release dataset on HuggingFace + AIKosh
• Submit to arXiv cs.CL
• Add voice interface (Bhojpuri ASR + TTS)

Contribute

Native Bhojpuri speaker? Help improve this model by contributing examples. DM on LinkedIn or open a discussion here.

Author

Shubham Gond — AI/ML researcher working on low-resource Indian language models.

Built with ❤️ for 50 million Bhojpuri speakers who deserve AI too.