BharatGPT-Instruct

BharatGPT-Instruct E2B is a multilingual Large Language model developed by CoRover.ai, trained to generate high-quality text across major Indic languages. The model is pretrained on a diverse, curated corpus spanning Hindi, Punjabi, Marathi, Malayalam, Odia, Kannada, Gujarati, Bengali, Urdu, Tamil, and Telugu, drawing on authentic Indian conversational data to capture the linguistic and cultural nuances of each language.

Usage

Load and use the model directly via the Hugging Face transformers library.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "CoRover/BharatGPT-Instruct"

tokenizer = AutoTokenizer.from_pretrained(
    MODEL_ID,
    token=HF_TOKEN,
    fix_mistral_regex=True,
    trust_remote_code=True
)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    token=HF_TOKEN,
    dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

model.eval()

torch.backends.cudnn.benchmark = True


def generate_response(prompt, max_new_tokens=256):

    messages = [
        {
            "role": "user",
            "content": prompt
        }
    ]

    encoded = tokenizer.apply_chat_template(
        messages,
        tokenize=True,
        add_generation_prompt=True,
        return_tensors="pt",
        return_dict=True,
    ).to(model.device)

    with torch.inference_mode():

        output = model.generate(
            input_ids=encoded["input_ids"],
            attention_mask=encoded["attention_mask"],
            max_new_tokens=max_new_tokens,
            do_sample=False,
            use_cache=True,
            pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id,
        )

    generated_tokens = output[0][
        encoded["input_ids"].shape[1]:
    ]

    return tokenizer.decode(
        generated_tokens,
        skip_special_tokens=True
    ).strip()


print(generate_response("Who developed you?"))

Multilingual MMLU Benchmarks

BharatGPT-Instruct evaluated across Indian languages against Qwen-1.7B and Sarvam-30B.

Language	BharatGPT-mini	BharatGPT-Instruct	Qwen-1.7B	Sarvam-30B
Hindi	23.1	41.59	34.9	32.1
Bengali	23.9	39.22	32.3	27.4
Marathi	23.4	39.40	31.7	29.2
Telugu	23.6	38.44	30.7	25.9
Gujarati	23.0	38.38	32.7	32.1
Malayalam	23.2	37.96	30.9	26.3
Punjabi	23.7	37.58	31.5	25.8
Tamil	23.5	37.30	30.9	30.1
Odia	23.9	34.82	29.8	30.3
Kannada	23.8	38.30	31.3	29.4

RAG Evaluation

This is the most relevant comparison for our actual product surface-enterprise AI-Assistant grounded in a customer knowledge base. BharatGPT-Instruct scores 100.00 on Faithfulness, essentially tied with GPT-4o-mini and Qwen 3 1.7B, and matches both on Top-K Accuracy (%).

RAG Metric	BharatGPT-mini	BharatGPT-Instruct	GPT-4o-mini	Qwen 3 1.7B
Faithfulness	80.09	100.00	95.00	95.51
Top-K Accuracy	100	100	100	100
Relevance	70.95	91.07	92.12	95.76
Recall	94.1	94.12	93.07	92.51

Benchmark Results

BharatGPT models were evaluated on a diverse and comprehensive set of benchmark datasets and evaluation metrics to measure their performance across multiple dimensions of language understanding, reasoning, and text generation. The reported results cover a broad range of tasks, including knowledge-based question answering, reasoning, comprehension, and Safety and Truthfulness.

Benchmark	BharatGPT-mini	BharatGPT-3B-Indic	BharatGPT-Instruct E2B	Qwen-1.7B	Sarvam-30B
General Knowledge
MMLU	23.60	52.90	57.56	55.50	45.60
AGIEval	28.32	30.40	32.57	40.20	30.70
Commonsense Reasoning
HellaSwag	29.93	67.60	55.59	46.10	51.90
PIQA	62.08	75.70	78.51	72.40	62.80
WinoGrande	51.46	64.80	68.67	60.80	50.90
Reading Comprehension
BoolQ	60.98	78.70	78.13	77.40	76.80
Science Reasoning
ARC-Easy	52.78	73.40	83.21	72.60	44.80
ARC-Challenge	21.33	42.90	53.84	39.70	33.30
Math Reasoning
GSM8K	1.00	31.40	23.50	68.00	70.60
Safety & Truthfulness
TruthfulQA	18.20	44.70	38.00	50.30	64.80
ToxiGen	46.38	52.60	41.70	42.00	55.40
Fairness & Bias
WinoGender	50.83	56.70	60.28	56.30	51.00
CrowS-Pairs	54.26	54.00	68.28	55.20	55.20
BBQ	38.40	79.80	55.07	38.30	59.40

Usage and Limitations

• License: Non-Commercial. For academic and research purposes only. For commercial use, please visit Conversational Agentic AI Platform or Contact Us.

• Terms of Use: Terms and Conditions

• PIPEE -Responsible AI Framework:

  • P - Purpose: Risk depends on context.
  • I - Inclusivity: As the Hon'ble PM of India says, "AI means All Inclusive" - it must be human-centric and multi-modal - Text, Voice, and Video.
  • P - Privacy: Where data is stored and inferred.
  • E - Explainability: Can we audit the decision?
  • E - Energy Efficiency: Just because someone has money for GPUs doesn't mean we should waste energy.