CoRover/BharatGPT-mini

Model Description

BharatGPT mini is a Transformer-based language model pretrained on a large corpus of publicly available text data using a self-supervised learning approach. This means the model was trained without any human-labeled annotations—learning directly from raw text using an automatic mechanism to generate training signals.

During pretraining, BharatGPT mini was optimized for the causal language modeling task: given a sequence of tokens, the model learns to predict the next token in the sequence. More specifically, it takes a sequence of continuous text as input and is trained to predict the next word or subword by shifting the target sequence one position to the right. A masking mechanism ensures that predictions for token i are based only on tokens from positions 1 to i, without peeking at future tokens. This preserves the autoregressive nature of language modeling.

Through this training process, BharatGPT mini develops a deep internal understanding of language patterns, grammar, and semantics. While it can be fine-tuned for various downstream tasks such as classification, summarization, or question answering, it performs best in text generation tasks, which align with its original training objective.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo_id_or_path = "CoRover/BharatGPT-mini"

tok = AutoTokenizer.from_pretrained(repo_id_or_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo_id_or_path, dtype=torch.bfloat16, trust_remote_code=True)
model.eval()

def ask(question, max_new_tokens=150, do_sample=False):
    messages = [{"role": "user", "content": question}]
    prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tok(prompt, return_tensors="pt").to(model.device)
    with torch.no_grad():
        output = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            do_sample=do_sample,
            repetition_penalty=1.1,
            eos_token_id=tok.eos_token_id,
            pad_token_id=tok.eos_token_id,
        )
    return tok.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)

print("Q: तुम्हें किसने बनाया?")
print("A:", ask("तुम्हें किसने बनाया?"))
print()

print("Q: What is CoRover?")
print("A:", ask("What is CoRover?"))

It is best suited for S-RAG (Secure Retrieval-Augmented Generation) or fine-tuning with your own data. For enhanced performance, integration with Conversational Agentic AI platform is recommended (though not mandatory). This platform enables the creation of multi-modal and multi-lingual AI Agents, Co-Pilots, and Virtual Assistants (such as ChatBots, VoiceBots, and VideoBots) using a sovereign AI and composite AI approach. It leverages classic NLP, grounded generative AI with BharatGPT, and Generally Available LLMs to deliver powerful, versatile AI solutions.

Multilingual MMLU Benchmarks

BharatGPT-Instruct evaluated across Indian languages against Qwen-1.7B and Sarvam-30B.

Language	BharatGPT-mini	BharatGPT-Instruct	Qwen-1.7B	Sarvam-30B
Hindi	23.1	41.59	34.9	32.1
Bengali	23.9	39.22	32.3	27.4
Marathi	23.4	39.40	31.7	29.2
Telugu	23.6	38.44	30.7	25.9
Gujarati	23.0	38.38	32.7	32.1
Malayalam	23.2	37.96	30.9	26.3
Punjabi	23.7	37.58	31.5	25.8
Tamil	23.5	37.30	30.9	30.1
Odia	23.9	34.82	29.8	30.3
Kannada	23.8	38.30	31.3	29.4

RAG Evaluation

This is the most relevant comparison for our actual product surface-enterprise AI-Assistant grounded in a customer knowledge base. BharatGPT-Instruct scores 100.00 on Faithfulness, essentially tied with GPT-4o-mini and Qwen 3 1.7B, and matches both on Top-K Accuracy (%).

RAG Metric	BharatGPT-mini	BharatGPT-Instruct	GPT-4o-mini	Qwen 3 1.7B
Faithfulness	80.09	100.00	95.00	95.51
Top-K Accuracy	100	100	100	100
Relevance	70.95	91.07	92.12	95.76
Recall	94.1	94.12	93.07	92.51

Benchmark Results

BharatGPT models were evaluated on a diverse and comprehensive set of benchmark datasets and evaluation metrics to measure their performance across multiple dimensions of language understanding, reasoning, and text generation. The reported results cover a broad range of tasks, including knowledge-based question answering, reasoning, comprehension, and safty and Truthfulness.

Benchmark	BharatGPT-mini	BharatGPT-3B-Indic	BharatGPT-Instruct E2B	Qwen-1.7B	Sarvam-30B
General Knowledge
MMLU	23.60	52.90	57.56	55.50	45.60
AGIEval	28.32	30.40	32.57	40.20	30.70
Commonsense Reasoning
HellaSwag	29.93	67.60	55.59	46.10	51.90
PIQA	62.08	75.70	78.51	72.40	62.80
WinoGrande	51.46	64.80	68.67	60.80	50.90
Reading Comprehension
BoolQ	60.98	78.70	78.13	77.40	76.80
Science Reasoning
ARC-Easy	52.78	73.40	83.21	72.60	44.80
ARC-Challenge	21.33	42.90	53.84	39.70	33.30
Math Reasoning
GSM8K	1.00	31.40	23.50	68.00	70.60
Safety & Truthfulness
TruthfulQA	18.20	44.70	38.00	50.30	64.80
ToxiGen	46.38	52.60	41.70	42.00	55.40
Fairness & Bias
WinoGender	50.83	56.70	60.28	56.30	51.00
CrowS-Pairs	54.26	54.00	68.28	55.20	55.20
BBQ	38.40	79.80	55.07	38.30	59.40

Usage and Limitations

• License: Non-Commercial. For academic and research purposes only. For commercial use, please visit Conversational Agentic AI Platform or Contact Us.

• Terms of Use: Terms and Conditions

• PIPEE -Responsible AI Framework:

  • P - Purpose: Risk depends on context.
  • I - Inclusivity: As the Hon'ble PM of India says, "AI means All Inclusive" - it must be human-centric and multi-modal - Text, Voice, and Video.
  • P - Privacy: Where data is stored and inferred.
  • E - Explainability: Can we audit the decision?
  • E - Energy Efficiency: Just because someone has money for GPUs doesn't mean we should waste energy.

• Developed by: CoRover.ai