urdu-286m-instruct
Instruction-tuned version of urdu-286m. It answers questions in Urdu, in the chat format, from a base model that was pretrained from random initialisation on 2.85B tokens of Urdu text.
It has no reliable knowledge. That is stated here rather than buried, because the benchmark below says so.
What changed from the base model
| base | instruct | |
|---|---|---|
| Behaviour | continues text in news register | answers a question in the chat format |
| Validation bpb | 0.6113 on raw text | 0.4050 on held-out instruction data |
The two bpb numbers are not comparable. The base model's is next-token loss on raw documents, the instruct model's is loss on held-out instruction pairs, which are shorter and easier.
Training data, all public, nothing generated
| Source | Rows used | Licence |
|---|---|---|
large-traversaal/urdu-instruct |
49,275 | CC-BY-SA-4.0 |
CohereLabs/aya_collection_language_split, urdu, filtered to 80-900 chars |
27,909 | Apache-2.0 |
large-traversaal/commonsenseqa_urdu_final, four options kept |
9,737 | unstated upstream |
large-traversaal/openbookqa_urdu_final |
4,957 | unstated upstream |
Total 90,878 rows, about 5.7M tokens. A 5,000-row holdout from the same sources is used for validation and is never trained on.
Aya's Urdu split is mostly translated long-context question answering, median 2,219 characters per row, so only the short-answer band is used. CommonsenseQA ships five options, so four are subsampled with the correct answer kept, which makes every multiple choice item four-option.
Training
Instruction tuning only, from the published base checkpoint at step 4,200.
| Setting | Value |
|---|---|
| Steps | 101, one pass |
| Batch | 65,536 tokens |
| Sequence length | 1,024 |
| Learning rate | 0.8 of the pretraining value |
| Validation bpb | 0.4050 |
Two passes over four passes were tested and rejected: with four passes the validation bpb rose from 0.3488 to 0.4077 while training loss fell, which is overfitting on 5.7M tokens.
What it does and does not do
Works: answers in Urdu, stays on topic, produces a paragraph when asked for one, keeps the answer short when the question is short. 19 of 19 sampled prompts returned real Urdu text, median 123 characters.
Does not work: facts. It says Pakistan beat India in 2011 in Lahore, it says it will snow in Lahore. It cannot hold a long conversation, because the training data is single-turn. Multiple choice answers collapse to a letter prior, because the model has no knowledge to retrieve, so the letter it picks follows corpus frequency rather than the question.
The honest summary: this is a 286M parameter model that learned the shape of an Urdu assistant, without the knowledge that would make the answers true.
Samples
Six prompts from a sheet of 33 across ten task types. Answers are verbatim, including the wrong ones.
Definitions work. Prompt جمہوریت کیا ہے؟
جمہوریت ایک ایسی حکومت یا ادارہ ہے جس میں ہر شخص کو اپنے خیالات کا حق حاصل ہے، بشمول سیاسی، عسکری اور سماجی۔
Prompt فوٹو سنتھیسس کیا ہے؟
فوٹو سنتھیسس سورج کی روشنی کو توڑ کر پانی میں موجود کاربن ڈائی آکسائیڈ کو جذب کرکے زمین کو آکسیجن فراہم کرتا ہے۔
Some facts land. Prompt پاکستان کا قومی پھول کیا ہے؟
پاکستان کا قومی پھول چنبیلی ہے۔
Structured writing is the strongest thing here. Prompt پرنسپل کو چھٹی کی درخواست لکھیں۔
محترم پرنسپل صاحب، میں اپنی بیماری کی وجہ سے پچھلے دو ہفتوں سے چھٹی پر جا رہا ہوں۔ برا کرم میری درخواست پر غور فرمائیں۔ شکریہ۔
Arithmetic shows the limit precisely. The format is perfect, the sum is not. Prompt
2 اور 2 کا مجموعہ کتنا ہے؟
Reasoning: 2 کا مجموعہ: 2 + 2 = 4 2 کا مجموعہ: 4 + 2 = 7 Answer: 7
And on a two-turn exchange it gives the wrong city, then corrects itself when challenged.
First turn پاکستان کا سب سے بڑا شہر کون سا ہے؟
پاکستان کا سب سے بڑا شہر لاہور ہے۔
Then کراچی ہے یا لاہور؟
کراچی پاکستان کا سب سے بڑا شہر ہے۔
The full sheet is in samples.md in this repository: 33 prompts, 39 answers including the
follow-ups, none empty, median 107 characters.
How to use
The weights are in the nanochat checkpoint layout, so they load with the same code as the base model:
from nanochat.checkpoint_manager import load_model
# NANOCHAT_BASE_DIR points at this repository's directory
model, tokenizer, meta = load_model("sft", device, phase="eval",
model_tag="urdu_286m_instruct")
ids, _ = tokenizer.render_conversation(
{"messages": [{"role": "user", "content": "پاکستان کا سب سے بڑا شہر کون سا ہے؟"}]})
ids.append(tokenizer.encode_special("<|assistant_start|>"))
The <|assistant_start|> token matters. render_conversation stops after the user turn,
and generation has to be seeded with the assistant token or the model continues from a
position training never covered.
There is no GGUF. nanochat's architecture uses value embeddings, per-layer residual lambdas and a logit cap, and no llama.cpp implementation exists for it.
Licence
Weights: CC-BY-NC-4.0. Free for research, evaluation and personal use, with attribution. Commercial use requires a licence, available on request at me@awahab.dev.
The training data carries its own terms and they sit underneath this licence:
urdu-instruct is CC-BY-SA-4.0, Aya is Apache-2.0, and the two MCQ translations state no
licence, so their terms are inherited from CommonsenseQA and OpenBookQA upstream.
Citation
@misc{urdu286m_instruct,
title = {urdu-286m-instruct: an Urdu instruction-tuned model trained from scratch},
author = {Abdul Wahab},
year = {2026},
note = {https://huggingface.co/awahab111/urdu-286m-instruct}
}
Model tree for awahab111/urdu-286m-instruct
Base model
awahab111/urdu-286m