You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

DevKoshurSarvam: English-to-Kashmiri Translation

November 15, 2026

An English-to-Kashmiri translator that improves on the shipped model, especially on short spoken sentences, the kind people actually type into a translation box. Everything for the pipeline is in this repository: the translator weights at the top level, the diacritic restorer in restorer/, and the training corpus in corpus/.

Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms.

Results

This release against the shipped model. Both are the translator at 4-bit with the same four-beam decoding and the same restorer; cells are no-diacritic BLEU / KATHE score.

Test set Shipped This release Change
IN22-Conv (1,503) 20.84 / 17.40 21.36 / 17.99 +0.52 / +0.59
IN22-Gen (1,024) 21.47 / 19.19 21.43 / 19.28 −0.04 / +0.09
Voice test (555) 11.30 / 9.96 19.84 / 13.58 +8.54 / +3.62

IN22-Conv is conversational written Kashmiri and the closest of the two IN22 sets to how people write. The voice test is spoken Kashmiri. The large gain there is the point of this release: the shipped model was trained on written text and could not handle short spoken sentences.

No-diacritic BLEU comes entirely from the translator. KATHE adds the diacritics back on top.

Training recipe

Start from Sarvam-Translate 4B. Fine-tune it twice, with the diacritics removed from every Kashmiri target, then average the two checkpoints with equal weights:

  1. Main — 107,302 cleaned human pairs (BPCC human seed, NLLB-Seed and BPCC-Daily), about two passes. Learning rate 1e-5, label smoothing 0.1, 16,384 tokens per update, checkpoint picked on held-out no-diacritic BLEU.
  2. Voice — the same main pairs plus 41,819 cleaned spoken pairs from IndicVoices. Each pair is written three ways on the English side (the original and two rewordings), with the Kashmiri unchanged, for 447,363 rows, one pass.

At inference: four beams, length penalty 0.7, then the diacritic restorer.

The corpus is in corpus/ in this repository, with the cleaning described there.

How to run

pip install torch transformers safetensors huggingface_hub

The translator needs a GPU with at least 16 GB of memory in bfloat16; the restorer also runs on CPU.

1. Translate with the training prompt and four-beam decoding:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "Jibby1729/DevKoshurSarvam"
tok = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForCausalLM.from_pretrained(REPO, dtype=torch.bfloat16).to("cuda").eval()

messages = [{"role": "system", "content": "Translate the text below to Kashmiri."},
            {"role": "user", "content": "I'll run this by you tomorrow."}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt",
                              return_dict=True)["input_ids"].to("cuda")
out = model.generate(ids, num_beams=4, length_penalty=0.7, early_stopping=True, do_sample=False,
                     max_new_tokens=512,
                     eos_token_id=[tok.convert_tokens_to_ids("<end_of_turn>"), tok.eos_token_id])
draft = tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True).strip()

2. Restore diacritics with the restorer in restorer/, whose loading code ships with its weights:

import sys
from huggingface_hub import snapshot_download

repo_dir = snapshot_download("Jibby1729/DevKoshurSarvam", allow_patterns=["restorer/*"])
sys.path.insert(0, repo_dir + "/restorer")
from kashmiri_restorer import KashmiriRestorer

restorer = KashmiriRestorer.from_pretrained(repo_dir + "/restorer")
final = restorer.restore([draft])[0]

The defaults are the settings behind every reported score. For batches, pad translator inputs on the left.

Alternative restorer

restorer-v2/ is a second restorer with the same network and loading code, trained with a different recipe. It marks more densely in the IndicVoices spelling style and gets short vowels (pesh and zer) right more often on spoken Kashmiri. To use it, replace restorer/* and /restorer with restorer-v2/* and /restorer-v2 in the code above.

Test set restorer/ restorer-v2/
IN22-Conv (1,503) 17.99 16.87
IN22-Gen (1,024) 19.35 18.52
Voice test (555) 13.58 19.90

KATHE scores on this translator's 4-bit drafts; no-diacritic BLEU is the same for both. The IN22 references use a lighter marking style, so restorer/ scores higher there. In blind word-by-word comparisons where the two differ, Claude Opus and Gemini Argon preferred restorer-v2 for 79% and 76% of words. See restorer-v2/README.md.

4-bit

The translator quantized to 4-bit NF4 (bitsandbytes) with the same decoding and restorer is the configuration behind the numbers above. The weights then occupy 2.79 GB, about 6.4 GB peak with a batch of 32. Full precision (BF16) scores 21.45 / 17.99 on IN22-Conv, 21.42 / 19.33 on IN22-Gen and 19.08 / 13.04 on the voice test.

Files

README.md                     this card
config.json, *.safetensors    translator weights (Sarvam-Translate 4B fine-tune)
tokenizer.json, ...           tokenizer and chat template
restorer/                     152.5M diacritic restorer, Apache-2.0, own LICENSE
restorer-v2/                  alternative 152.5M restorer, IndicVoices-style marking, Apache-2.0
corpus/                       cleaned English-Kashmiri training corpus
corpus/README.md              what is in the corpus and how it was cleaned

License

Part License
Translator (fine-tuned Sarvam-Translate 4B) GPL-3.0, plus the Gemma Terms of Use. It is a derivative of Sarvam-Translate 4B (GPL-3.0), itself a Gemma 3 derivative; both licenses carry over and the Gemma use restrictions apply to anyone using these weights.
Restorers (restorer/, restorer-v2/) Apache-2.0. Trained from scratch on Kashmiri text only; contain no Sarvam or Gemma weights.
Corpus (corpus/) CC-BY-SA-4.0, see corpus/README.md.

This repository includes LICENSE (GPL-3.0), NOTICE, Gemma_Terms_of_Use.txt and Gemma_Prohibited_Use_Policy.txt. The weights were modified by full fine-tuning; the tokenizer and chat template are unchanged from Sarvam-Translate.

Attributions

Base model. Sarvam-Translate by Sarvam AI and AI4Bharat (GPL-3.0), built on Gemma 3 4B by Google (Gemma Terms of Use).

Training data. BPCC human seed and BPCC-H-Daily (AI4Bharat, CC-BY-4.0), NLLB-Seed (Meta, CC-BY-SA-4.0, packaged by AI4Bharat) and IndicVoices Kashmiri transcripts (AI4Bharat, CC-BY-4.0).

Evaluation data. IN22-Gen and IN22-Conv (AI4Bharat, CC-BY-4.0). Scoring follows the KATHE 2026 benchmark and its published Kashmiri normalizer.

Prior work. The restorer design builds on Noore's KATHE 2026 restorer.

Downloads last month
15
Safetensors
Model size
4B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Jibby1729/DevKoshurSarvam

Finetuned
(20)
this model

Datasets used to train Jibby1729/DevKoshurSarvam