YAML Metadata Warning:The pipeline tag "text2text-generation" is not in the official list: text-classification, token-classification, table-question-answering, question-answering, zero-shot-classification, translation, summarization, feature-extraction, text-generation, fill-mask, sentence-similarity, text-to-speech, text-to-audio, automatic-speech-recognition, audio-to-audio, audio-classification, audio-text-to-text, voice-activity-detection, depth-estimation, image-classification, object-detection, image-segmentation, text-to-image, image-to-text, image-to-image, image-to-video, unconditional-image-generation, video-classification, reinforcement-learning, robotics, tabular-classification, tabular-regression, tabular-to-text, table-to-text, multiple-choice, text-ranking, text-retrieval, time-series-forecasting, text-to-video, image-text-to-text, image-text-to-image, image-text-to-video, visual-question-answering, document-question-answering, zero-shot-image-classification, graph-ml, mask-generation, zero-shot-object-detection, text-to-3d, image-to-3d, image-feature-extraction, video-text-to-text, keypoint-detection, visual-document-retrieval, any-to-any, video-to-video, other

Indic Inverse Text Normalizer (small)

A 300M-parameter model that rewrites spoken-form text, as speech recognition writes it, into written form with digits, in 10 Indian languages: numbers, rupee amounts, times, dates, years, percentages, decimals, phone numbers, ranges and cricket scores. It is the reverse of altslate/indic-tts-normalizer-small.

बिल एक हज़ार दो सौ पचास रुपये और पचास पैसे है → बिल ₹1,250.50 है

Try it in your browser: altslate/indic-tts-normalizer, mode "Spoken → text" (runs locally with transformers.js).

Use it after an ASR model (IndicConformer, Whisper, MMS…) when you want "₹1,250" and "4:30" instead of the words.

Results

No test sentence was used for training or for choosing settings. The score is the digit match: the digits in the output, in order, equal those in the reference (separators ignored, so ₹1,250 and Rs 1250 count as equal). It is strict: an extra or missing number makes the sentence wrong.

test this model indic-punct ITN (rules, 2022) input unchanged
A: Wikipedia sentences (568, Dakshina validation split; spoken form made by rules) 0.935 0.389 0
B: messages (263, spoken form written by Qwen3.8-27B and accepted by a judge) 0.449 0.281 0
held-out programmatic sentences, one number (2,194) 0.985 0.554 0.076
held-out programmatic sentences, numbers with internal zeros (300) 0.983 – –

Per language (this model / indic-punct):

hi bn ta te mr gu kn ml pa ur
A .95/.82 .96/.63 .88/.25 .95/.43 .93/.54 .93/.18 .94/.23 .97/.17 .95/.65 .90/0
B .52/.50 .61/.50 .52/.43 .23/.15 .45/.35 .57/.25 .21/.11 .30/.20 .35/.20 .51/0

indic-punct has no Urdu. Read test B with care. Its spoken forms come from an LLM: 56% of them misread the number and were dropped, and some accepted ones are still wrong (245/6 spoken as "दो सौ पचास पर छह"), so part of the gap to 1.0 is in the references. 32 of the 145 misses are ordinals, which this model keeps as words by design (రెండవ, ஒன்றாம்); without them the score is 0.502. The real misses on B are colloquial and digit-by-digit readings (Bengali "ত্রিশ দুই" for 32), long digit strings read one by one (a train number एक दो तीन चार पाँच came out 123445) and some lakh amounts in Telugu. Test A's spoken forms use this model's own reading style, so A measures real sentences, not other ways of saying a number.

Examples

Model output, unchanged (test-A sentences and hand-written ones):

input output
બે હજાર અગિયારના વર્ષમાં ભારત સરકાર તરફથી પદ્મશ્રી વડે સન્માનિત કરવામાં આવ્યા હતા. 2011ના વર્ષમાં ભારત સરકાર તરફથી પદ્મશ્રી વડે સન્માનિત કરવામાં આવ્યા હતા.
ट्रेन सुबह आठ बजकर पंद्रह मिनट पर प्लेटफॉर्म तीन से जाएगी ट्रेन सुबह 8:15 पर प्लेटफॉर्म 3 से जाएगी
पेट्रोल दो दशमलव पाँच प्रतिशत महंगा हुआ पेट्रोल 2.5% महंगा हुआ
என் நம்பர் ஒன்பது எட்டு ஏழு ஆறு ஐந்து நான்கு மூன்று இரண்டு ஒன்று பூஜ்ஜியம், மாலை ஆறு மணிக்கு அழைக்கவும் என் நம்பர் 9876543210, மாலை 6 மணிக்கு அழைக்கவும்
ಹಂಚಿಕೆಯಾದ ಒಟ್ಟು ನೂರ ಎಪ್ಪತ್ತಮೂರು ಟಿಎಂಸಿ ಪೂರ್ಣ ಪ್ರಮಾಣದ ನೀರನ್ನು ಬಲಸಿಕೊಳ್ಳಲಾಗುತ್ತಿದೆ. ಹಂಚಿಕೆಯಾದ ಒಟ್ಟು 173 ಟಿಎಂಸಿ ಪೂರ್ಣ ಪ್ರಮಾಣದ ನೀರನ್ನು ಬಲಸಿಕೊಳ್ಳಲಾಗುತ್ತಿದೆ.
சுமாத்திராவில் மென்டவாய் தீவுகளுக்கு அருகே பெங்கூலுவுக்கு மேற்கே இருநூற்று நாற்பது கிமீ தூரத்தில் இந்நிலநடுக்கம் பதிவாகியது. சுமாத்திராவில் மென்டவாய் தீவுகளுக்கு அருகே பெங்கூலுவுக்கு மேற்கே 240 கிமீ தூரத்தில் இந்நிலநடுக்கம் பதிவாகியது.

Output conventions

One written form per pattern, so you know what to expect:

  • Number words → ASCII digits, no grouping (12951), except rupee amounts, which get ₹ and Indian grouping (₹1,23,456); paise become decimals (₹1,250.50).
  • Percent → 37%; the decimal-point word → . (दो पूर्णांक पाच → 2.5).
  • Times: चार बजकर तीस मिनट → 4:30, दस बजे → 10 बजे; words like सुबह / दोपहर are kept.
  • Dates: day and year become digits, the month stays a word (19 एप्रिल 2003); years become digits (1971).
  • A run of single digits (phone numbers) becomes one number (9876543210).
  • Units, cricket phrasing and ordinal words stay as words around the digits (145 किलोमीटर, 6 विकेट पर 245 रन, तीसरा).
  • Everything else is copied unchanged. Inputs may come with or without punctuation (IndicConformer writes none).

Usage

The input is <language code>-itn: <text>.

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("altslate/indic-itn-small")
model = AutoModelForSeq2SeqLM.from_pretrained("altslate/indic-itn-small")
enc = tok(["hi-itn: बिल एक हज़ार दो सौ पचास रुपये और पचास पैसे है"], return_tensors="pt")
print(tok.batch_decode(model.generate(**enc, max_new_tokens=512), skip_special_tokens=True))

Language codes: hi bn ta te mr gu kn ml pa ur. Feed one sentence at a time.

ONNX (onnxruntime, transformers.js)

onnx/ has the encoder and decoder (with and without past, and merged) in fp32 and per-channel dynamic int8 (*_quantized.onnx, about 460 MB for the int8 set). Checked with greedy decoding in onnxruntime against PyTorch on held-out programmatic tests:

one number (150 rows) exact several numbers (100 rows) exact outputs identical to PyTorch
PyTorch 0.980 0.930 –
ONNX fp32 0.980 0.930 99.3% / 99%
ONNX int8 0.980 0.930 99.3% / 98%

In transformers.js, ByT5 has no tokenizer.json, so encode bytes yourself (each UTF-8 byte + 3, then </s> = 1):

import { AutoModelForSeq2SeqLM, Tensor } from "@huggingface/transformers";
const model = await AutoModelForSeq2SeqLM.from_pretrained("altslate/indic-itn-small", { dtype: "q8" });
const ids = [...new TextEncoder().encode("hi-itn: ट्रेन तीन बजे आएगी")].map((b) => b + 3).concat([1]);
const t = (a) => new Tensor("int64", BigInt64Array.from(a.map(BigInt)), [1, a.length]);
const out = await model.generate({ input_ids: t(ids), attention_mask: t(ids.map(() => 1)), max_new_tokens: ids.length + 64 });
console.log(new TextDecoder().decode(new Uint8Array(out.tolist()[0].map(Number).filter((i) => i >= 3).map((i) => i - 3))));

Training

Training and evaluation code: github.com/AltSlate-Labs/indic-chat-toolkit (src/itn/, jobs itn-*).

  • Base: altslate/indic-tts-normalizer-small (which already knows the number words), 1 epoch on 163k pairs with a new -itn prefix (batch 32, lr 2e-4, bf16), then 1 epoch on a 60k mix that adds 15k sentences with internal-zero numbers (lr 1e-4): the first round dropped the zero after "thousand" (1024 → 124). About 4 hours in total on one NVIDIA GB10 (DGX Spark).
  • Data: the TTS normalizer's training data reversed, with the written side rewritten to the conventions above: carrier sentences with one typed slot written by Ornith-1.5-35B-A3B (MIT), filled programmatically; number words from indic-numtowords (AI4Bharat, MIT) with our corrections; month and unit names from Unicode CLDR; Dakshina train-split Wikipedia sentences (CC BY-SA 4.0). Half of the inputs have punctuation removed, as many ASR systems write none.

Known limitations

  • One canonical reading per number. Inputs are expected in the plain reading this model's data uses ("चार बजकर तीस मिनट"). Colloquial forms ("साढ़े चार", "सवा तीन सौ", "दस पंद्रह" for 10:15) are mostly not handled; see test B.
  • Small numbers become digits even when a writer would use a word. Wikipedia text often writes "चार" or "ஐம்பது" as words; this model writes 4 and 50. About half of its test-A misses are of this kind.
  • Tamil inflected large numbers are the weakest spot (இருபதாயிரம் → 2000 instead of 20000; the இலட்சம் spelling of lakh is not in the training data), and digits inside long numbers are occasionally dropped or doubled.
  • Two numbers in a row without punctuation can merge ("…రూపాయలు నూట ఇరవై ఎనిమిది గిగాబైట్లు" → ₹15,999.12); a comma between them helps.
  • Ordinal words are kept as words, and an article before a number stays a word (एक 1982).
  • Not checked by native speakers. Not covered: Sindhi, Sinhala, Roman-script input, numbers beyond crores, English words for numbers inside Indic text ("five thousand").

License

Apache-2.0. Training data: Ornith outputs (MIT model), indic-numtowords (MIT), CLDR (Unicode license), Dakshina (CC BY-SA 4.0).

Downloads last month
-
Safetensors
Model size
0.3B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for altslate/indic-itn-small

Quantized
(1)
this model

Space using altslate/indic-itn-small 1

Collection including altslate/indic-itn-small