hari31416/indictrans2-indic-en-1B-ONNX-q4f16

IndicTrans2 1B (indic→en) — ONNX bundle [Q4F16 (4-bit Block Quantization, Lossy)]

This model is part of a suite of optimized/quantized ONNX versions of the base model. Other variants in this direction:

• FP32 (Full Precision / Base): hari31416/indictrans2-indic-en-1B-ONNX
• FP16 (Half Precision): hari31416/indictrans2-indic-en-1B-ONNX-fp16
• INT8 (Dynamic Quantization): hari31416/indictrans2-indic-en-1B-ONNX-int8
• Q4F16 (4-bit Block Quantization): hari31416/indictrans2-indic-en-1B-ONNX-q4f16 (Current)

ONNX-exported and quantized version of ai4bharat/indictrans2-indic-en-1B for in-browser and local edge inference.

• Precision: Q4F16 (4-bit Block Quantization, Lossy)
• Description: 4-bit quantization with float16 scale factors and block size of 32. Reduces model size significantly.
• Source Pipeline & Details: For pipeline details, benchmarks, and usage instructions, see the indictrans2-onnx-export GitHub repository.

Built for use with Transformers.js and onnxruntime-web in the browser, with fast BPE tokenizer.json files that don't require the SentencePiece WASM runtime.

Performance Visualizations

These charts show overall tradeoffs, language-level parity, and category breakdown.

Performance Tradeoffs & Size Comparison

Compared against the FP32 ONNX oracle on the golden evaluation fixtures.

Format	Model Size	Exact Match (Token)	Exact Match (Text)	SacreBLEU (Raw)	Latency (Mean)	Speedup vs. FP32
FP32	4.88 GB	100.00%	100.00%	100.00	49.0 ms	1.000x
FP16	2.44 GB	99.82%	99.82%	99.96	49.7 ms	0.987x
INT8	1.22 GB	94.45%	94.45%	98.00	25.2 ms	1.900x
Q4F16	625.0 MB	88.55%	88.55%	95.44	42.7 ms	1.080x

Language-Level Parity (Q4F16)

Exact match rates and translation quality (SacreBLEU / chrF) per language pair under this precision:

Language Code	Total Fixtures	Token Match Rate	Text Match Rate	SacreBLEU	SacreBLEU (chrF)
eng_Latn	1100	88.5%	88.5%	95.44	97.73

Category-Level Parity (Q4F16)

Exact match rates and translation quality grouped by category types:

Category	Total Fixtures	Token Match Rate	Text Match Rate	SacreBLEU	SacreBLEU (chrF)
Generic	286	81.1%	81.1%	92.66	96.43
Lexicon	264	88.3%	88.3%	95.21	97.66
Numerals	264	95.1%	95.1%	98.30	99.25
Politics	286	90.2%	90.2%	95.57	97.63

Translation Mismatch Examples

Here is a sample of up to 5 translation mismatches compared to the FP32 oracle. Many mismatches represent minor synonym differences or spacing variations.

Mismatch #1 (Category: Generic)

• Source (asm_Beng → eng_Latn): শিশুসকলে উদ্যানখনত খেলিছে।
• Expected (FP32): Children playing in the park.
• Actual (Q4F16): The children are playing in the park.

Mismatch #2 (Category: Numerals)

• Source (asm_Beng → eng_Latn): চিকিৎসালয়খনে 24 ঘণ্টা জৰুৰীকালীন চিকিৎসা সেৱা প্ৰদান কৰে।
• Expected (FP32): The hospital provides 24-hour emergency medical care.
• Actual (Q4F16): The hospital provides 24-hour emergency medical services.

Mismatch #3 (Category: Generic)

• Source (ben_Beng → eng_Latn): কৃত্রিম বুদ্ধিমত্তা শিক্ষার ক্ষেত্রকে বদলে দিচ্ছে।
• Expected (FP32): Artificial intelligence is transforming the education sector.
• Actual (Q4F16): Artificial intelligence is changing the education sector.

Mismatch #4 (Category: Lexicon)

• Source (ben_Beng → eng_Latn): সমুদ্রকে রক্ষা করতে আমাদের প্লাস্টিক বর্জ্য হ্রাস করতে হবে।
• Expected (FP32): We need to reduce plastic waste to protect the oceans.
• Actual (Q4F16): To protect the oceans, we need to reduce plastic waste.

Mismatch #5 (Category: Generic)

• Source (ben_Beng → eng_Latn): সপ্তাহান্তে বন্ধুদের সঙ্গে ফুটবল খেলতে সে উপভোগ করে।
• Expected (FP32): He enjoys playing football with his friends on weekends.
• Actual (Q4F16): He enjoys playing football with his friends on the weekends.

Files

• encoder_model.onnx (and optional .onnx.data weights sidecar)
• decoder_model.onnx and decoder_with_past_model.onnx (share decoder_shared.onnx.data when present)
• translate.py — self-contained Python inference helper (see Usage below)
• Fast tokenizer config files (tokenizer_src.json, tokenizer_tgt.json, tokenizer_meta.json)
• Model configuration configs (config.json, generation_config.json)

Usage Example (Python, onnxruntime)

# translate.py is included in this repo alongside the ONNX bundle.
# You can also find it (and read the full source) at:
#   https://github.com/Hari31416/indictrans2-onnx-export/blob/main/src/translate.py

from translate import IndicTransONNX

# Pass a HF repo ID for automatic download, or a local bundle directory path
model = IndicTransONNX("hari31416/indictrans2-indic-en-1B-ONNX-q4f16")
print(model.translate("चुनाव कौन जीतेगा?", src_lang="hin_Deva", tgt_lang="eng_Latn"))

Required packages:

pip install onnxruntime tokenizers huggingface-hub

License

MIT (preserved from upstream AI4Bharat).