Zweiarm/opus-mt-onnx-int8

OPUS-MT ONNX int8 packages

The official Helsinki-NLP OPUS-MT Marian models exported to ONNX and quantized to int8 for on-device translation with ONNX Runtime (used by the Hi Talk Android app, among others). Each opus-mt-<src>-<tgt>/ folder holds one direction:

pair	converted from	size
ar-en	Helsinki-NLP/opus-mt-ar-en	231 MB
cs-en	Helsinki-NLP/opus-mt-cs-en	230 MB
de-en	Helsinki-NLP/opus-mt-de-en	219 MB
en-ar	Helsinki-NLP/opus-mt-en-ar	231 MB
en-cs	Helsinki-NLP/opus-mt-en-cs	230 MB
en-de	Helsinki-NLP/opus-mt-en-de	219 MB
en-es	Helsinki-NLP/opus-mt-en-es	237 MB
en-fr	Helsinki-NLP/opus-mt-en-fr	223 MB
en-he	Helsinki-NLP/opus-mt-en-he	239 MB
en-ru	Helsinki-NLP/opus-mt-en-ru	231 MB
en-tr	Helsinki-NLP/opus-tatoeba-en-tr	224 MB
en-vi	Helsinki-NLP/opus-mt-en-vi	207 MB
en-zh	Helsinki-NLP/opus-mt-en-zh	237 MB
es-en	Helsinki-NLP/opus-mt-es-en	237 MB
fr-en	Helsinki-NLP/opus-mt-fr-en	223 MB
id-en	Helsinki-NLP/opus-mt-id-en	210 MB
ko-en	Helsinki-NLP/opus-mt-ko-en	237 MB
ru-en	Helsinki-NLP/opus-mt-ru-en	231 MB
th-en	Helsinki-NLP/opus-mt-th-en	231 MB
tr-en	Helsinki-NLP/opus-mt-tr-en	230 MB
vi-en	Helsinki-NLP/opus-mt-vi-en	208 MB
zh-en	Helsinki-NLP/opus-mt-zh-en	237 MB

Files per pair

• encoder_model.onnx, decoder_model.onnx, decoder_with_past_model.onnx — 🤗 Optimum export (text2text-generation-with-past) of the original weights, dynamically quantized to int8 with ONNX Runtime (quantize_dynamic).
• tokenizer.bin — the source SentencePiece unigram model and the Marian vocabulary (source.spm, vocab.json) repacked into one binary table (layout documented in the converter's mtok.py); the originals stay in the Helsinki-NLP repositories.
• manifest.json — byte sizes and SHA-256 sums of the four files, for consumers that pin and verify what they download.

The graphs bind the input/output names Optimum assigns (input_ids, attention_mask, encoder_hidden_states, past_key_values.N.*, present.N.*, logits) and run with any ONNX Runtime.

Attribution and licence

The models are derivative works of the Helsinki-NLP OPUS-MT models listed above, trained by the OPUS-MT project (Jörg Tiedemann and Santhosh Thottingal, OPUS-MT — Building open translation services for the World, EAMT 2020) with the Marian NMT framework, and published under CC-BY-4.0. This repository is likewise published under CC-BY-4.0. Changes made to the originals: ONNX export, int8 dynamic quantization, and repacking of the tokenizer files; the weights are otherwise unchanged.

Converted with helsinki-onnx-converter; the exact pipeline, checks and the tokenizer.bin layout are documented there.