software-mansion/Hy-MT2-1.8B-ExecuTorch

hy_mt2

This repository hosts the hy_mt2 models exported for the React Native ExecuTorch library as ExecuTorch .pte programs, ready to run on device.

Upstream model: tencent/Hy-MT2-1.8B

Variants

Path	Backend	Precision
1_8b/xnnpack/hy_mt2_1_8b_xnnpack_8da4w.pte	xnnpack	8da4w
1_8b/xnnpack/hy_mt2_1_8b_xnnpack_8da8w.pte	xnnpack	8da8w
1_8b/mlx/hy_mt2_1_8b_mlx_int8.pte	mlx	int8

Repository structure

1_8b/mlx/config.json                         1.5 kB
1_8b/mlx/hy_mt2_1_8b_mlx_int8.pte            1.8 GB
1_8b/tokenizer.json                          5.2 MB
1_8b/tokenizer_config.json                   166.5 kB
1_8b/xnnpack/config.json                     2.6 kB
1_8b/xnnpack/hy_mt2_1_8b_xnnpack_8da4w.pte   1.2 GB
1_8b/xnnpack/hy_mt2_1_8b_xnnpack_8da8w.pte   2.0 GB
config.json                                  28 B

Compatibility

These files are published for the ExecuTorch v1.4.1 runtime. ExecuTorch gives no forward compatibility guarantee, so an older runtime may fail to load them.

To use them in React Native ExecuTorch, pass the model constant shipped in the library's model registry to the corresponding task pipeline. See the documentation.

To load these files in your own ExecuTorch runtime, read the compatibility note first.

Export notes

XNNPACK uses dynamic int8 activations with int4 weights (group 32, the default) or per-channel int8 weights; 4-bit costs 0.2-1.5 BLEU for 43% less size and 1.5x faster decode. MLX uses int8 weight-only. Prompt it as the model card does, with no system prompt:

<｜hy_begin▁of▁sentence｜><｜hy_User｜>Translate the following text into German. Note that you should only output the translated result without any additional explanation:\n\n{text}<｜hy_Assistant｜>

When either language is Chinese, use the card's Chinese prompt (将以下文本翻译为{目标语言}，注意只需要输出翻译后的结果，不要额外解释：\n\n{text}).

1_8b/tokenizer.json is upstream's with its no-op normalizer set to null and each Split regex extended to match the text between its matches. Token ids are unchanged; the rewrite works around a loader bug in older pytorch/tokenizers. 1_8b/tokenizer_config.json carries upstream's chat_template.jinja as chat_template.