| --- |
| license: apache-2.0 |
| pipeline_tag: translation |
| library_name: executorch |
| base_model: utter-project/EuroLLM-1.7B-Instruct |
| --- |
| |
| # eurollm |
|
|
| This repository hosts the **eurollm** models exported for the |
| [React Native ExecuTorch](https://www.npmjs.com/package/react-native-executorch) |
| library as ExecuTorch `.pte` programs, ready to run on device. |
|
|
| Upstream model: [utter-project/EuroLLM-1.7B-Instruct](https://huggingface.co/utter-project/EuroLLM-1.7B-Instruct) |
|
|
| ## Variants |
|
|
| | Path | Backend | Precision | |
| | --- | --- | --- | |
| | `1_7b/xnnpack/eurollm_1_7b_xnnpack_8da8w.pte` | xnnpack | 8da8w | |
| | `1_7b/mlx/eurollm_1_7b_mlx_int8.pte` | mlx | int8 | |
|
|
| ## Unshipped backends |
|
|
| A backend directory carrying a `NOTES.md` and no `.pte` was exported and |
| not published. Its note records why. |
|
|
| - `1_7b/vulkan` ([why](https://huggingface.co/software-mansion/react-native-executorch-eurollm/blob/main/1_7b/vulkan/NOTES.md)) |
|
|
| ## Repository structure |
|
|
| ``` |
| 1_7b/mlx/config.json 1.5 kB |
| 1_7b/mlx/eurollm_1_7b_mlx_int8.pte 1.7 GB |
| 1_7b/tokenizer.json 8.2 MB |
| 1_7b/tokenizer_config.json 47.4 kB |
| 1_7b/vulkan/NOTES.md 261 B |
| 1_7b/xnnpack/config.json 1.5 kB |
| 1_7b/xnnpack/eurollm_1_7b_xnnpack_8da8w.pte 1.7 GB |
| config.json 29 B |
| ``` |
|
|
| ## Compatibility |
|
|
| These files are published for the **ExecuTorch v1.4.1** runtime. ExecuTorch |
| gives no forward compatibility guarantee, so an older runtime may fail to load |
| them. |
|
|
| To use them in React Native ExecuTorch, pass the model constant shipped in the |
| library's model registry to the corresponding task pipeline. See the |
| [documentation](https://docs.swmansion.com/react-native-executorch/docs/fundamentals/downloading-models). |
|
|
| To load these files in your own ExecuTorch runtime, read the |
| [compatibility note](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md) |
| first. |
|
|
| ## Export notes |
|
|
| Weights are 8-bit: 4-bit (8da4w) lost 6-9 BLEU on WMT13 en->de and often failed |
| to stop. XNNPACK uses dynamic int8 activations with per-channel int8 weights; |
| MLX uses int8 weight-only. For translation, prompt it as the model |
| card does, without a BOS token: |
|
|
| ``` |
| <|im_start|>system\n<|im_end|>\n<|im_start|>user\nTranslate the following English source text to German:\nEnglish: {text} \nGerman: <|im_end|>\n<|im_start|>assistant\n |
| ``` |
|
|
