Unpublish variants that lose on device

README.md

@@ -25,40 +25,34 @@ library as ExecuTorch `.pte` programs, ready to run on device.
 | Path | Backend | Precision |
 | --- | --- | --- |
 | `en_de/xnnpack/opus_mt_en_de_xnnpack_int8.pte` | xnnpack | int8 |
-| `en_de/coreml/opus_mt_en_de_coreml_fp16.pte` | coreml | fp16 |
-| `en_de/mlx/opus_mt_en_de_mlx_bf16.pte` | mlx | bf16 |
-| `en_de/mlx/opus_mt_en_de_mlx_int8.pte` | mlx | int8 |
 | `de_en/xnnpack/opus_mt_de_en_xnnpack_int8.pte` | xnnpack | int8 |
-| `de_en/coreml/opus_mt_de_en_coreml_fp16.pte` | coreml | fp16 |
-| `de_en/mlx/opus_mt_de_en_mlx_bf16.pte` | mlx | bf16 |
-| `de_en/mlx/opus_mt_de_en_mlx_int8.pte` | mlx | int8 |
 | `en_es/xnnpack/opus_mt_en_es_xnnpack_int8.pte` | xnnpack | int8 |
-| `en_es/coreml/opus_mt_en_es_coreml_fp16.pte` | coreml | fp16 |
-| `en_es/mlx/opus_mt_en_es_mlx_bf16.pte` | mlx | bf16 |
-| `en_es/mlx/opus_mt_en_es_mlx_int8.pte` | mlx | int8 |
 | `es_en/xnnpack/opus_mt_es_en_xnnpack_int8.pte` | xnnpack | int8 |
-| `es_en/coreml/opus_mt_es_en_coreml_fp16.pte` | coreml | fp16 |
-| `es_en/mlx/opus_mt_es_en_mlx_bf16.pte` | mlx | bf16 |
-| `es_en/mlx/opus_mt_es_en_mlx_int8.pte` | mlx | int8 |
 | `en_fr/xnnpack/opus_mt_en_fr_xnnpack_int8.pte` | xnnpack | int8 |
-| `en_fr/coreml/opus_mt_en_fr_coreml_fp16.pte` | coreml | fp16 |
-| `en_fr/mlx/opus_mt_en_fr_mlx_bf16.pte` | mlx | bf16 |
-| `en_fr/mlx/opus_mt_en_fr_mlx_int8.pte` | mlx | int8 |
 | `fr_en/xnnpack/opus_mt_fr_en_xnnpack_int8.pte` | xnnpack | int8 |
-| `fr_en/coreml/opus_mt_fr_en_coreml_fp16.pte` | coreml | fp16 |
-| `fr_en/mlx/opus_mt_fr_en_mlx_bf16.pte` | mlx | bf16 |
-| `fr_en/mlx/opus_mt_fr_en_mlx_int8.pte` | mlx | int8 |
 
 ## Unshipped backends
 
 A backend directory carrying a `NOTES.md` and no `.pte` was exported and
 not published. Its note records why.
 
+- `de_en/coreml` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/de_en/coreml/NOTES.md))
+- `de_en/mlx` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/de_en/mlx/NOTES.md))
 - `de_en/vulkan` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/de_en/vulkan/NOTES.md))
+- `en_de/coreml` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/en_de/coreml/NOTES.md))
+- `en_de/mlx` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/en_de/mlx/NOTES.md))
 - `en_de/vulkan` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/en_de/vulkan/NOTES.md))
+- `en_es/coreml` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/en_es/coreml/NOTES.md))
+- `en_es/mlx` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/en_es/mlx/NOTES.md))
 - `en_es/vulkan` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/en_es/vulkan/NOTES.md))
+- `en_fr/coreml` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/en_fr/coreml/NOTES.md))
+- `en_fr/mlx` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/en_fr/mlx/NOTES.md))
 - `en_fr/vulkan` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/en_fr/vulkan/NOTES.md))
+- `es_en/coreml` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/es_en/coreml/NOTES.md))
+- `es_en/mlx` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/es_en/mlx/NOTES.md))
 - `es_en/vulkan` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/es_en/vulkan/NOTES.md))
+- `fr_en/coreml` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/fr_en/coreml/NOTES.md))
+- `fr_en/mlx` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/fr_en/mlx/NOTES.md))
 - `fr_en/vulkan` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/fr_en/vulkan/NOTES.md))
 
 ## Repository structure
@@ -66,56 +60,38 @@ not published. Its note records why.
 ```
 LICENSE                                        360 B
 config.json                                    29 B
-de_en/coreml/config.json                       5.5 kB
-de_en/coreml/opus_mt_de_en_coreml_fp16.pte     313.6 MB
-de_en/mlx/config.json                          10.6 kB
-de_en/mlx/opus_mt_de_en_mlx_bf16.pte           148.9 MB
-de_en/mlx/opus_mt_de_en_mlx_int8.pte           135.8 MB
+de_en/coreml/NOTES.md                          230 B
+de_en/mlx/NOTES.md                             169 B
 de_en/tokenizer.json                           2.4 MB
 de_en/vulkan/NOTES.md                          218 B
 de_en/xnnpack/config.json                      5.5 kB
 de_en/xnnpack/opus_mt_de_en_xnnpack_int8.pte   107.0 MB
-en_de/coreml/config.json                       5.4 kB
-en_de/coreml/opus_mt_en_de_coreml_fp16.pte     313.6 MB
-en_de/mlx/config.json                          10.6 kB
-en_de/mlx/opus_mt_en_de_mlx_bf16.pte           148.9 MB
-en_de/mlx/opus_mt_en_de_mlx_int8.pte           135.8 MB
+en_de/coreml/NOTES.md                          230 B
+en_de/mlx/NOTES.md                             169 B
 en_de/tokenizer.json                           2.4 MB
 en_de/vulkan/NOTES.md                          218 B
 en_de/xnnpack/config.json                      5.5 kB
 en_de/xnnpack/opus_mt_en_de_xnnpack_int8.pte   107.0 MB
-en_es/coreml/config.json                       5.5 kB
-en_es/coreml/opus_mt_en_es_coreml_fp16.pte     334.8 MB
-en_es/mlx/config.json                          10.6 kB
-en_es/mlx/opus_mt_en_es_mlx_bf16.pte           156.0 MB
-en_es/mlx/opus_mt_en_es_mlx_int8.pte           146.5 MB
+en_es/coreml/NOTES.md                          230 B
+en_es/mlx/NOTES.md                             169 B
 en_es/tokenizer.json                           2.7 MB
 en_es/vulkan/NOTES.md                          218 B
 en_es/xnnpack/config.json                      5.5 kB
 en_es/xnnpack/opus_mt_en_es_xnnpack_int8.pte   114.1 MB
-en_fr/coreml/config.json                       5.5 kB
-en_fr/coreml/opus_mt_en_fr_coreml_fp16.pte     318.0 MB
-en_fr/mlx/config.json                          10.6 kB
-en_fr/mlx/opus_mt_en_fr_mlx_bf16.pte           150.4 MB
-en_fr/mlx/opus_mt_en_fr_mlx_int8.pte           138.0 MB
+en_fr/coreml/NOTES.md                          230 B
+en_fr/mlx/NOTES.md                             169 B
 en_fr/tokenizer.json                           2.4 MB
 en_fr/vulkan/NOTES.md                          218 B
 en_fr/xnnpack/config.json                      5.5 kB
 en_fr/xnnpack/opus_mt_en_fr_xnnpack_int8.pte   108.4 MB
-es_en/coreml/config.json                       5.5 kB
-es_en/coreml/opus_mt_es_en_coreml_fp16.pte     334.8 MB
-es_en/mlx/config.json                          10.6 kB
-es_en/mlx/opus_mt_es_en_mlx_bf16.pte           156.0 MB
-es_en/mlx/opus_mt_es_en_mlx_int8.pte           146.5 MB
+es_en/coreml/NOTES.md                          230 B
+es_en/mlx/NOTES.md                             169 B
 es_en/tokenizer.json                           2.7 MB
 es_en/vulkan/NOTES.md                          218 B
 es_en/xnnpack/config.json                      5.5 kB
 es_en/xnnpack/opus_mt_es_en_xnnpack_int8.pte   114.1 MB
-fr_en/coreml/config.json                       5.5 kB
-fr_en/coreml/opus_mt_fr_en_coreml_fp16.pte     318.0 MB
-fr_en/mlx/config.json                          10.6 kB
-fr_en/mlx/opus_mt_fr_en_mlx_bf16.pte           150.4 MB
-fr_en/mlx/opus_mt_fr_en_mlx_int8.pte           138.0 MB
+fr_en/coreml/NOTES.md                          230 B
+fr_en/mlx/NOTES.md                             169 B
 fr_en/tokenizer.json                           2.4 MB
 fr_en/vulkan/NOTES.md                          218 B
 fr_en/xnnpack/config.json                      5.5 kB
@@ -155,7 +131,5 @@ Each directory is one language pair with its own upstream model and license.
 Each `.pte` has two methods: `encode` returns the cross-attention keys and
 values of all decoder layers, and `decode` runs one greedy step against a
 256-token self-attention cache held in the program. The tokenizer adds no
-`</s>`; append it, start decoding from `<pad>` and stop at `</s>`. Core ML
-encodes in fp16 on all compute units and decodes in fp32 on the CPU, because
-Core ML's fp16 state corrupts the cache. XNNPACK ships int8 only: fp32 is 3.9x larger and
-2-3x slower on device for the same BLEU.
+`</s>`; append it, start decoding from `<pad>` and stop at `</s>`. Only XNNPACK int8
+ships: it was the fastest variant on both a Galaxy S26 Ultra and an iPhone 16.

de_en/coreml/NOTES.md

@@ -0,0 +1,3 @@
+Exported and not published. On an iPhone 16 the best Core ML setup (fp16, Neural
+Engine encode and decode) takes ~84 ms per sentence against ~46 ms for XNNPACK int8:
+a one-token decode step costs 3.8 ms vs 2.0 ms. Use `xnnpack/`.

de_en/coreml/opus_mt_de_en_coreml_fp16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:d9d38cc0ed7af5bf488ac2e0f07eebb273d7428570835d071ee9ac983a67914e
-size 313623341

de_en/mlx/NOTES.md

@@ -0,0 +1,2 @@
+Exported and not published. On an iPhone 16 MLX decodes at 3.2 ms/step (int8) and
+3.9 ms/step (bf16) against 1.95 ms for XNNPACK int8, at the same BLEU. Use `xnnpack/`.

de_en/mlx/opus_mt_de_en_mlx_bf16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:a55547ee440e87fcdd3a5d59a6461cfd0e4b60b8850a7c21ead47d3ab1ef2745
-size 148918506

de_en/mlx/opus_mt_de_en_mlx_int8.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:020f4e120e0cf69abdae9f2a0afd48a12ba7df53905bb82e37aabffeb1df58d7
-size 135845120

en_de/coreml/NOTES.md

@@ -0,0 +1,3 @@
+Exported and not published. On an iPhone 16 the best Core ML setup (fp16, Neural
+Engine encode and decode) takes ~84 ms per sentence against ~46 ms for XNNPACK int8:
+a one-token decode step costs 3.8 ms vs 2.0 ms. Use `xnnpack/`.

en_de/coreml/opus_mt_en_de_coreml_fp16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:69d2015429da9d13abe190ff435467bd9fafe7ceb27bdd6995c51ed426ca267e
-size 313623341

en_de/mlx/NOTES.md

@@ -0,0 +1,2 @@
+Exported and not published. On an iPhone 16 MLX decodes at 3.2 ms/step (int8) and
+3.9 ms/step (bf16) against 1.95 ms for XNNPACK int8, at the same BLEU. Use `xnnpack/`.

en_de/mlx/opus_mt_en_de_mlx_bf16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:2043f502b1a76a80225b60d0c1660c0380398174f2432a4862cf701047b15f4d
-size 148918506

en_de/mlx/opus_mt_en_de_mlx_int8.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f81efa7c2d98906e52a049e2e8f7e4ddb60891448928ca0e65aafbf7bc4b9be9
-size 135845120

en_es/coreml/NOTES.md

@@ -0,0 +1,3 @@
+Exported and not published. On an iPhone 16 the best Core ML setup (fp16, Neural
+Engine encode and decode) takes ~84 ms per sentence against ~46 ms for XNNPACK int8:
+a one-token decode step costs 3.8 ms vs 2.0 ms. Use `xnnpack/`.

en_es/coreml/opus_mt_en_es_coreml_fp16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:9dbb45583caa55aa154075c9000ddec49f0d0d273fe3906fea835a6f6f195981
-size 334847755

en_es/mlx/NOTES.md

@@ -0,0 +1,2 @@
+Exported and not published. On an iPhone 16 MLX decodes at 3.2 ms/step (int8) and
+3.9 ms/step (bf16) against 1.95 ms for XNNPACK int8, at the same BLEU. Use `xnnpack/`.

en_es/mlx/opus_mt_en_es_mlx_bf16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:01a90618dc9044ad8af4b87411542eb744c077a335d65ef6bb1f7c3a0d1ffed1
-size 155997906

en_es/mlx/opus_mt_en_es_mlx_int8.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:e6b75da51d10e52fb9440571c19010aaa71a4fc5fbdddc5bb1d26965a2171d2b
-size 146512512

en_fr/coreml/NOTES.md

@@ -0,0 +1,3 @@
+Exported and not published. On an iPhone 16 the best Core ML setup (fp16, Neural
+Engine encode and decode) takes ~84 ms per sentence against ~46 ms for XNNPACK int8:
+a one-token decode step costs 3.8 ms vs 2.0 ms. Use `xnnpack/`.

en_fr/coreml/opus_mt_en_fr_coreml_fp16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:268b212402567bc52e23efedf4146eda2ef686e5c2ed7077eba26731ced4d222
-size 317969715

en_fr/mlx/NOTES.md

@@ -0,0 +1,2 @@
+Exported and not published. On an iPhone 16 MLX decodes at 3.2 ms/step (int8) and
+3.9 ms/step (bf16) against 1.95 ms for XNNPACK int8, at the same BLEU. Use `xnnpack/`.

en_fr/mlx/opus_mt_en_fr_mlx_bf16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:ea9d5f65a8169c4f8bda6e442a150bff7ca31bba20d99fd697f500a2a254a8ec
-size 150368244

en_fr/mlx/opus_mt_en_fr_mlx_int8.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:19499664133a66fa343eb2a37d8ac7ee4484be8b786faedeaf0141ecae5c0be0
-size 138029568

es_en/coreml/NOTES.md

@@ -0,0 +1,3 @@
+Exported and not published. On an iPhone 16 the best Core ML setup (fp16, Neural
+Engine encode and decode) takes ~84 ms per sentence against ~46 ms for XNNPACK int8:
+a one-token decode step costs 3.8 ms vs 2.0 ms. Use `xnnpack/`.

es_en/coreml/opus_mt_es_en_coreml_fp16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:4c7b851436f2f0514bbe1ba80495de40c9afb228dca7fdc075b65d18261b28d0
-size 334847755

es_en/mlx/NOTES.md

@@ -0,0 +1,2 @@
+Exported and not published. On an iPhone 16 MLX decodes at 3.2 ms/step (int8) and
+3.9 ms/step (bf16) against 1.95 ms for XNNPACK int8, at the same BLEU. Use `xnnpack/`.

es_en/mlx/opus_mt_es_en_mlx_bf16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:35b4b1f8b76194622c31d54a43f8715f0ef4a24bb988fb33130e1e6ee68f893f
-size 155997906

es_en/mlx/opus_mt_es_en_mlx_int8.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:cac335c8ed672451483d27562fe096f65955a757b0f9d3d780c8537b6f08d05c
-size 146512512

fr_en/coreml/NOTES.md

@@ -0,0 +1,3 @@
+Exported and not published. On an iPhone 16 the best Core ML setup (fp16, Neural
+Engine encode and decode) takes ~84 ms per sentence against ~46 ms for XNNPACK int8:
+a one-token decode step costs 3.8 ms vs 2.0 ms. Use `xnnpack/`.

fr_en/coreml/opus_mt_fr_en_coreml_fp16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:31f60a251a10a1dee8ad23563f3fa825c9951bf241e1ce05f7c8a0e9d6214255
-size 317969715

fr_en/mlx/NOTES.md

@@ -0,0 +1,2 @@
+Exported and not published. On an iPhone 16 MLX decodes at 3.2 ms/step (int8) and
+3.9 ms/step (bf16) against 1.95 ms for XNNPACK int8, at the same BLEU. Use `xnnpack/`.

fr_en/mlx/opus_mt_fr_en_mlx_bf16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:1e8288d887c4a028ea10449eee7a42566bf5270c4ed92356c2397b408b6c412e
-size 150368244

fr_en/mlx/opus_mt_fr_en_mlx_int8.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:b70315f4fc42d805bc169db2ec8d2563c03ea761d9b38b7846b1c9e59afe1299
-size 138029568