Unpublish variants that lose on device

README.md

@@ -24,42 +24,42 @@ library as ExecuTorch `.pte` programs, ready to run on device.
 
 | Path | Backend | Precision |
 | --- | --- | --- |
-| `en_de/xnnpack/opus_mt_en_de_xnnpack_fp32.pte` | xnnpack | fp32 |
 | `en_de/xnnpack/opus_mt_en_de_xnnpack_int8.pte` | xnnpack | int8 |
 | `en_de/coreml/opus_mt_en_de_coreml_fp16.pte` | coreml | fp16 |
 | `en_de/mlx/opus_mt_en_de_mlx_bf16.pte` | mlx | bf16 |
 | `en_de/mlx/opus_mt_en_de_mlx_int8.pte` | mlx | int8 |
-| `en_de/vulkan/opus_mt_en_de_vulkan_fp16.pte` | vulkan | fp16 |
-| `de_en/xnnpack/opus_mt_de_en_xnnpack_fp32.pte` | xnnpack | fp32 |
 | `de_en/xnnpack/opus_mt_de_en_xnnpack_int8.pte` | xnnpack | int8 |
 | `de_en/coreml/opus_mt_de_en_coreml_fp16.pte` | coreml | fp16 |
 | `de_en/mlx/opus_mt_de_en_mlx_bf16.pte` | mlx | bf16 |
 | `de_en/mlx/opus_mt_de_en_mlx_int8.pte` | mlx | int8 |
-| `de_en/vulkan/opus_mt_de_en_vulkan_fp16.pte` | vulkan | fp16 |
-| `en_es/xnnpack/opus_mt_en_es_xnnpack_fp32.pte` | xnnpack | fp32 |
 | `en_es/xnnpack/opus_mt_en_es_xnnpack_int8.pte` | xnnpack | int8 |
 | `en_es/coreml/opus_mt_en_es_coreml_fp16.pte` | coreml | fp16 |
 | `en_es/mlx/opus_mt_en_es_mlx_bf16.pte` | mlx | bf16 |
 | `en_es/mlx/opus_mt_en_es_mlx_int8.pte` | mlx | int8 |
-| `en_es/vulkan/opus_mt_en_es_vulkan_fp16.pte` | vulkan | fp16 |
-| `es_en/xnnpack/opus_mt_es_en_xnnpack_fp32.pte` | xnnpack | fp32 |
 | `es_en/xnnpack/opus_mt_es_en_xnnpack_int8.pte` | xnnpack | int8 |
 | `es_en/coreml/opus_mt_es_en_coreml_fp16.pte` | coreml | fp16 |
 | `es_en/mlx/opus_mt_es_en_mlx_bf16.pte` | mlx | bf16 |
 | `es_en/mlx/opus_mt_es_en_mlx_int8.pte` | mlx | int8 |
-| `es_en/vulkan/opus_mt_es_en_vulkan_fp16.pte` | vulkan | fp16 |
-| `en_fr/xnnpack/opus_mt_en_fr_xnnpack_fp32.pte` | xnnpack | fp32 |
 | `en_fr/xnnpack/opus_mt_en_fr_xnnpack_int8.pte` | xnnpack | int8 |
 | `en_fr/coreml/opus_mt_en_fr_coreml_fp16.pte` | coreml | fp16 |
 | `en_fr/mlx/opus_mt_en_fr_mlx_bf16.pte` | mlx | bf16 |
 | `en_fr/mlx/opus_mt_en_fr_mlx_int8.pte` | mlx | int8 |
-| `en_fr/vulkan/opus_mt_en_fr_vulkan_fp16.pte` | vulkan | fp16 |
-| `fr_en/xnnpack/opus_mt_fr_en_xnnpack_fp32.pte` | xnnpack | fp32 |
 | `fr_en/xnnpack/opus_mt_fr_en_xnnpack_int8.pte` | xnnpack | int8 |
 | `fr_en/coreml/opus_mt_fr_en_coreml_fp16.pte` | coreml | fp16 |
 | `fr_en/mlx/opus_mt_fr_en_mlx_bf16.pte` | mlx | bf16 |
 | `fr_en/mlx/opus_mt_fr_en_mlx_int8.pte` | mlx | int8 |
-| `fr_en/vulkan/opus_mt_fr_en_vulkan_fp16.pte` | vulkan | fp16 |
+
+## Unshipped backends
+
+A backend directory carrying a `NOTES.md` and no `.pte` was exported and
+not published. Its note records why.
+
+- `de_en/vulkan` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/de_en/vulkan/NOTES.md))
+- `en_de/vulkan` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/en_de/vulkan/NOTES.md))
+- `en_es/vulkan` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/en_es/vulkan/NOTES.md))
+- `en_fr/vulkan` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/en_fr/vulkan/NOTES.md))
+- `es_en/vulkan` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/es_en/vulkan/NOTES.md))
+- `fr_en/vulkan` ([why](https://huggingface.co/software-mansion/react-native-executorch-opus-mt/blob/main/fr_en/vulkan/NOTES.md))
 
 ## Repository structure
 
@@ -72,10 +72,8 @@ de_en/mlx/config.json                          10.6 kB
 de_en/mlx/opus_mt_de_en_mlx_bf16.pte           148.9 MB
 de_en/mlx/opus_mt_de_en_mlx_int8.pte           135.8 MB
 de_en/tokenizer.json                           2.4 MB
-de_en/vulkan/config.json                       5.5 kB
-de_en/vulkan/opus_mt_de_en_vulkan_fp16.pte     129.3 MB
-de_en/xnnpack/config.json                      10.6 kB
-de_en/xnnpack/opus_mt_de_en_xnnpack_fp32.pte   416.8 MB
+de_en/vulkan/NOTES.md                          218 B
+de_en/xnnpack/config.json                      5.5 kB
 de_en/xnnpack/opus_mt_de_en_xnnpack_int8.pte   107.0 MB
 en_de/coreml/config.json                       5.4 kB
 en_de/coreml/opus_mt_en_de_coreml_fp16.pte     313.6 MB
@@ -83,10 +81,8 @@ en_de/mlx/config.json                          10.6 kB
 en_de/mlx/opus_mt_en_de_mlx_bf16.pte           148.9 MB
 en_de/mlx/opus_mt_en_de_mlx_int8.pte           135.8 MB
 en_de/tokenizer.json                           2.4 MB
-en_de/vulkan/config.json                       5.4 kB
-en_de/vulkan/opus_mt_en_de_vulkan_fp16.pte     129.3 MB
-en_de/xnnpack/config.json                      10.6 kB
-en_de/xnnpack/opus_mt_en_de_xnnpack_fp32.pte   416.8 MB
+en_de/vulkan/NOTES.md                          218 B
+en_de/xnnpack/config.json                      5.5 kB
 en_de/xnnpack/opus_mt_en_de_xnnpack_int8.pte   107.0 MB
 en_es/coreml/config.json                       5.5 kB
 en_es/coreml/opus_mt_en_es_coreml_fp16.pte     334.8 MB
@@ -94,10 +90,8 @@ en_es/mlx/config.json                          10.6 kB
 en_es/mlx/opus_mt_en_es_mlx_bf16.pte           156.0 MB
 en_es/mlx/opus_mt_en_es_mlx_int8.pte           146.5 MB
 en_es/tokenizer.json                           2.7 MB
-en_es/vulkan/config.json                       5.5 kB
-en_es/vulkan/opus_mt_en_es_vulkan_fp16.pte     136.4 MB
-en_es/xnnpack/config.json                      10.6 kB
-en_es/xnnpack/opus_mt_en_es_xnnpack_fp32.pte   445.1 MB
+en_es/vulkan/NOTES.md                          218 B
+en_es/xnnpack/config.json                      5.5 kB
 en_es/xnnpack/opus_mt_en_es_xnnpack_int8.pte   114.1 MB
 en_fr/coreml/config.json                       5.5 kB
 en_fr/coreml/opus_mt_en_fr_coreml_fp16.pte     318.0 MB
@@ -105,10 +99,8 @@ en_fr/mlx/config.json                          10.6 kB
 en_fr/mlx/opus_mt_en_fr_mlx_bf16.pte           150.4 MB
 en_fr/mlx/opus_mt_en_fr_mlx_int8.pte           138.0 MB
 en_fr/tokenizer.json                           2.4 MB
-en_fr/vulkan/config.json                       5.5 kB
-en_fr/vulkan/opus_mt_en_fr_vulkan_fp16.pte     130.7 MB
-en_fr/xnnpack/config.json                      10.6 kB
-en_fr/xnnpack/opus_mt_en_fr_xnnpack_fp32.pte   422.6 MB
+en_fr/vulkan/NOTES.md                          218 B
+en_fr/xnnpack/config.json                      5.5 kB
 en_fr/xnnpack/opus_mt_en_fr_xnnpack_int8.pte   108.4 MB
 es_en/coreml/config.json                       5.5 kB
 es_en/coreml/opus_mt_es_en_coreml_fp16.pte     334.8 MB
@@ -116,10 +108,8 @@ es_en/mlx/config.json                          10.6 kB
 es_en/mlx/opus_mt_es_en_mlx_bf16.pte           156.0 MB
 es_en/mlx/opus_mt_es_en_mlx_int8.pte           146.5 MB
 es_en/tokenizer.json                           2.7 MB
-es_en/vulkan/config.json                       5.5 kB
-es_en/vulkan/opus_mt_es_en_vulkan_fp16.pte     136.4 MB
-es_en/xnnpack/config.json                      10.6 kB
-es_en/xnnpack/opus_mt_es_en_xnnpack_fp32.pte   445.1 MB
+es_en/vulkan/NOTES.md                          218 B
+es_en/xnnpack/config.json                      5.5 kB
 es_en/xnnpack/opus_mt_es_en_xnnpack_int8.pte   114.1 MB
 fr_en/coreml/config.json                       5.5 kB
 fr_en/coreml/opus_mt_fr_en_coreml_fp16.pte     318.0 MB
@@ -127,10 +117,8 @@ fr_en/mlx/config.json                          10.6 kB
 fr_en/mlx/opus_mt_fr_en_mlx_bf16.pte           150.4 MB
 fr_en/mlx/opus_mt_fr_en_mlx_int8.pte           138.0 MB
 fr_en/tokenizer.json                           2.4 MB
-fr_en/vulkan/config.json                       5.5 kB
-fr_en/vulkan/opus_mt_fr_en_vulkan_fp16.pte     130.7 MB
-fr_en/xnnpack/config.json                      10.6 kB
-fr_en/xnnpack/opus_mt_fr_en_xnnpack_fp32.pte   422.6 MB
+fr_en/vulkan/NOTES.md                          218 B
+fr_en/xnnpack/config.json                      5.5 kB
 fr_en/xnnpack/opus_mt_fr_en_xnnpack_int8.pte   108.4 MB
 ```
 
@@ -169,5 +157,5 @@ values of all decoder layers, and `decode` runs one greedy step against a
 256-token self-attention cache held in the program. The tokenizer adds no
 `</s>`; append it, start decoding from `<pad>` and stop at `</s>`. Core ML
 encodes in fp16 on all compute units and decodes in fp32 on the CPU, because
-Core ML's fp16 state corrupts the cache. The Vulkan variant runs the encoder on
-the GPU and the decoder on XNNPACK int8.
+Core ML's fp16 state corrupts the cache. XNNPACK ships int8 only: fp32 is 3.9x larger and
+2-3x slower on device for the same BLEU.

de_en/vulkan/NOTES.md

@@ -0,0 +1,3 @@
+Exported and not published. On a Galaxy S26 Ultra the Vulkan encoder takes 6.0 ms
+against 1.66 ms for XNNPACK int8, and the decoder is the same XNNPACK int8 graph, so it
+only adds load time. Use `xnnpack/` on Android.

de_en/vulkan/opus_mt_de_en_vulkan_fp16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:6af8b91624b4c9abe9d74b6a05f9910bd0bd38c9242e3c07a57f83b23a487e33
-size 129254740

de_en/xnnpack/opus_mt_de_en_xnnpack_fp32.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:fc38f6ee3706a3bb0853dc6be1e6956d6791cdfcbd29f1104c1e72903c237e7d
-size 416786004

en_de/vulkan/NOTES.md

@@ -0,0 +1,3 @@
+Exported and not published. On a Galaxy S26 Ultra the Vulkan encoder takes 6.0 ms
+against 1.66 ms for XNNPACK int8, and the decoder is the same XNNPACK int8 graph, so it
+only adds load time. Use `xnnpack/` on Android.

en_de/vulkan/opus_mt_en_de_vulkan_fp16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:cd6ceadef0226c06b2a190b62abaf32b8c95414bff59df4838a66c3680a979d5
-size 129254740

en_de/xnnpack/opus_mt_en_de_xnnpack_fp32.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f722e39388b5060134899e77e966edefcb9e150b1d1f71becfc3ec3f514c635f
-size 416786004

en_es/vulkan/NOTES.md

@@ -0,0 +1,3 @@
+Exported and not published. On a Galaxy S26 Ultra the Vulkan encoder takes 6.0 ms
+against 1.66 ms for XNNPACK int8, and the decoder is the same XNNPACK int8 graph, so it
+only adds load time. Use `xnnpack/` on Android.

en_es/vulkan/opus_mt_en_es_vulkan_fp16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:1090c247d3e2e2fd6a148d4933f8802a0e621ed5bc82aae21bb567d271205364
-size 136403108

en_es/xnnpack/opus_mt_en_es_xnnpack_fp32.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:81bb8d13a5bd08c3b55fe8f1b7ae30cb829b5502f62f1346574c6341c8cbb935
-size 445076004

en_fr/vulkan/NOTES.md

@@ -0,0 +1,3 @@
+Exported and not published. On a Galaxy S26 Ultra the Vulkan encoder takes 6.0 ms
+against 1.66 ms for XNNPACK int8, and the decoder is the same XNNPACK int8 graph, so it
+only adds load time. Use `xnnpack/` on Android.

en_fr/vulkan/opus_mt_en_fr_vulkan_fp16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:bb6174fd6c60f01974b6c7a6d9ec5843141b9054d9806c178183e9d89e7db982
-size 130718568

en_fr/xnnpack/opus_mt_en_fr_xnnpack_fp32.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:a7338e93648ea0f64a545afb2bcfaedbc6688047301a8e2a160210c5d04e7023
-size 422579304

es_en/vulkan/NOTES.md

@@ -0,0 +1,3 @@
+Exported and not published. On a Galaxy S26 Ultra the Vulkan encoder takes 6.0 ms
+against 1.66 ms for XNNPACK int8, and the decoder is the same XNNPACK int8 graph, so it
+only adds load time. Use `xnnpack/` on Android.

es_en/vulkan/opus_mt_es_en_vulkan_fp16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:be2353e6db6215b6cc1023a6c37fca73199b226a7c614fb0f94375078e1fafd0
-size 136403108

es_en/xnnpack/opus_mt_es_en_xnnpack_fp32.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:5cfb011c45730ddabf18de26b9729509f642bd8b2be9a4f9dbe8dd36babebafa
-size 445076004

fr_en/vulkan/NOTES.md

@@ -0,0 +1,3 @@
+Exported and not published. On a Galaxy S26 Ultra the Vulkan encoder takes 6.0 ms
+against 1.66 ms for XNNPACK int8, and the decoder is the same XNNPACK int8 graph, so it
+only adds load time. Use `xnnpack/` on Android.

fr_en/vulkan/opus_mt_fr_en_vulkan_fp16.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:6a517bda0f386fe0b14560940184c302490f25a5245ee76dd12db0fb8f973f91
-size 130718568

fr_en/xnnpack/opus_mt_fr_en_xnnpack_fp32.pte

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:e33a9214cf4582420bf8960c14a45861f21643f70cd92e95a0f3c63dfea72a2c
-size 422579304