Unpublish variants that lose on device
Browse files- 1_7b/vulkan/NOTES.md +3 -0
- 1_7b/vulkan/config.json +0 -0
- 1_7b/vulkan/eurollm_1_7b_vulkan_int8.pte +0 -3
- README.md +9 -4
1_7b/vulkan/NOTES.md
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Exported and not published. On a Galaxy S26 Ultra it decodes at 93 ms/token against
|
| 2 |
+
28.5 ms for XNNPACK 8da8w, and its output is wrong: RMSNorm is decomposed in fp16 and
|
| 3 |
+
the residual stream's outliers (|x| up to 1749) overflow `x*x`. Use `xnnpack/` on Android.
|
1_7b/vulkan/config.json
DELETED
|
Binary file (1.47 kB)
|
|
|
1_7b/vulkan/eurollm_1_7b_vulkan_int8.pte
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:d5661739af034088ec8b27c9af8de6de9d8bb3a85e0c3ffdb0a743bb38df5ea7
|
| 3 |
-
size 1760632576
|
|
|
|
|
|
|
|
|
|
|
|
README.md
CHANGED
|
@@ -19,7 +19,13 @@ Upstream model: [utter-project/EuroLLM-1.7B-Instruct](https://huggingface.co/utt
|
|
| 19 |
| --- | --- | --- |
|
| 20 |
| `1_7b/xnnpack/eurollm_1_7b_xnnpack_8da8w.pte` | xnnpack | 8da8w |
|
| 21 |
| `1_7b/mlx/eurollm_1_7b_mlx_int8.pte` | mlx | int8 |
|
| 22 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
## Repository structure
|
| 25 |
|
|
@@ -28,8 +34,7 @@ Upstream model: [utter-project/EuroLLM-1.7B-Instruct](https://huggingface.co/utt
|
|
| 28 |
1_7b/mlx/eurollm_1_7b_mlx_int8.pte 1.7 GB
|
| 29 |
1_7b/tokenizer.json 8.2 MB
|
| 30 |
1_7b/tokenizer_config.json 47.4 kB
|
| 31 |
-
1_7b/vulkan/
|
| 32 |
-
1_7b/vulkan/eurollm_1_7b_vulkan_int8.pte 1.8 GB
|
| 33 |
1_7b/xnnpack/config.json 1.5 kB
|
| 34 |
1_7b/xnnpack/eurollm_1_7b_xnnpack_8da8w.pte 1.7 GB
|
| 35 |
config.json 29 B
|
|
@@ -53,7 +58,7 @@ first.
|
|
| 53 |
|
| 54 |
Weights are 8-bit: 4-bit (8da4w) lost 6-9 BLEU on WMT13 en->de and often failed
|
| 55 |
to stop. XNNPACK uses dynamic int8 activations with per-channel int8 weights;
|
| 56 |
-
MLX
|
| 57 |
card does, without a BOS token:
|
| 58 |
|
| 59 |
```
|
|
|
|
| 19 |
| --- | --- | --- |
|
| 20 |
| `1_7b/xnnpack/eurollm_1_7b_xnnpack_8da8w.pte` | xnnpack | 8da8w |
|
| 21 |
| `1_7b/mlx/eurollm_1_7b_mlx_int8.pte` | mlx | int8 |
|
| 22 |
+
|
| 23 |
+
## Unshipped backends
|
| 24 |
+
|
| 25 |
+
A backend directory carrying a `NOTES.md` and no `.pte` was exported and
|
| 26 |
+
not published. Its note records why.
|
| 27 |
+
|
| 28 |
+
- `1_7b/vulkan` ([why](https://huggingface.co/software-mansion/react-native-executorch-eurollm/blob/main/1_7b/vulkan/NOTES.md))
|
| 29 |
|
| 30 |
## Repository structure
|
| 31 |
|
|
|
|
| 34 |
1_7b/mlx/eurollm_1_7b_mlx_int8.pte 1.7 GB
|
| 35 |
1_7b/tokenizer.json 8.2 MB
|
| 36 |
1_7b/tokenizer_config.json 47.4 kB
|
| 37 |
+
1_7b/vulkan/NOTES.md 261 B
|
|
|
|
| 38 |
1_7b/xnnpack/config.json 1.5 kB
|
| 39 |
1_7b/xnnpack/eurollm_1_7b_xnnpack_8da8w.pte 1.7 GB
|
| 40 |
config.json 29 B
|
|
|
|
| 58 |
|
| 59 |
Weights are 8-bit: 4-bit (8da4w) lost 6-9 BLEU on WMT13 en->de and often failed
|
| 60 |
to stop. XNNPACK uses dynamic int8 activations with per-channel int8 weights;
|
| 61 |
+
MLX uses int8 weight-only. For translation, prompt it as the model
|
| 62 |
card does, without a BOS token:
|
| 63 |
|
| 64 |
```
|