msluszniak commited on
Commit
6ef6540
·
verified ·
1 Parent(s): a3ab009

Unpublish variants that lose on device

Browse files
1_7b/vulkan/NOTES.md ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ Exported and not published. On a Galaxy S26 Ultra it decodes at 93 ms/token against
2
+ 28.5 ms for XNNPACK 8da8w, and its output is wrong: RMSNorm is decomposed in fp16 and
3
+ the residual stream's outliers (|x| up to 1749) overflow `x*x`. Use `xnnpack/` on Android.
1_7b/vulkan/config.json DELETED
Binary file (1.47 kB)
 
1_7b/vulkan/eurollm_1_7b_vulkan_int8.pte DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:d5661739af034088ec8b27c9af8de6de9d8bb3a85e0c3ffdb0a743bb38df5ea7
3
- size 1760632576
 
 
 
 
README.md CHANGED
@@ -19,7 +19,13 @@ Upstream model: [utter-project/EuroLLM-1.7B-Instruct](https://huggingface.co/utt
19
  | --- | --- | --- |
20
  | `1_7b/xnnpack/eurollm_1_7b_xnnpack_8da8w.pte` | xnnpack | 8da8w |
21
  | `1_7b/mlx/eurollm_1_7b_mlx_int8.pte` | mlx | int8 |
22
- | `1_7b/vulkan/eurollm_1_7b_vulkan_int8.pte` | vulkan | int8 |
 
 
 
 
 
 
23
 
24
  ## Repository structure
25
 
@@ -28,8 +34,7 @@ Upstream model: [utter-project/EuroLLM-1.7B-Instruct](https://huggingface.co/utt
28
  1_7b/mlx/eurollm_1_7b_mlx_int8.pte 1.7 GB
29
  1_7b/tokenizer.json 8.2 MB
30
  1_7b/tokenizer_config.json 47.4 kB
31
- 1_7b/vulkan/config.json 1.5 kB
32
- 1_7b/vulkan/eurollm_1_7b_vulkan_int8.pte 1.8 GB
33
  1_7b/xnnpack/config.json 1.5 kB
34
  1_7b/xnnpack/eurollm_1_7b_xnnpack_8da8w.pte 1.7 GB
35
  config.json 29 B
@@ -53,7 +58,7 @@ first.
53
 
54
  Weights are 8-bit: 4-bit (8da4w) lost 6-9 BLEU on WMT13 en->de and often failed
55
  to stop. XNNPACK uses dynamic int8 activations with per-channel int8 weights;
56
- MLX and Vulkan use int8 weight-only. For translation, prompt it as the model
57
  card does, without a BOS token:
58
 
59
  ```
 
19
  | --- | --- | --- |
20
  | `1_7b/xnnpack/eurollm_1_7b_xnnpack_8da8w.pte` | xnnpack | 8da8w |
21
  | `1_7b/mlx/eurollm_1_7b_mlx_int8.pte` | mlx | int8 |
22
+
23
+ ## Unshipped backends
24
+
25
+ A backend directory carrying a `NOTES.md` and no `.pte` was exported and
26
+ not published. Its note records why.
27
+
28
+ - `1_7b/vulkan` ([why](https://huggingface.co/software-mansion/react-native-executorch-eurollm/blob/main/1_7b/vulkan/NOTES.md))
29
 
30
  ## Repository structure
31
 
 
34
  1_7b/mlx/eurollm_1_7b_mlx_int8.pte 1.7 GB
35
  1_7b/tokenizer.json 8.2 MB
36
  1_7b/tokenizer_config.json 47.4 kB
37
+ 1_7b/vulkan/NOTES.md 261 B
 
38
  1_7b/xnnpack/config.json 1.5 kB
39
  1_7b/xnnpack/eurollm_1_7b_xnnpack_8da8w.pte 1.7 GB
40
  config.json 29 B
 
58
 
59
  Weights are 8-bit: 4-bit (8da4w) lost 6-9 BLEU on WMT13 en->de and often failed
60
  to stop. XNNPACK uses dynamic int8 activations with per-channel int8 weights;
61
+ MLX uses int8 weight-only. For translation, prompt it as the model
62
  card does, without a BOS token:
63
 
64
  ```