msluszniak commited on
Commit
11dd3b8
·
verified ·
1 Parent(s): 7564c16

Model card: list the XNNPACK int8 variants and their accuracy

Browse files
Files changed (1) hide show
  1. README.md +13 -0
README.md CHANGED
@@ -22,6 +22,18 @@ Upstream model: [whisper-base](https://huggingface.co/openai/whisper-base)
22
  | `vulkan/whisper_base_vulkan_fp16.pte` | vulkan | fp16 |
23
  | `vulkan/whisper_base_vulkan_int8.pte` | vulkan | int8 |
24
  | `xnnpack/whisper_base_xnnpack_fp32.pte` | xnnpack | fp32 |
 
 
 
 
 
 
 
 
 
 
 
 
25
 
26
  ## Repository structure
27
 
@@ -38,6 +50,7 @@ vulkan/whisper_base_vulkan_fp16.pte 335 MB
38
  vulkan/whisper_base_vulkan_int8.pte 196 MB
39
  xnnpack/config.json 1.4 kB
40
  xnnpack/whisper_base_xnnpack_fp32.pte 380 MB
 
41
  ```
42
 
43
  ## Compatibility
 
22
  | `vulkan/whisper_base_vulkan_fp16.pte` | vulkan | fp16 |
23
  | `vulkan/whisper_base_vulkan_int8.pte` | vulkan | int8 |
24
  | `xnnpack/whisper_base_xnnpack_fp32.pte` | xnnpack | fp32 |
25
+ | `xnnpack/whisper_base_xnnpack_int8.pte` | xnnpack | int8 |
26
+
27
+ ## XNNPACK int8
28
+
29
+ `xnnpack/whisper_base_xnnpack_int8.pte` is 8da8w (int8 dynamic activations, int8
30
+ per-channel weights). Greedy WER (CER for zh) against the fp32 file, LibriSpeech
31
+ test-clean (150 clips) and FLEURS (80 per language):
32
+
33
+ | | en | de | es | pl | fr | zh |
34
+ | --- | --- | --- | --- | --- | --- | --- |
35
+ | fp32 | 4.12 | 24.88 | 13.66 | 47.35 | 40.76 | 21.27 |
36
+ | int8 | 3.94 | 25.04 | 13.87 | 46.39 | 41.09 | 21.27 |
37
 
38
  ## Repository structure
39
 
 
50
  vulkan/whisper_base_vulkan_int8.pte 196 MB
51
  xnnpack/config.json 1.4 kB
52
  xnnpack/whisper_base_xnnpack_fp32.pte 380 MB
53
+ xnnpack/whisper_base_xnnpack_int8.pte 254 MB
54
  ```
55
 
56
  ## Compatibility