Commit ·
c5615d7
1
Parent(s): 1cd2d0f
Quantize the whole decoder in the int8 variant
Browse filesThe int8 build quantized only the tied logit head, leaving the rest of the
decoder in fp32. Quantizing every decode-side linear takes the file from
154 MB to 130 MB with a transcript identical to the fp32 decoder.
Encode is untouched and its output is bit-identical to the fp16 variant:
the cross-attention key and value projections are excluded, since the
encoder hoists them and they run in the Vulkan partition.
vulkan/whisper_tiny_vulkan_int8.pte
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1953ba74947ee23d5f076fa8a198011bd0d6213e5a5e615ee4696ebb418fe20a
|
| 3 |
+
size 129521380
|