mbCrypto commited on
Commit
443b385
·
verified ·
1 Parent(s): dd67234

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +89 -0
README.md ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ - fr
5
+ - it
6
+ - pt
7
+ - es
8
+ - de
9
+ license: mit
10
+ pipeline_tag: automatic-speech-recognition
11
+ tags:
12
+ - ASR
13
+ - quantization
14
+ - cpu-inference
15
+ - gguf
16
+ - bitnet
17
+ library_name: ggml
18
+ base_model: microsoft/VibeVoice-ASR-BitNet
19
+ ---
20
+
21
+ # VibeVoice-ASR-BitNet-slim
22
+
23
+ A repack of [microsoft/VibeVoice-ASR-BitNet](https://huggingface.co/microsoft/VibeVoice-ASR-BitNet)
24
+ with a redundant tensor removed. **No retraining, no re-quantisation of the ternary
25
+ weights** — the transformer body is byte-for-byte the original.
26
+
27
+ | | LM | VAE | Total |
28
+ |:--|--:|--:|--:|
29
+ | microsoft/VibeVoice-ASR-BitNet | 992.9 MB | 703.1 MB | 1.70 GB |
30
+ | this repo | **526.1 MB** | 703.1 MB | **1.23 GB** |
31
+
32
+ ## What changed
33
+
34
+ The released LM GGUF carries `output.weight` as **F16, 466.7 MB — 47% of the file**,
35
+ next to `token_embd.weight` already stored as Q6_K at 191.4 MB.
36
+
37
+ In the source checkpoint `tie_word_embeddings` is `true`, and `lm_head.weight` is
38
+ **bit-identical** to `embed_tokens.weight`. So the F16 tensor is the same matrix a
39
+ second time, at higher precision than the copy the file already holds.
40
+
41
+ llama.cpp loads `LLM_TENSOR_OUTPUT` as `TENSOR_NOT_REQUIRED` and falls back to
42
+ `token_embd` when it is absent, so the tensor can simply be dropped. The output
43
+ projection then runs through the Q6_K copy instead of the F16 one — the only
44
+ numerical change in this repack.
45
+
46
+ Removing it also takes 466.7 MB off the memory read on **every decoded token**, which
47
+ was roughly half the LM's per-token bandwidth.
48
+
49
+ ### Bit budget
50
+
51
+ | Component | Type | Weights | MB | bits/wt |
52
+ |:--|:--|--:|--:|--:|
53
+ | transformer body | I2_S | 1,310,195,712 | 327.6 | 2.00 |
54
+ | token embedding | Q6_K | 233,373,696 | 191.4 | 6.56 |
55
+ | norms / biases | F32 | 144,896 | 0.6 | 32.00 |
56
+ | **total** | | 1,543,714,304 | **519.6** | **2.69** |
57
+
58
+ The released file is 4.44 bits/weight overall; this one is 2.69. Note that the
59
+ ternary body is packed at exactly **2.000** bits/weight, not log₂3 = 1.585 — I2_S
60
+ stores four ternary values per byte and leaves one of four codes unused, which is
61
+ 68 MB of padding (20.8% of the body).
62
+
63
+ ## Usage
64
+
65
+ Drop-in for the released model — same runtime, same flags:
66
+
67
+ ```bash
68
+ ./build/bin/asr_infer \
69
+ --vae-model vibeasr-vae-encoder-i8_s.gguf \
70
+ --lm-model vibeasr-lm-i2_s-tied.gguf \
71
+ --audio input.wav -t 4 --greedy
72
+ ```
73
+
74
+ ## Languages
75
+
76
+ VibeVoice-ASR was trained on **en, zh, fr, it, ko, pt, vi**. Among EU official
77
+ languages that means English, French, Italian and Portuguese are in-distribution;
78
+ Spanish and German are not in the training mix but generalise usably. The other EU
79
+ languages degrade sharply and this repack does not change that — it is a packaging
80
+ fix, not a capability change.
81
+
82
+ ## Provenance
83
+
84
+ Produced with `tools/requant_lm_head.cpp --drop` from
85
+ [martinobettucci/VibeASR-bitnet.cpp](https://github.com/martinobettucci/VibeASR-bitnet.cpp),
86
+ branch `claude/asr-cpu-optimization-cztnh9`. The VAE encoder and tokenizer files are
87
+ copied unmodified from the upstream repo.
88
+
89
+ Licensed MIT, as upstream.