VibeVoice-ASR-BitNet-slim

A repack of microsoft/VibeVoice-ASR-BitNet with a redundant tensor removed, served by a substantially optimised CPU runtime. No retraining; the ternary transformer body is byte-for-byte the original.

	LM	VAE	Total	bits/weight (LM)
microsoft/VibeVoice-ASR-BitNet	992.9 MB	703.1 MB	1.70 GB	4.44
this repo	526.1 MB	703.1 MB	1.23 GB	2.69

What changed in the weights

The released LM GGUF stores output.weight as F16 (466.7 MB — 47% of the file) next to token_embd.weight as Q6_K. In the source checkpoint tie_word_embeddings is true and the two matrices are bit-identical, so the F16 tensor is the same matrix twice at higher precision. llama.cpp loads LLM_TENSOR_OUTPUT as optional and falls back to token_embd, so the duplicate is simply dropped: the output projection runs through the Q6_K copy. That is the only numerical change, and it also removes 466.7 MB from every decoded token's memory traffic.

Accuracy

FLEURS test clips, 24 per language, greedy decoding, scored corpus-level with digits and years spelled out on both sides (methodology and scripts: bench/README.md). Both models measured on the same deterministic runtime build:

Language	Released (1.70 GB)	Slim (1.23 GB)	Δ
Spanish	5.96	7.67	+1.70
English	6.48	6.48	0.00
Portuguese	9.23	8.90	−0.33
Italian	9.67	9.52	−0.16
German	13.41	15.51	+2.09
French	34.63	30.46	−4.17
corpus	13.95	13.65	−0.30

Corpus-level the slim model scores marginally better; per-language differences swing both ways by up to ±2–4 points on ~400-word samples, which is sampling noise, not a head-precision effect. Practical read: no measurable accuracy cost for the 27% size reduction.

The runtime is deterministic: identical transcripts at any thread count (this was not true of the original runtime; see the repo for the fix).

Languages

VibeVoice-ASR was trained on en, zh, fr, it, ko, pt, vi. Of the EU languages above, Spanish and German are out-of-distribution but generalise usably; French is in-distribution yet hard on FLEURS' register (heavy in proper nouns — the tech report's 17.4 on MLC-FR is a different corpus, not a contradiction). Other EU languages degrade sharply; this repack does not change language coverage.

Speed

Documented in the runtime repo, not here — it is a property of the code, and the code moved a lot: ~2.8× end to end versus the upstream runtime on a 4-core AVX-512 VM (compute RTF ≈ 0.37 for an 8 s clip), via VNNI kernels, a register-tiled INT8 GEMM, vectorised quantisation epilogues and a layout-native depthwise convolution. Tables, profiler methodology, and full reproduction scripts:

➡️ martinobettucci/VibeASR-bitnet.cpp — "CPU optimisation on AVX-512"

Usage

Drop-in for the released model — same runtime, same flags:

./build/bin/asr_infer \
    --vae-model vibeasr-vae-encoder-i8_s.gguf \
    --lm-model  vibeasr-lm-i2_s-tied.gguf \
    --audio input.wav -t 4 --greedy

Provenance

Produced with tools/requant_lm_head.cpp --drop from the repo above. The VAE encoder and tokenizer files are copied unmodified from upstream. Licensed MIT, as upstream.