Automatic Speech Recognition
GGUF
VibeVoice
ggml
ASR
quantization
cpu-inference
bitnet
conversational
Instructions to use P2Enjoy/VibeVoice-ASR-BitNet-slim with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf P2Enjoy/VibeVoice-ASR-BitNet-slim # Run inference directly in the terminal: llama cli -hf P2Enjoy/VibeVoice-ASR-BitNet-slim
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf P2Enjoy/VibeVoice-ASR-BitNet-slim # Run inference directly in the terminal: llama cli -hf P2Enjoy/VibeVoice-ASR-BitNet-slim
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf P2Enjoy/VibeVoice-ASR-BitNet-slim # Run inference directly in the terminal: ./build/bin/llama cli -hf P2Enjoy/VibeVoice-ASR-BitNet-slim
Use Docker
docker model run hf.co/P2Enjoy/VibeVoice-ASR-BitNet-slim
- Llama new
- LM Studio
- Jan
- Ollama
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with Ollama:
ollama run hf.co/P2Enjoy/VibeVoice-ASR-BitNet-slim
- Unsloth Desktop
- Docker Model Runner
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with Docker Model Runner:
docker model run hf.co/P2Enjoy/VibeVoice-ASR-BitNet-slim
- Lemonade
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull P2Enjoy/VibeVoice-ASR-BitNet-slim
Run and chat with the model
lemonade run user.VibeVoice-ASR-BitNet-slim-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,89 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
- fr
|
| 5 |
+
- it
|
| 6 |
+
- pt
|
| 7 |
+
- es
|
| 8 |
+
- de
|
| 9 |
+
license: mit
|
| 10 |
+
pipeline_tag: automatic-speech-recognition
|
| 11 |
+
tags:
|
| 12 |
+
- ASR
|
| 13 |
+
- quantization
|
| 14 |
+
- cpu-inference
|
| 15 |
+
- gguf
|
| 16 |
+
- bitnet
|
| 17 |
+
library_name: ggml
|
| 18 |
+
base_model: microsoft/VibeVoice-ASR-BitNet
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
+
# VibeVoice-ASR-BitNet-slim
|
| 22 |
+
|
| 23 |
+
A repack of [microsoft/VibeVoice-ASR-BitNet](https://huggingface.co/microsoft/VibeVoice-ASR-BitNet)
|
| 24 |
+
with a redundant tensor removed. **No retraining, no re-quantisation of the ternary
|
| 25 |
+
weights** — the transformer body is byte-for-byte the original.
|
| 26 |
+
|
| 27 |
+
| | LM | VAE | Total |
|
| 28 |
+
|:--|--:|--:|--:|
|
| 29 |
+
| microsoft/VibeVoice-ASR-BitNet | 992.9 MB | 703.1 MB | 1.70 GB |
|
| 30 |
+
| this repo | **526.1 MB** | 703.1 MB | **1.23 GB** |
|
| 31 |
+
|
| 32 |
+
## What changed
|
| 33 |
+
|
| 34 |
+
The released LM GGUF carries `output.weight` as **F16, 466.7 MB — 47% of the file**,
|
| 35 |
+
next to `token_embd.weight` already stored as Q6_K at 191.4 MB.
|
| 36 |
+
|
| 37 |
+
In the source checkpoint `tie_word_embeddings` is `true`, and `lm_head.weight` is
|
| 38 |
+
**bit-identical** to `embed_tokens.weight`. So the F16 tensor is the same matrix a
|
| 39 |
+
second time, at higher precision than the copy the file already holds.
|
| 40 |
+
|
| 41 |
+
llama.cpp loads `LLM_TENSOR_OUTPUT` as `TENSOR_NOT_REQUIRED` and falls back to
|
| 42 |
+
`token_embd` when it is absent, so the tensor can simply be dropped. The output
|
| 43 |
+
projection then runs through the Q6_K copy instead of the F16 one — the only
|
| 44 |
+
numerical change in this repack.
|
| 45 |
+
|
| 46 |
+
Removing it also takes 466.7 MB off the memory read on **every decoded token**, which
|
| 47 |
+
was roughly half the LM's per-token bandwidth.
|
| 48 |
+
|
| 49 |
+
### Bit budget
|
| 50 |
+
|
| 51 |
+
| Component | Type | Weights | MB | bits/wt |
|
| 52 |
+
|:--|:--|--:|--:|--:|
|
| 53 |
+
| transformer body | I2_S | 1,310,195,712 | 327.6 | 2.00 |
|
| 54 |
+
| token embedding | Q6_K | 233,373,696 | 191.4 | 6.56 |
|
| 55 |
+
| norms / biases | F32 | 144,896 | 0.6 | 32.00 |
|
| 56 |
+
| **total** | | 1,543,714,304 | **519.6** | **2.69** |
|
| 57 |
+
|
| 58 |
+
The released file is 4.44 bits/weight overall; this one is 2.69. Note that the
|
| 59 |
+
ternary body is packed at exactly **2.000** bits/weight, not log₂3 = 1.585 — I2_S
|
| 60 |
+
stores four ternary values per byte and leaves one of four codes unused, which is
|
| 61 |
+
68 MB of padding (20.8% of the body).
|
| 62 |
+
|
| 63 |
+
## Usage
|
| 64 |
+
|
| 65 |
+
Drop-in for the released model — same runtime, same flags:
|
| 66 |
+
|
| 67 |
+
```bash
|
| 68 |
+
./build/bin/asr_infer \
|
| 69 |
+
--vae-model vibeasr-vae-encoder-i8_s.gguf \
|
| 70 |
+
--lm-model vibeasr-lm-i2_s-tied.gguf \
|
| 71 |
+
--audio input.wav -t 4 --greedy
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
## Languages
|
| 75 |
+
|
| 76 |
+
VibeVoice-ASR was trained on **en, zh, fr, it, ko, pt, vi**. Among EU official
|
| 77 |
+
languages that means English, French, Italian and Portuguese are in-distribution;
|
| 78 |
+
Spanish and German are not in the training mix but generalise usably. The other EU
|
| 79 |
+
languages degrade sharply and this repack does not change that — it is a packaging
|
| 80 |
+
fix, not a capability change.
|
| 81 |
+
|
| 82 |
+
## Provenance
|
| 83 |
+
|
| 84 |
+
Produced with `tools/requant_lm_head.cpp --drop` from
|
| 85 |
+
[martinobettucci/VibeASR-bitnet.cpp](https://github.com/martinobettucci/VibeASR-bitnet.cpp),
|
| 86 |
+
branch `claude/asr-cpu-optimization-cztnh9`. The VAE encoder and tokenizer files are
|
| 87 |
+
copied unmodified from the upstream repo.
|
| 88 |
+
|
| 89 |
+
Licensed MIT, as upstream.
|