Automatic Speech Recognition
GGUF
VibeVoice
ggml
ASR
quantization
cpu-inference
bitnet
conversational
Instructions to use P2Enjoy/VibeVoice-ASR-BitNet-slim with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf P2Enjoy/VibeVoice-ASR-BitNet-slim # Run inference directly in the terminal: llama cli -hf P2Enjoy/VibeVoice-ASR-BitNet-slim
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf P2Enjoy/VibeVoice-ASR-BitNet-slim # Run inference directly in the terminal: llama cli -hf P2Enjoy/VibeVoice-ASR-BitNet-slim
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf P2Enjoy/VibeVoice-ASR-BitNet-slim # Run inference directly in the terminal: ./build/bin/llama cli -hf P2Enjoy/VibeVoice-ASR-BitNet-slim
Use Docker
docker model run hf.co/P2Enjoy/VibeVoice-ASR-BitNet-slim
- Llama new
- LM Studio
- Jan
- Ollama
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with Ollama:
ollama run hf.co/P2Enjoy/VibeVoice-ASR-BitNet-slim
- Unsloth Desktop
- Docker Model Runner
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with Docker Model Runner:
docker model run hf.co/P2Enjoy/VibeVoice-ASR-BitNet-slim
- Lemonade
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull P2Enjoy/VibeVoice-ASR-BitNet-slim
Run and chat with the model
lemonade run user.VibeVoice-ASR-BitNet-slim-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
Correction: long-form is this model's strength, not its weakness (WER 8.88 vs whisper-small 27.57)
Browse files
README.md
CHANGED
|
@@ -85,28 +85,42 @@ of the benchmark host, not of the model); > 1 is faster than upstream.
|
|
| 85 |
| whisper.cpp small q5_1 | 8.5 | 4.6 | 7.1 | 12.9 | 16.7 | 6.6 | 8.0 | **9.92** | **3.06Γ** |
|
| 86 |
| whisper.cpp large-v3-turbo q5_0 | 4.1 | 3.7 | 4.0 | 3.3 | 10.3 | 3.3 | 4.3 | **5.12** | **0.75Γ** |
|
| 87 |
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
**
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 110 |
|
| 111 |
The runtime also ships opt-in decoder-level hotword biasing (`--hotwords`,
|
| 112 |
token-trie logit boosting): on FLEURS-French with oracle terms it recovers ~4 WER
|
|
|
|
| 85 |
| whisper.cpp small q5_1 | 8.5 | 4.6 | 7.1 | 12.9 | 16.7 | 6.6 | 8.0 | **9.92** | **3.06Γ** |
|
| 86 |
| whisper.cpp large-v3-turbo q5_0 | 4.1 | 3.7 | 4.0 | 3.3 | 10.3 | 3.3 | 4.3 | **5.12** | **0.75Γ** |
|
| 87 |
|
| 88 |
+
Slim-vs-original is parity (+0.32, per-language scatter both ways β German
|
| 89 |
+
improves under the fork's overflow-fixed kernels, Spanish regresses). On these
|
| 90 |
+
**short clips** whisper-small is more accurate and ~1.3Γ faster.
|
| 91 |
+
|
| 92 |
+
**But short clips are the wrong test for this architecture, and the difference is
|
| 93 |
+
dramatic.** VibeVoice-ASR compresses audio 3200Γ into an LLM context so a whole
|
| 94 |
+
recording is one encode with full context available to the decoder. On 76 s of
|
| 95 |
+
continuous French audiobook speech, same host, back to back:
|
| 96 |
+
|
| 97 |
+
| Engine | RTF | **WER** |
|
| 98 |
+
|:--|--:|--:|
|
| 99 |
+
| **This model, fork runtime** | 0.44 | **8.88** |
|
| 100 |
+
| whisper.cpp small q5_1 | 0.30 | 27.57 |
|
| 101 |
+
| whisper.cpp large-v3-turbo q5 | 0.99 | 5.61 |
|
| 102 |
+
|
| 103 |
+
The ranking inverts: **8.88 against whisper-small's 27.57**, a 3Γ gap the other
|
| 104 |
+
way, and within 3 points of large-v3-turbo at 2.3Γ less compute. The same weights
|
| 105 |
+
score ~21 on this corpus's short clips β more context makes this model better,
|
| 106 |
+
which is the point of the architecture.
|
| 107 |
+
|
| 108 |
+
**Known limit: ~80 seconds.** Past that the decoder emits its end token early and
|
| 109 |
+
drops the tail (measured: 101 s of audio β 119 of 280 reference words), with the
|
| 110 |
+
token budget 98% unused. It is a property of the weights, not the runtime β
|
| 111 |
+
upstream behaves identically. Segment recordings longer than ~75 s.
|
| 112 |
+
|
| 113 |
+
Two things this model does **not** do, despite the runtime supporting the format:
|
| 114 |
+
it emits **no speaker labels**, and the `{Start, End, Speaker, Content}` JSON
|
| 115 |
+
prompt is meant for the 7B model β asking this 1.5B for it costs ~1.4 WER. Use
|
| 116 |
+
plain text.
|
| 117 |
+
|
| 118 |
+
Guidance: **use whisper for short-clip transcription; use this for continuous
|
| 119 |
+
speech in the 30β75 s range**, where it is far more accurate than whisper-small at
|
| 120 |
+
comparable cost and near turbo at a fraction of it β plus decoder-level hotword
|
| 121 |
+
biasing (FLEURS-French 36.0 β 31.9 with domain terms). Methodology and
|
| 122 |
+
reproduction scripts:
|
| 123 |
+
[runtime repo](https://github.com/martinobettucci/VibeASR-bitnet.cpp/tree/claude/asr-cpu-optimization-cztnh9).
|
| 124 |
|
| 125 |
The runtime also ships opt-in decoder-level hotword biasing (`--hotwords`,
|
| 126 |
token-trie logit boosting): on FLEURS-French with oracle terms it recovers ~4 WER
|