Automatic Speech Recognition
GGUF
VibeVoice
ggml
ASR
quantization
cpu-inference
bitnet
conversational
Instructions to use P2Enjoy/VibeVoice-ASR-BitNet-slim with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf P2Enjoy/VibeVoice-ASR-BitNet-slim # Run inference directly in the terminal: llama cli -hf P2Enjoy/VibeVoice-ASR-BitNet-slim
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf P2Enjoy/VibeVoice-ASR-BitNet-slim # Run inference directly in the terminal: llama cli -hf P2Enjoy/VibeVoice-ASR-BitNet-slim
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf P2Enjoy/VibeVoice-ASR-BitNet-slim # Run inference directly in the terminal: ./build/bin/llama cli -hf P2Enjoy/VibeVoice-ASR-BitNet-slim
Use Docker
docker model run hf.co/P2Enjoy/VibeVoice-ASR-BitNet-slim
- Llama new
- LM Studio
- Jan
- Ollama
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with Ollama:
ollama run hf.co/P2Enjoy/VibeVoice-ASR-BitNet-slim
- Unsloth Desktop
- Docker Model Runner
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with Docker Model Runner:
docker model run hf.co/P2Enjoy/VibeVoice-ASR-BitNet-slim
- Lemonade
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull P2Enjoy/VibeVoice-ASR-BitNet-slim
Run and chat with the model
lemonade run user.VibeVoice-ASR-BitNet-slim-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
Final benchmark table: WER + speed over upstream baseline, all engines
Browse files
README.md
CHANGED
|
@@ -74,22 +74,28 @@ difficulty, not quantisation damage.
|
|
| 74 |
back-to-back on one host at 4 threads, later runtime build, whisper given each
|
| 75 |
clip's language code (generous: this model runs unhinted):
|
| 76 |
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
|
| 81 |
-
|
|
| 82 |
-
|
|
| 83 |
-
|
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
than
|
| 92 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 93 |
[runtime repo](https://github.com/martinobettucci/VibeASR-bitnet.cpp/tree/claude/asr-cpu-optimization-cztnh9)
|
| 94 |
and are not duplicated here.
|
| 95 |
|
|
|
|
| 74 |
back-to-back on one host at 4 threads, later runtime build, whisper given each
|
| 75 |
clip's language code (generous: this model runs unhinted):
|
| 76 |
|
| 77 |
+
Speed is given as a ratio over the upstream runtime (absolute RTF is a property
|
| 78 |
+
of the benchmark host, not of the model); > 1 is faster than upstream.
|
| 79 |
+
|
| 80 |
+
| Engine / weights | de | en | es | fr | fr-MLS | it | pt | **all** | **speed** |
|
| 81 |
+
|:--|--:|--:|--:|--:|--:|--:|--:|--:|--:|
|
| 82 |
+
| Upstream engine, original 1.70 GB | 16.7 | 6.6 | 5.8 | 32.7 | 21.6 | 8.3 | 10.8 | **15.75** | 1.00× |
|
| 83 |
+
| **This repo (slim) + fork engine, defaults** | 15.6 | 6.6 | 7.1 | 33.6 | 21.9 | 8.6 | 11.1 | **16.07** | **2.35×** |
|
| 84 |
+
| This repo, fork engine bit-exact mode | 13.9 | 7.2 | 5.8 | 35.4 | 21.2 | 10.2 | 10.5 | **16.03** | **2.01×** |
|
| 85 |
+
| whisper.cpp small q5_1 | 8.5 | 4.6 | 7.1 | 12.9 | 16.7 | 6.6 | 8.0 | **9.92** | **3.06×** |
|
| 86 |
+
| whisper.cpp large-v3-turbo q5_0 | 4.1 | 3.7 | 4.0 | 3.3 | 10.3 | 3.3 | 4.3 | **5.12** | **0.75×** |
|
| 87 |
+
|
| 88 |
+
Read it straight. Slim-vs-original is parity (+0.32, per-language scatter both
|
| 89 |
+
ways — German improves under the fork's overflow-fixed kernels, Spanish
|
| 90 |
+
regresses). But **on this corpus whisper-small is both more accurate and ~1.3×
|
| 91 |
+
faster** than this model, and turbo is far more accurate again. These clips are
|
| 92 |
+
5–25 s, which is whisper's ideal case and this architecture's worst: it never
|
| 93 |
+
exercises what VibeVoice-ASR is built for — single-pass long-form (an hour of
|
| 94 |
+
audio is one encode, not ~120 stitched 30 s windows), native speaker/segment
|
| 95 |
+
JSON, and decoder-level hotword biasing. Choose accordingly: for short-clip
|
| 96 |
+
transcription accuracy, use whisper; for long recordings needing speakers,
|
| 97 |
+
timestamps and domain-term biasing in one CPU pass, this is the cheaper stack.
|
| 98 |
+
Speed methodology and reproduction scripts live in the
|
| 99 |
[runtime repo](https://github.com/martinobettucci/VibeASR-bitnet.cpp/tree/claude/asr-cpu-optimization-cztnh9)
|
| 100 |
and are not duplicated here.
|
| 101 |
|