Automatic Speech Recognition
GGUF
VibeVoice
ggml
ASR
quantization
cpu-inference
bitnet
conversational
Instructions to use P2Enjoy/VibeVoice-ASR-BitNet-slim with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf P2Enjoy/VibeVoice-ASR-BitNet-slim # Run inference directly in the terminal: llama cli -hf P2Enjoy/VibeVoice-ASR-BitNet-slim
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf P2Enjoy/VibeVoice-ASR-BitNet-slim # Run inference directly in the terminal: llama cli -hf P2Enjoy/VibeVoice-ASR-BitNet-slim
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf P2Enjoy/VibeVoice-ASR-BitNet-slim # Run inference directly in the terminal: ./build/bin/llama cli -hf P2Enjoy/VibeVoice-ASR-BitNet-slim
Use Docker
docker model run hf.co/P2Enjoy/VibeVoice-ASR-BitNet-slim
- Llama new
- LM Studio
- Jan
- Ollama
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with Ollama:
ollama run hf.co/P2Enjoy/VibeVoice-ASR-BitNet-slim
- Unsloth Desktop
- Docker Model Runner
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with Docker Model Runner:
docker model run hf.co/P2Enjoy/VibeVoice-ASR-BitNet-slim
- Lemonade
How to use P2Enjoy/VibeVoice-ASR-BitNet-slim with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull P2Enjoy/VibeVoice-ASR-BitNet-slim
Run and chat with the model
lemonade run user.VibeVoice-ASR-BitNet-slim-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
Residual-fusion default: +0.04 WER row, engine ratio ~3.0x upstream
Browse files
README.md
CHANGED
|
@@ -78,6 +78,7 @@ clip's language code (generous: this model runs unhinted):
|
|
| 78 |
|:--|--:|--:|--:|--:|--:|--:|--:|--:|
|
| 79 |
| Upstream engine, original 1.70 GB | 16.7 | 6.6 | 5.8 | 32.7 | 21.6 | 8.3 | 10.8 | **15.75** |
|
| 80 |
| **This repo (slim), fork engine** | 13.9 | 7.2 | 5.8 | 35.4 | 21.2 | 10.2 | 10.5 | **16.03** |
|
|
|
|
| 81 |
| whisper.cpp small q5_1 | 8.5 | 4.6 | 7.1 | 12.9 | 16.7 | 6.6 | 8.0 | **9.92** |
|
| 82 |
| whisper.cpp large-v3-turbo q5_0 | 4.1 | 3.7 | 4.0 | 3.3 | 10.3 | 3.3 | 4.3 | **5.12** |
|
| 83 |
|
|
@@ -109,9 +110,10 @@ languages degrade sharply; this repack does not change language coverage.
|
|
| 109 |
|
| 110 |
Documented in the runtime repo, not here — it is a property of the code, and
|
| 111 |
absolute RTF is a property of the host, so speed is published as **ratios**
|
| 112 |
-
between engines measured back-to-back: **
|
| 113 |
-
(paired per-clip median
|
| 114 |
-
|
|
|
|
| 115 |
with a packed-B INT8 GEMM (fused dequantisation epilogue), conv-as-GEMM
|
| 116 |
downsampling, and a layout-native depthwise convolution. Tables, profiler
|
| 117 |
methodology, and full reproduction scripts:
|
|
|
|
| 78 |
|:--|--:|--:|--:|--:|--:|--:|--:|--:|
|
| 79 |
| Upstream engine, original 1.70 GB | 16.7 | 6.6 | 5.8 | 32.7 | 21.6 | 8.3 | 10.8 | **15.75** |
|
| 80 |
| **This repo (slim), fork engine** | 13.9 | 7.2 | 5.8 | 35.4 | 21.2 | 10.2 | 10.5 | **16.03** |
|
| 81 |
+
| This repo, fork engine + residual fusion (default) | 15.6 | 6.6 | 7.1 | 33.6 | 21.9 | 8.6 | 11.1 | **16.07** |
|
| 82 |
| whisper.cpp small q5_1 | 8.5 | 4.6 | 7.1 | 12.9 | 16.7 | 6.6 | 8.0 | **9.92** |
|
| 83 |
| whisper.cpp large-v3-turbo q5_0 | 4.1 | 3.7 | 4.0 | 3.3 | 10.3 | 3.3 | 4.3 | **5.12** |
|
| 84 |
|
|
|
|
| 110 |
|
| 111 |
Documented in the runtime repo, not here — it is a property of the code, and
|
| 112 |
absolute RTF is a property of the host, so speed is published as **ratios**
|
| 113 |
+
between engines measured back-to-back: **≈3.0× faster than the upstream runtime**
|
| 114 |
+
(2.56× paired per-clip median over 100 clips, × 1.17× from the residual-fusion
|
| 115 |
+
default gated at +0.04 WER above), ~1.25× faster than whisper-small overall and
|
| 116 |
+
1.76× faster below 8 s of audio even before fusion. Achieved via VNNI kernels
|
| 117 |
with a packed-B INT8 GEMM (fused dequantisation epilogue), conv-as-GEMM
|
| 118 |
downsampling, and a layout-native depthwise convolution. Tables, profiler
|
| 119 |
methodology, and full reproduction scripts:
|