Instructions to use prithivMLmods/VibeVoice-ASR-HF-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use prithivMLmods/VibeVoice-ASR-HF-GGUF with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("prithivMLmods/VibeVoice-ASR-HF-GGUF", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use prithivMLmods/VibeVoice-ASR-HF-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf prithivMLmods/VibeVoice-ASR-HF-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf prithivMLmods/VibeVoice-ASR-HF-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf prithivMLmods/VibeVoice-ASR-HF-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf prithivMLmods/VibeVoice-ASR-HF-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf prithivMLmods/VibeVoice-ASR-HF-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf prithivMLmods/VibeVoice-ASR-HF-GGUF:Q4_K_M
Use Docker
docker model run hf.co/prithivMLmods/VibeVoice-ASR-HF-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use prithivMLmods/VibeVoice-ASR-HF-GGUF with Ollama:
ollama run hf.co/prithivMLmods/VibeVoice-ASR-HF-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use prithivMLmods/VibeVoice-ASR-HF-GGUF with Docker Model Runner:
docker model run hf.co/prithivMLmods/VibeVoice-ASR-HF-GGUF:Q4_K_M
- Lemonade
How to use prithivMLmods/VibeVoice-ASR-HF-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull prithivMLmods/VibeVoice-ASR-HF-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.VibeVoice-ASR-HF-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
VibeVoice-ASR-HF-GGUF
VibeVoice-ASR (Transformers-compatible version,
microsoft/VibeVoice-ASR-HF) is Microsoft Research's MIT-licensed unified speech-to-text model. It processes up to 60 minutes of continuous audio in a single pass within a 64K-token context, so speaker tracking and semantic coherence hold across the whole recording. It jointly performs transcription, speaker diarization, and timestamping, and outputs JSON-like structured text giving who (Speaker), when (Start/End), and what (Content). It handles over 50 languages without an explicit language setting, including code-switching, and accepts optional context or hotwords such as names and technical terms to improve accuracy on domain-specific content. To handle long audio, it chunks the input into 60-second segments and caches convolution states between them. The chunk size can be reduced throughtokenizer_chunk_sizeif memory is tight. It is available in Transformers v5.3.0 and later throughVibeVoiceAsrForConditionalGeneration. It supports batch inference, chat-template inputs, training with the model's built-in loss,torch.compile, and pipeline usage. On the Open ASR Leaderboard it averages 7.77% WER (for example 2.20% on LibriSpeech test-clean and 17.20% on AMI) with an RTFx of 51.8. The technical report also gives diarization (DER) and speaker-attributed WER (cpWER, tcpWER) results.
Model Files
| File Name | Quant Type | File Size | File Link | Description |
|---|---|---|---|---|
| VibeVoice-ASR-HF.BF16.gguf | BF16 | 15.2 GB | Link | Full BF16 weights. Highest quality, largest file size. |
| VibeVoice-ASR-HF.Q3_K_L.gguf | Q3_K_L | 4.09 GB | Link | Lower quality but usable, good for low RAM availability. |
| VibeVoice-ASR-HF.Q3_K_M.gguf | Q3_K_M | 3.81 GB | Link | Low quality, smallest file size. |
| VibeVoice-ASR-HF.Q4_K_M.gguf | Q4_K_M | 4.68 GB | Link | Good quality, default size for most use cases, recommended. |
| VibeVoice-ASR-HF.Q4_K_S.gguf | Q4_K_S | 4.46 GB | Link | Slightly lower quality with more space savings, recommended. |
| VibeVoice-ASR-HF.Q5_K_M.gguf | Q5_K_M | 5.44 GB | Link | High quality, recommended. |
| VibeVoice-ASR-HF.Q5_K_S.gguf | Q5_K_S | 5.32 GB | Link | High quality with slight space savings, recommended. |
| VibeVoice-ASR-HF.Q6_K.gguf | Q6_K | 6.25 GB | Link | Very high quality, near perfect, recommended. |
| VibeVoice-ASR-HF.Q8_0.gguf | Q8_0 | 8.1 GB | Link | Extremely high quality, generally unneeded but max available quant. |
| VibeVoice-ASR-HF.mmproj-bf16.gguf | mmproj-bf16 | 1.43 GB | Link | Multimodal projection file in BF16 format. Used for audio/ASR capabilities. |
Install dependencies and clone llama.cpp
!pip install transformers huggingface-hub
!git clone https://github.com/ggml-org/llama.cpp.git
%cd /data/llama.cpp
Download GGUF model files
Download the quantized Q5_K_M model and multimodal projector into ./model/VibeVoice-ASR-HF-GGUF:
from huggingface_hub import snapshot_download
# Target local directory matching your path structure
local_dir = "./model/VibeVoice-ASR-HF-GGUF"
snapshot_download(
repo_id="prithivMLmods/VibeVoice-ASR-HF-GGUF",
local_dir=local_dir,
allow_patterns=[
"VibeVoice-ASR-HF.Q5_K_M.gguf",
"VibeVoice-ASR-HF.mmproj-bf16.gguf",
],
resume_download=True,
)
Build llama-mtmd-cli
Build the multimodal CLI with the CUDA backend enabled:
!cmake --build . --config Release -j$(nproc) --target llama-mtmd-cli
Download sample audio
import os
os.makedirs("/data/llama.cpp/model/VibeVoice-ASR-HF-GGUF", exist_ok=True)
!wget -O /data/llama.cpp/model/VibeVoice-ASR-HF-GGUF/sample.wav \
"https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/gObkjtDcqnbGl2wu_p3UP.wav"
Audio transcription (Plain Text Output)
Run inference using llama-mtmd-cli with dynamic duration injection and Python regex post-processing to output only clean plain text:
import subprocess
import wave
import re
audio_path = "/data/llama.cpp/model/VibeVoice-ASR-HF-GGUF/sample.wav"
# 1. Get exact audio duration
with wave.open(audio_path, "rb") as wf:
duration = wf.getnframes() / float(wf.getframerate())
# 2. Keep 'Start time, End time, Content' prompt anchor to prevent infinite repetition loops
prompt = (
f"This is a {duration:.2f} seconds audio, "
"please transcribe it with these keys: Start time, End time, Content"
)
cmd = [
"/data/llama.cpp/build/bin/llama-mtmd-cli",
"-m", "/data/llama.cpp/model/VibeVoice-ASR-HF-GGUF/VibeVoice-ASR-HF.Q5_K_M.gguf",
"--mmproj", "/data/llama.cpp/model/VibeVoice-ASR-HF-GGUF/VibeVoice-ASR-HF.mmproj-bf16.gguf",
"--audio", audio_path,
"-p", prompt,
"-ngl", "99",
"-c", "8192",
"-n", "512",
"--temp", "0.0"
]
# 3. Capture output and extract only plain text content
res = subprocess.run(cmd, capture_output=True, text=True)
segments = re.findall(r'\{"Start":.*?"Content":\s*"(.*?)"\}', res.stdout)
plain_text = " ".join(segments)
print(plain_text)
Output
The stale smell of old beer lingers. It takes heat to bring out the odor. A cold dip restores health and zest. A salt pickle tastes fine with ham. Tacos al pastor are my favorite. A zestful food is the hot cross bun.
llama.cpp
LLM inference in C/C++ β https://github.com/ggml-org/llama.cpp
Quantized using llama.cpp by prithivMLmods.
- Downloads last month
- 281
3-bit
4-bit
5-bit
6-bit
8-bit
16-bit
Model tree for prithivMLmods/VibeVoice-ASR-HF-GGUF
Base model
microsoft/VibeVoice-ASR-HF