| --- |
| license: cc-by-nc-4.0 |
| language: |
| - da |
| pipeline_tag: automatic-speech-recognition |
| tags: |
| - audio.cpp |
| - gguf |
| - whisper |
| - qwen3 |
| base_model: |
| - syvai/hviske-v6 |
| --- |
| |
| # Hviske v6 GGUF |
|
|
| For use with [audio.cpp](https://github.com/0xShug0/audio.cpp), family |
| `hviske_asr_v6`. These self-contained packages include the tokenizer, configuration, |
| and model spec; no separate Python runtime or tokenizer download is required. |
|
|
| Converted from [syvai/hviske-v6](https://huggingface.co/syvai/hviske-v6/tree/323b5ffdf54ef22196340a0b37cb11e668fefc5a), |
| revision `323b5ffdf54ef22196340a0b37cb11e668fefc5a`. |
| The architecture is a rotary-position Whisper encoder, frame-stacking projector, |
| and Qwen3 decoder. It is separate from the Conformer-based Hviske v5 family. |
|
|
| ## Packages |
|
|
| | File | Storage | Size | Recommendation | |
| |---|---|---:|---| |
| | `hviske-v6-bf16.gguf` | Native BF16 | 2.61 GiB | Default; preserves source tensor precision. | |
| | `hviske-v6-q8_0.gguf` | Q8_0 | 1.54 GiB | Lower memory use, with expected transcript drift. | |
| |
| The model transcribes Danish. It supports offline longform transcription with |
| silence-based splitting, but does not provide timestamps, diarization, or native |
| streaming. CUDA, Vulkan, and CPU were tested. |
| |
| `audio_chunk_mode=auto` (default) splits recordings longer than 30 seconds at |
| silence boundaries. `audio_chunk_duration_sec=28` sets the maximum chunk length |
| when splitting (up to 30 seconds). The automatic trigger stays at 30 seconds. |
| Use `audio_chunk_mode=silence` to split at silence boundaries as soon as input |
| exceeds the configured chunk duration, `fixed` for fixed-duration chunks, or `none` for unsplit |
| input of at most 30 seconds. Changing chunk boundaries can change transcripts. |
|
|
| ```bash |
| audiocpp_cli --task asr --family hviske_asr_v6 \ |
| --model /path/to/hviske-v6-bf16.gguf --backend cuda \ |
| --audio danish.wav --threads 8 --log |
| ``` |
|
|
| Formatting controls require both `cased` and `punctuated`, for example |
| `--request-option cased=true --request-option punctuated=true`. |
| The default uses two beams; `--request-option num_beams=1` selects greedy decoding. |
| See the [model documentation](https://github.com/0xShug0/audio.cpp/blob/main/docs/models/hviske_asr_v6.md) |
| for all options and conversion instructions. |
|
|
| ## CUDA Performance |
|
|
| RTX 5090, eight CPU threads, Debug C++ build. Each process first transcribed a |
| 16.15-second warmup clip, then a 180-second real Danish recording. Model loading |
| is excluded from inference time. Peak VRAM is process-scoped NVML sampling at |
| 20-ms intervals across the request sequence, not retained memory. |
|
|
| Python used the official wrapper with its defaults, including BF16 on CUDA, |
| two beams, and longform batch size eight. C++ used native package storage and |
| the same default two-beam policy. No TF32 or precision overrides were used for |
| these performance measurements. These are individual warmed measurements, not |
| a statistical benchmark. |
|
|
| | Runtime | Warm 180-s inference | RTF | Observed peak VRAM | |
| |---|---:|---:|---:| |
| | Official Python (BF16 default) | 2.774 s | 0.01541 | 4,260 MiB | |
| | C++ BF16 | 2.239 s | 0.01244 | 3,198 MiB | |
| | C++ Q8_0 | 2.014 s | 0.01119 | 2,246 MiB | |
| |
| C++ BF16 is 1.24x faster in this comparison, with about 25% lower peak VRAM. |
| An earlier Python run took 2.641 s, against which C++ BF16 is 1.18x faster. |
| The same-dtype improvement does not depend on quantization: writing prefill K/V |
| directly into the backend cache removed a CPU export/import round trip and reduced |
| C++ BF16 inference from 2.800 s to 2.398 s without changing its transcripts. |
| Packing Q/K/V and gate/up projections and reusing the host logits buffer further |
| reduced it to 2.239 s. These use the same framework modules on all backends, |
| without backend-specific gates or changing weight precision. |
| Q8 is an optional storage/memory trade-off, not evidence for the BF16 optimization. |
| Python was PyTorch 2.11.0+cu128 with Transformers 4.57.6. The upstream custom |
| model code did not load with Transformers 5.12.1 in this validation. |
| |
| The longform input was the first 180 seconds of |
| [Da-Solen.ogg](https://commons.wikimedia.org/wiki/File:Da-Solen.ogg), read by |
| Danielle dk, licensed [CC BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/), |
| converted to mono 16-kHz PCM16. It contains no synthesized or repeated segments. |
| |
| ## CPU Performance |
| |
| AMD Ryzen 9 7900X, eight threads in both implementations. Python used its default |
| FP32 CPU weights; C++ used `hviske_asr_v6.weight_type=f32` for the same precision. |
| The request sequence was a short warmup, the same 180-second recording, then the |
| short recording again. Python batches longform chunks by default; C++ processes |
| them sequentially. Times exclude loading, and each row is a single warmed request. |
|
|
| | Input | C++ | Official Python | C++ RTF | Python RTF | |
| |---|---:|---:|---:|---:| |
| | 16.15 seconds | 4.962 s | 8.434 s | 0.307 | 0.522 | |
| | 180 seconds | 54.988 s | 110.282 s | 0.305 | 0.613 | |
|
|
| Short transcripts match exactly. On longform, Python CPU and C++ FP32 differ on |
| `stof` versus `støv` and four words in the repeated ending. These differences were |
| already present before projection packing; the optimization preserves C++ output. |
| Its additional CPU gain is modest: 56.352 s to 54.988 s on longform. |
|
|
| ## Parity and Limitations |
|
|
| Controlled parity was tested separately: Python FP32 versus C++ with |
| `hviske_asr_v6.weight_type=f32`, both with `NVIDIA_TF32_OVERRIDE=0`. |
| All three short Danish recordings produced identical transcripts. This is |
| transcript parity, not a claim of bit-identical intermediate tensors. |
|
|
| With normal runtime settings, BF16 CUDA and CPU differ from Python on one phrase |
| in one short recording: Python emits `reachdirektivet`, while C++ emits |
| `reighedsdirektivet og`. Vulkan BF16 and Q8 CPU match Python on this recording. |
| The controlled FP32 comparison removes this difference; disabling TF32 alone does |
| not. Backend and weight precision can affect beam selection. |
|
|
| On the 180-second recording, BF16 C++ has zero normalized word differences from |
| Python's 352-word transcript, with punctuation/casing differences. Q8 has two |
| word substitutions relative to Python (0.57% transcript disagreement): |
| `wikpedia` becomes `wikipedia`, and `støv` becomes `stod`. This is not WER |
| against a human ground-truth transcript. Normalization uses lowercase Unicode |
| word tokens, ignoring punctuation. Q8 output changed slightly with projection |
| packing; BF16 output and controlled FP32 parity remained unchanged. |
|
|
| Formatting controls, greedy decoding, changing inputs, and repeat requests were |
| tested. The short request after longform retains its original transcript. Model |
| quality remains upstream-dependent; the Python output itself repeats part of the |
| last sentence in this excerpt. |
|
|
| ## License |
|
|
| The weights are CC BY-NC 4.0, not licensed for commercial use. Preserve the |
| upstream `NOTICE` and `LICENSES/` files, which describe the underlying Saga-2, |
| Whisper, and Qwen components and their notices. |
|
|
