parakeet-ultra β ggml (whisper.cpp format), q8_0 and f16
A conversion of Moondream's parakeet-ultra speech-to-text model into the ggml file format that
whisper.cpp's Parakeet runtime reads, built for the OxSay speech-to-text engine on macOS. parakeet-ultra
is a post-trained version of NVIDIA's parakeet-tdt-0.6b-v3 (same architecture, tokenizer and
parameter count). This repository publishes only the conversion; the weights are Moondream's and NVIDIA's,
under the same licence (CC BY 4.0). See NOTICE.md.
Model Details
- Upstream:
moondream/parakeet-ultraat revision73175eb7aeb0d82f1e2a6b53b3aabc10a90bcd0b(CC BY 4.0), itself a post-trainednvidia/parakeet-tdt-0.6b-v3. Language set, tokenizer and output conventions (punctuation, casing, numerals) are v3's: 25 European languages, automatic language detection. The NVIDIA files used are at revision541d1f99c6b0c3cd0b11a95167540bb8edefd82b. - This build: FastConformer-TDT, 24 encoder layers, d_model 1024, 8 heads, 8Γ subsampling, a 2-layer LSTM prediction network (640), 8192-token SentencePiece vocabulary plus a blank, TDT durations {0, 1, 2, 3, 4}; about 0.6 B parameters. Input: 16 kHz mono audio (the runtime computes the 128-bin log-mel features itself from the filterbank and window stored in the file). Output: text with punctuation and capitalisation, plus word timings from the runtime.
- Precision: two files with the same 721 tensors.
q8_0: the 273 large weight matrices are 8-bit (blocks of 32 values with a float16 scale).f16: the same 273 matrices are float16. In both, the 448 small tensors (biases, norms, convolution kernels) stay float32. The pipeline that made this file reproduces NVIDIA's v3 files fromggml-org/parakeet-GGUFbyte for byte β see Conversion andPROVENANCE.md. - Runtime: whisper.cpp's Parakeet support. Built and checked at whisper.cpp commit
6e4ab854f67f743900934a703d5603419384c961(2026-09-28; not a release tag, it sits betweenv1.9.4andv1.9.5). - Converter:
convert/in this repository (Apache-2.0), plus whisper.cpp's ownmodels/convert-parakeet-to-ggml.pyandparakeet-quantize, unmodified. Rebuilds these bytes from the pinned inputs. SeePROVENANCE.md. - Variants:
q8_0(669 MB) is the default.f16(1.26 GB) is slightly more accurate on our Russian-accented English proxy and about twice as slow (see Evaluation, Precision). Mixed-precision builds (q8_0 with chosen tensors in float16) were tried and not published: no significant gain, and the stock Parakeet loader inox-sttrefused files that mix the two types.
Files
| file | bytes | SHA-256 |
|---|---|---|
ggml-parakeet-ultra-q8_0.bin |
668,757,119 | fa28ec8c6afd9893da9829ec9f2fa908a60bc0fbebd37fd60bbd2f0953076fb2 |
ggml-parakeet-ultra-f16.bin |
1,255,897,319 | ee42f83ae751d7b3f9c435543df92ea6926f2d73df3129186a1ab58a8c7b6150 |
manifest.json has the same entries plus upstream revisions and the known-answer record;
python3 -I convert/verify_manifest.py checks the downloaded files against it.
Uses
With whisper.cpp's CLI (usage from examples/parakeet-cli/README.md at the pinned commit; we did not
build or run parakeet-cli for this release):
$ cmake -B build -S . && cmake --build build --target parakeet-cli -j
$ ./build/bin/parakeet-cli -m ggml-parakeet-ultra-q8_0.bin -f audio.wav
With OxSay's ox-stt (this is the engine the checks below ran on):
$ ox-stt -m ggml-parakeet-ultra-q8_0.bin -f audio.wav # JSON on stdout: text, words, timings
$ printf 'ValueError\t1\n' | ox-stt -m ggml-parakeet-ultra-q8_0.bin -f audio.wav --terms-stdin --boost-alpha 3
The f16 file is used the same way: pass ggml-parakeet-ultra-f16.bin to -m.
--terms-stdin reads terms on stdin (we used one term<TAB>weight per line, weight 1) and biases decoding towards them;
--boost-alpha scales the bias. Audio must be 16 kHz mono WAV for ox-stt
(ffmpeg -i in -ar 16000 -ac 1 out.wav); parakeet-cli also reads flac, mp3 and ogg.
Out of scope: speaker diarisation, translation, streaming latency guarantees, and any use that needs the model's own voice-activity head (it is not in this file).
Limitations
- Apple Silicon only. Built, run and measured on one machine, an Apple M5 Pro (Metal). No other CPU, GPU or operating system was tried.
- The voice-activity head was dropped. ultra ships six
vad_head.*tensors that Moondream's runtime (Photon) uses to cut long recordings at pauses. whisper.cpp's converter ignores them, so they are not in this file. Long audio is chunked by the engine instead (ox-stt --chunk-s, 30 in its usage line). The vendor's long-form number below depends on that head and does not transfer. - The preprocessor is assumed to be v3's. Moondream publishes no filterbank or feature-extractor
file; this build uses NVIDIA's v3
featurizer.fbandfeaturizer.window. Unproven, but consistent with the outputs below. - Our evaluation is synthetic and small: 25 + 20 sentences of macOS text-to-speech (125 + 20 for the precision table), no real recordings, no noise, one voice per condition.
- Language is auto-detected and cannot be pinned, as in v3. Short or accented English can come out transliterated into Cyrillic (v3 did this on 3 of 25 clips of our Russian-accented proxy voice; ultra on 0 of 25).
- Accented English still misrecognises short identifiers (
isinstance,elif,enumerate); term boosting helps but does not close the gap (table below). - The vendor's accuracy figures were measured in Moondream's Photon runtime on NVIDIA GPUs, not in whisper.cpp and not on these files.
Known-answer self-check (run this after every download)
The expected output is for whisper.cpp's public-domain sample, not bundled here:
samples/jfk.wav
at the pinned commit (SHA-256 59dfb9a4acb36fe2a2affc14bacbee2920ff435cb13cc314a08c13f66ba7860e, 11.0 s, 16 kHz mono).
$ ox-stt -m ggml-parakeet-ultra-q8_0.bin -f jfk.wav
Expected text, exactly:
And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.
SHA-256 of that string (UTF-8, no trailing newline):
37d003a932256f11d07e00d0c1478443140ea5e87d817b0f8bc577c1d2aa2e1b. Recorded in manifest.json under known_answer.
Run twice on 2026-11-18 (Apple M5 Pro, Metal, ox-stt SHA-256 93eaf72bβ¦bf7): the two JSON outputs are
identical in every field except elapsed_s (0.271 s and 0.278 s) β same text, same word boundaries,
same probabilities. ggml-parakeet-ultra-f16.bin returns the same text, also on two runs that day.
What this check does and does not tell you. It catches a corrupt or mis-converted file and a
broken engine path, which produce garbage or an empty string. It does not tell ultra from v3:
NVIDIA's v3 q8_0 returns the same sentence on the same clip. The identity of the file is established by its SHA-256 above, not by this transcript.
Conversion (CC BY 4.0 Β§3(a)(1)(B) list of changes)
Converted 2026-11-18 with the scripts in convert/ (steps in convert/build.sh, bytes and checks in
PROVENANCE.md). A format conversion only; nothing is trained, pruned or merged:
convert/hf2nemo.pyrenames moondream's Hugging Face tensor names back to NeMo names, using transformers' own NeMo-to-HF mapping applied to v3's 723 tensor names, checked injective, then inverted. A pure rename, no transposes.- The 6
vad_head.*tensors are not included. - ultra's 356 float16 tensors are widened to float32 exactly.
featurizer.fb,featurizer.window,model_config.yamland the tokenizer come from NVIDIA's v3.nemo. ultra'sconfig.jsonandtokenizer.jsonare byte-identical to v3's Hugging Face files.q8_0: whisper.cppmodels/convert-parakeet-to-ggml.py --use-f32, thenparakeet-quantize <f32> <out> q8_0(the only lossy step), both at the pinned commit.f16: the same converter without--use-f32(its default), which rounds the 273 large matrices to float16 (the only lossy step) and keeps the rest float32. The converter itself folds the prediction-network LSTMbias_ihintobias_hhand reorders the LSTM gates, as it does for every Parakeet model.
Round-trip check. The same pipeline fed NVIDIA's v3 Hugging Face weights reproduces
ggml-org/parakeet-GGUF's ggml-parakeet-tdt-0.6b-v3-f32.bin, β¦-q8_0.bin and β¦-f16.bin byte for byte (SHA-256
b17956caβ¦ae7f, 4d64e9e9β¦600e and 833bffc9β¦426f, both sides recomputed or read from the Hub's LFS metadata). That validates the rename,
the featurizer/tokenizer sourcing and the converter. It cannot validate what only exists on the ultra side (the
float16 cast and the dropped head); see PROVENANCE.md.
Evaluation
Ours β synthetic, small, one machine. Word error rate (%, corpus level, punctuation and casing
ignored; eval/score.py, computed 2026-11-18 from the transcripts in eval/transcripts/). "v3" is NVIDIA's
v3 as ggml-org/parakeet-GGUF q8_0 (same bytes as our own rebuild; the transcripts of the two are identical on all 100 set-1 clips); "ultra" is this file. Same
engine (ox-stt, Metal), same clips, same settings on both sides. Sources of audio: macOS say, 16 kHz mono.
| set | audio | words | condition | v3 WER | ultra WER | v3 β ultra, term hits |
|---|---|---|---|---|---|---|
| English code review, 25 sentences | voice Milena (Russian-accented proxy) | 323 | plain | 38.39 | 31.89 | 31/75 β 33/75 |
| term boost (see below) | 34.67 | 28.79 | 44/75 β 44/75 | |||
| voice Samantha (native English) | 323 | plain | 13.62 | 13.00 | 48/75 β 51/75 | |
| term boost | 11.46 | 11.76 | 57/75 β 57/75 | |||
| Russian, 20 sentences | macOS TTS voice Milena (Russian) | 206 | plain | 1.46 | 0.97 | β |
How to read it: on the accented proxy voice ultra makes 21 fewer word errors than v3 out of 323 without
boosting (124 β 103) and 19 fewer with it (112 β 93); on the native voice the two are within 2 word errors of each other
in both conditions, and with boosting ultra is 1 error behind. On Russian the difference is one
word (3 errors β 2 of 206). With 25 sentences and one voice per cell, treat all of these as a sign, not a measurement.
"Term hits" counts the code identifiers of each sentence (75 in total, eval/set1_sentences.tsv) that appear in the transcript. "Term boost" is
--terms-stdin --boost-alpha 3 with, per sentence, that sentence's own terms plus those of the three sentences before and after it (weight 1) β so the list always contains
the right answer among distractors. The Cyrillic count (v3 3/25, ultra 0/25) is for the plain Milena condition and the same for the boosted one.
Timing, same machine. Apple M5 Pro, 48 GB, macOS 26.5.1; ox-stt default backend, which is Metal (the -v log of
the known-answer run shows using MTL0 backend); the timing run itself did not log its backend. 100 clips of
4.4 s on average per model (set 1, both voices, both conditions), elapsed_s as printed by ox-stt, run sequentially under nice -n 15;
other jobs may have shared the machine: v3 mean 271 ms per clip, ultra 266 ms; real-time factor 16.2Γ and 16.5Γ. Same file size and tensor shapes, so a
difference is not expected; this is a sanity check, not a benchmark.
Precision: q8_0 vs f16. A larger synthetic set than the table above: set 1 plus set 2
(125 English sentences, eval/set1_sentences.tsv and eval/set2_sentences.tsv; 178 code identifiers), each
spoken by the same two voices, plus the 20 Russian sentences. Plain decoding, no term boost. Transcripts in
eval/transcripts/precision-*.tsv; python3 -I eval/score_precision.py prints these numbers.
| set | words | q8_0 WER |
f16 WER |
term hits q8_0 β f16 |
|---|---|---|---|---|
| English, Milena (Russian-accented proxy) | 1,512 | 16.53 | 15.67 | 102/178 β 107/178 |
| English, Samantha (native) | 1,512 | 4.89 | 4.89 | 145/178 β 146/178 |
| Russian, Milena | 206 | 0.97 | 0.97 | β |
mean elapsed_s per clip (270 clips) |
0.225 s | 0.470 s |
On the accented proxy, f16 makes 13 fewer word errors out of 1,512 (250 β 237). A paired bootstrap over the
125 sentences puts the WER difference at β0.86 points (95% interval β1.57 to β0.26); 11 sentences got better and 1
worse. On the native voice and on Russian the two files give the same error counts. f16 took about twice as long per
clip on the same machine. One voice per cell and synthetic speech: a sign, not a measurement. The transcripts
were made with a local ox-stt build; the installed ox-stt (93eaf72bβ¦bf7) returns the identical text for all
270 f16 clips.
Vendor numbers β Moondream's own, not ours. From the parakeet-ultra model card (WER %, lower is better), measured by Moondream with the Open ASR Leaderboard pipeline of September 2026; ultra in Photon on an NVIDIA GPU, v3 in NeMo (bf16). Not reproduced here, not measured on this file or on whisper.cpp:
| benchmark | parakeet-tdt-0.6b-v3 | parakeet-ultra |
|---|---|---|
| Open ASR Leaderboard, 7 English sets (average) | 6.26 | 5.80 |
| FLEURS, 25 languages (average) | 11.62 | 9.55 |
| FLEURS, Russian | 5.91 | 5.21 |
Pitfalls
None hit during the build. Two things worth knowing when rebuilding or relying on the file:
- whisper.cpp's converter defaults to float16. Without
--use-f32it writes every weight tensor with two or more dimensions (norms, biases and convolution kernels excepted) as float16, which rounds float32 source values before they are quantized; the result then differs from the file published here. - The known-answer clip does not separate ultra from v3 (see above); compare the file's SHA-256.
Citation
@misc{sekoyan2025canary1bv2parakeettdt06bv3efficient,
title={Canary-1B-v2 & Parakeet-TDT-0.6B-v3: Efficient and High-Performance Models for Multilingual ASR and AST},
author={Monica Sekoyan and Nithin Rao Koluguri and Nune Tadevosyan and Piotr Zelasko and Travis Bartley and Nikolay Karpov and Jagadeesh Balam and Boris Ginsburg},
year={2025},
eprint={2509.14128},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2509.14128},
}
@misc{rekesh2023fastconformerlinearlyscalable,
title={Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition},
author={Dima Rekesh and Nithin Rao Koluguri and Samuel Kriman and Somshubra Majumdar and Vahid Noroozi and He Huang and Oleksii Hrinchuk and Krishna Puvvada and Ankur Kumar and Jagadeesh Balam and Boris Ginsburg},
year={2023},
eprint={2305.05084},
archivePrefix={arXiv},
primaryClass={eess.AS},
url={https://arxiv.org/abs/2305.05084},
}
@misc{xu2023efficientsequencetransductionjointly,
title={Efficient Sequence Transduction by Jointly Predicting Tokens and Durations},
author={Hainan Xu and Fei Jia and Somshubra Majumdar and He Huang and Shinji Watanabe and Boris Ginsburg},
year={2023},
eprint={2304.06795},
archivePrefix={arXiv},
primaryClass={eess.AS},
url={https://arxiv.org/abs/2304.06795},
}
@misc{koluguri2025granaryspeechrecognitiontranslation,
title={Granary: Speech Recognition and Translation Dataset in 25 European Languages},
author={Nithin Rao Koluguri and Monica Sekoyan and George Zelenfroynd and Sasha Meister and Shuoyang Ding and Sofia Kostandian and He Huang and Nikolay Karpov and Jagadeesh Balam and Vitaly Lavrukhin and Yifan Peng and Sara Papi and Marco Gaido and Alessio Brutti and Boris Ginsburg},
year={2025},
eprint={2505.13404},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.13404},
}
parakeet-ultra: Moondream, https://huggingface.co/moondream/parakeet-ultra β release post Introducing Parakeet Redux and Ultra.
- Downloads last month
- 30
Model tree for Koptev/parakeet-ultra-ggml
Base model
moondream/parakeet-ultra