Phonon-2.5 English Core ML

Phonon-2.5-English for the Apple Neural Engine. Phonon-2.5 English transcribes English with punctuation and casing. This package carries the Phonon-2.5-English weights exactly: on a 420-utterance gate set its transcripts are identical to the Phonon-2.5-English reference engine's on 389, and the rest differ only by 16-bit float rounding (accuracy below).

One 332 MB package holds the encoder for every utterance up to 35 seconds, over a single set of weights; the decoder is a 13 MB table file. Every word carries its start and end time, and the GPU stays free.

Run it

macOS 15 or later on Apple silicon. The runtime ships compiled in bin/.

Command line

hf download FermionResearch/Phonon-2.5-English-CoreML --local-dir Phonon-2.5-English-CoreML
chmod +x Phonon-2.5-English-CoreML/bin/phonon-coreml-cli
Phonon-2.5-English-CoreML/bin/phonon-coreml-cli Phonon-2.5-English-CoreML recording.wav --words

Swift: add bin/PhononCoreML.xcframework.zip (unzipped) to your target as a binary framework.

import PhononCoreML

let transcriber = try Transcriber(bundle: URL(fileURLWithPath: "Phonon-2.5-English-CoreML"))   // the folder this repository downloads to
let result = try transcriber.transcribe(url: URL(fileURLWithPath: "recording.wav"))
print(result.text)
for word in result.words { print(word.start, word.end, word.text) }

C, Python and other languages: bin/libphonon_coreml.dylib with the C header bin/include/phonon_coreml.h. From Python, with nothing to install, run from the downloaded folder:

import ctypes, json, sys
lib = ctypes.CDLL("bin/libphonon_coreml.dylib")
lib.pcm_open.restype = ctypes.c_void_p
lib.pcm_open.argtypes = [ctypes.c_char_p, ctypes.c_int32, ctypes.c_int32, ctypes.c_char_p, ctypes.c_int32]
lib.pcm_transcribe_long.restype = ctypes.c_void_p
lib.pcm_transcribe_long.argtypes = [ctypes.c_void_p, ctypes.c_char_p, ctypes.c_void_p, ctypes.c_int64, ctypes.c_int32,
                                    ctypes.c_double, ctypes.c_double, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p,
                                    ctypes.c_char_p, ctypes.c_int32]
lib.pcm_free.argtypes = [ctypes.c_void_p]
err = ctypes.create_string_buffer(1024)
h = lib.pcm_open(b".", 4, 0, err, len(err))            # this folder; 4 decoder threads; Neural Engine
if not h: sys.exit(err.value.decode())
p = lib.pcm_transcribe_long(h, sys.argv[1].encode(), None, 0, 1, 0, 0, None, None, None, err, len(err))
if not p: sys.exit(err.value.decode())
result = json.loads(ctypes.string_at(p).decode()); lib.pcm_free(p)
print(result["text"])
for w in result["words"]: print(w["start"], w["end"], w["text"])

All three take 16 kHz audio of any length (stereo is mixed down). The first run prepares the package for the Neural Engine once (about 50 seconds before the first transcript and about 3 minutes for all four windows on an M5 MacBook Air); the prepared copy is kept, so later loads take about a second.

Phonon-2.5-English also runs through the fermion command line on Apple silicon, Linux, Windows and NVIDIA GPUs. See the Phonon-2.5-English card.

Smaller download

If the model.fermion file of Phonon-2.5-English (the file the fermion runtime keeps) is already on the machine, the package can be built from it instead of downloaded: fetch only the small files (β‰ˆ39 MB) and expand. The result is checked byte for byte against the published package before it is written.

hf download FermionResearch/Phonon-2.5-English-CoreML --local-dir Phonon-2.5-English-CoreML --exclude "Phonon-2.5-English.mlpackage/*"
chmod +x Phonon-2.5-English-CoreML/bin/phonon-coreml-cli
Phonon-2.5-English-CoreML/bin/phonon-coreml-cli --expand model.fermion Phonon-2.5-English-CoreML/kit Phonon-2.5-English-CoreML/Phonon-2.5-English.mlpackage

Expansion takes about 2 seconds. From C: pcm_expand(container, kit_dir, out_package, 0, err, errlen).

Long audio and the leading pad

An utterance of up to 35 seconds is read in one pass by the smallest encoder window that holds it. Longer recordings are cut at pauses into windows of up to 15 seconds and their words are joined, so no word is split at a boundary. Pass recordings whole; the runtime does the cutting.

Every window is encoded with a quarter second of silence in front of it (leading_pad_s in manifest.json), the runtime default of Phonon-2.5-English: a recording that starts in the middle of a word is then read from its first word. Word times are reported on the recording's own clock.

Accuracy

Word error rate, %, lower is better: this package on the Neural Engine beside the Phonon-2.5-English reference engine on the same utterances. English sets are scored with the Open ASR Leaderboard's own code. The Phonon-2.5-English card lists the full-set accuracy.

Set Core ML, Neural Engine Reference engine
Open ASR Leaderboard English sets (280-utterance sample of the seven sets) 5.62 5.58
LibriSpeech test-clean (40 utterances) 1.46 1.46
One hour of LibriSpeech read as one file 1.13

Speed

M5 MacBook Air, Neural Engine, package loaded:

Long audio One hour of English audio in about 6 seconds (β‰ˆ600Γ— real time)
A 3 to 5 second utterance 8.6 ms
Later loads (all four windows) about a second
Memory while reading an hour 0.14 GB

Files

File Size Contents
Phonon-2.5-English.mlpackage 332 MB the encoder for 5, 10, 15 and 35 second windows over one shared weight set (the weights stored exactly)
decoder.bin 13 MB the prediction network, joint and vocabulary
manifest.json source checksum, window lengths, the leading pad and the input contract
kit/ 26 MB what --expand needs to build Phonon-2.5-English.mlpackage from model.fermion
bin/ 1 MB phonon-coreml-cli, libphonon_coreml.dylib + include/phonon_coreml.h, PhononCoreML.xcframework.zip (macOS, arm64)
ci/ three LibriSpeech clips and their transcripts, for a quick check
NOTICE, LICENSE-* attribution, the weights licence and the runtime licence
SHA256SUMS a checksum for every file above

Licence

The weights are released under CC-BY-4.0. They derive from NVIDIA's parakeet-tdt-0.6b-v3, whose tokenizer and output conventions (punctuation, casing, numerals) they keep, and NOTICE lists the changes. The runtime in bin/ is distributed in compiled form under the Apache License 2.0 (LICENSE-RUNTIME-Apache-2.0.txt).

Downloads last month
14
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for FermionResearch/Phonon-2.5-English-CoreML

Quantized
(5)
this model