---
license: cc-by-4.0
base_model: FermionResearch/Phonon-2
base_model_relation: quantized
language:
- en
library_name: coreml
pipeline_tag: automatic-speech-recognition
tags:
- coreml
- neural-engine
- apple-silicon
- speech-to-text
- asr
- stt
- on-device
metrics:
- wer
---

# Phonon-2 Core ML

Phonon-2 for the Apple Neural Engine. The Core ML package reads with the accuracy of the Phonon-2 reference engine: on the Open
ASR Leaderboard's seven English test sets, run with the leaderboard's own code on the full sets, it averages 5.21 % word error,
with LibriSpeech test-clean at 1.73 % and test-other at 3.91 %.

On an M5 MacBook Air an hour of speech becomes text in 6 seconds (606 times realtime), a few seconds of audio returns its text in about 11 milliseconds once the model is loaded, every word carries its start
and end time, and the GPU stays free.

## Run it

Phonon-2 is the model inside [Detta](https://www.fermionresearch.com/products/detta), the dictation app for the Mac. This
repository runs it on the Neural Engine from Swift or Python.

Download the model folder with the Hugging Face command line, in a virtual environment with Python 3.10 to 3.13
(macOS's built-in python3 is 3.9, and coremltools has no wheels for 3.14 yet):

```bash
python3.12 -m venv .venv && source .venv/bin/activate
pip install huggingface_hub
hf download FermionResearch/Phonon-2-CoreML --local-dir Phonon-2-CoreML
```

**Swift**, on macOS 15 or iOS 18 and later, with the Swift package and command-line tool shipped in this repository as
`phonon-coreml-src.tgz`. Unpack it, build the tool and point it at the model folder:

```bash
tar -xzf Phonon-2-CoreML/phonon-coreml-src.tgz          # unpacks to ./phonon-coreml
(cd phonon-coreml && swift build -c release)
phonon-coreml/.build/release/phonon-coreml-cli Phonon-2-CoreML recording.m4a --words
```

It prints the transcript, then every word on its own line with its start and end time in seconds; `--json words.json` writes
the same as JSON. From your own code:

```swift
import PhononCoreML

let transcriber = try Transcriber(bundle: URL(fileURLWithPath: "Phonon-2-CoreML"))   // the folder downloaded above
let result = try transcriber.transcribe(url: URL(fileURLWithPath: "recording.wav"))
print(result.text)
for word in result.words { print(word.start, word.end, word.text) }
```

**Python**, with coremltools, in the same virtual environment:

```bash
pip install coremltools numpy soundfile scipy
python Phonon-2-CoreML/phonon_coreml.py Phonon-2-CoreML recording.wav
```

The Swift runner reads wav, m4a, mp3, aiff, caf and flac at any sample rate and length; the Python runner reads wav and flac at
any sample rate. The first run prepares the package for the Neural Engine once, which can take a
minute or two; the compiled copy is kept, so later loads take under a second.

Phonon-2 also runs through the `fermion` command line on Apple silicon, Linux, Windows and NVIDIA GPUs. See the
[Phonon-2 card](https://huggingface.co/FermionResearch/Phonon-2).

## Long audio

Pass recordings whole, at any length; both runners split long audio at pauses and join the words, so no word is cut at a
boundary.

## Accuracy

Every utterance of the seven English test sets, scored with the Open ASR Leaderboard's own code, with the Core ML package beside
the Phonon-2 MLX engine.

| Set | Core ML, Neural Engine | Phonon-2 MLX engine |
|---|---:|---:|
| LibriSpeech clean | 1.73 | 1.72 |
| LibriSpeech other | 3.91 | 3.92 |
| AMI | 9.33 | 9.37 |
| Earnings-22 | 6.99 | 6.96 |
| GigaSpeech | 8.33 | 8.35 |
| SPGISpeech | 3.70 | 3.70 |
| VoxPopuli | 2.46 | 2.46 |
| **Mean of seven** | **5.21** | **5.21** |

Word error rate, %, lower is better. Phonon-2 transcribes English. The [Phonon-2 card](https://huggingface.co/FermionResearch/Phonon-2)
lists its accuracy in other languages.

## Speed

On an M5 MacBook Air, with the model loaded once, the encoder on the Neural Engine and the decoder on the CPU.

| Audio | Time | Times real time |
|---|---:|---:|
| One hour of LibriSpeech read as a single file | 6.0 s | 606× |
| 158 hours of test-set audio, utterance by utterance | 31 min | 305× |
| 20 recordings of 2 seconds to 2 minutes (797 s) | 2.0 s | 400× |
| A recording of 3 to 5 seconds, model loaded (median of 20) | 11 ms | |

## Files

| File | Size | Contents |
|---|---:|---|
| `Phonon-2.mlpackage` | 332 MB | the encoder, as a Core ML package |
| `decoder.bin` | 13 MB | the decoder and vocabulary |
| `manifest.json` | | source checksum and input contract |
| `phonon_coreml.py` | | the Python runner, with `frontend.py`, `segmenter_np.py` and its decoder library `libphonon_tdt.dylib` |
| `phonon-coreml-src.tgz` | | the Swift package and command-line tool |
| `ci/` | | three LibriSpeech clips and their transcripts, for a quick check |
| `SHA256SUMS` | | a checksum for every file above |

## Licence

The weights are released under CC-BY-4.0. They derive from NVIDIA's parakeet-tdt-0.6b-v3, whose tokenizer and output
conventions (punctuation, casing, numerals) they keep, and `NOTICE` lists the changes. The Swift package, the Python runner and
this repository's code are released under Apache 2.0.