BuilderHelm/Parakeet-CTC-110M-CoreML

Parakeet-CTC-110M CoreML

NVIDIA's Parakeet-TDT-CTC-110M model, CTC path, converted to CoreML for fast transcription and keyword spotting on Apple Silicon.

Model Description

This is the CTC half of NVIDIA's hybrid Parakeet-TDT-CTC-110M: the shared FastConformer encoder and its CTC head, without the TDT decoder.

• CTC Head: greedy transcription in one pass, plus log-probabilities every 80 ms for keyword spotting, custom vocabulary and alignment
• Neural Engine: int8 encoder on the ANE, float32 audio preprocessing on the CPU

Need TDT transcription too? Use BuilderHelm/Parakeet-TDT-CTC-110M-CoreML, which has the same encoder plus the TDT decoder.

Architecture

Component	Description	Size
Preprocessor	Audio → log-mel spectrogram (float32, CPU)	5.7 MB
Encoder	FastConformer encoder + CTC head (int8, Neural Engine)	112.7 MB
vocab.json	Token vocabulary (1,024 tokens)	9 KB
config.json	Windows, long-audio policy and I/O contracts	3 KB

Total size: 118 MB

Performance

Benchmarked on Apple M5, macOS 27.2 (file to text, median of 3 runs):

Metric	Value
Keyword Recall (Earnings22, 1 false alarm per clip)	89.8%
WER, Earnings22 (772 files)	17.62%
WER, LibriSpeech clean / other	2.95% / 5.88%
WER, Common Voice / long audio	8.25% / 2.63%
RTFx, short clips / long audio	1,071× / 998× real-time
Latency per short clip (median / p95)	11.2 / 25.3 ms
Peak memory, short clips / long audio	208 / 109 MB

Compared with FluidInference/parakeet-ctc-110m-coreml on the same Mac and audio (× real-time · WER):

Workload	This model	FluidInference	FluidInference, CPU mel
Short clips (262 Common Voice)	1,071× · 8.25%	96× · 94.86%	96× · 27.90%
Long audio (20 files, 79 min)	998× · 2.63%	123× · 96.93%	129× · 19.43%
Held-out long (18 files, 27 min)	1,023× · 6.70%	125× · 95.93%	126× · 31.89%

FluidInference returns empty text for 149 of the 262 short clips on this Mac; "CPU mel" runs only its mel step on the CPU, which recovers most of them. Speed was measured with 1.1.0; the 1.1.1 window join adds no measurable time. Full method and numbers: evaluation/BENCHMARK.md.

Requirements

• macOS 15+ (Sequoia or later), or iOS 18+ through BuilderHelmAudio
• Apple Silicon (tested on M5 and iPhone 13)
• Swift 6 toolchain (Xcode 16 or later)

Installation

# Using the Hugging Face CLI
pip install -U huggingface_hub

hf download BuilderHelm/Parakeet-CTC-110M-CoreML --revision v1.1.1 \
  --include "*.mlpackage/*" --include "config.json" --include "vocab.json" --include "swift/*" \
  --local-dir parakeet-ctc-110m

Or download the archive (101 MB), which includes everything above.

Usage

Swift (BuilderHelmAudio)

The easiest way to use this model on iPhone, iPad and Mac is the BuilderHelmAudio Swift package. It downloads the model, checks every file's SHA-256 and keeps it on the device. In Xcode, choose File → Add Package Dependencies and enter https://github.com/BuilderHelm/BuilderHelmAudio:

import BuilderHelmAudio
import Foundation

// Downloads the model on first use (124 MB), then loads it from the device
let recognizer = try await SpeechRecognizer.load(.parakeetCtc110M)

let audio = URL(fileURLWithPath: "audio.wav")
let transcript = try await recognizer.transcribe(file: audio)
print(transcript.text)

// Keyword spotting, with start and end times
let hits = try await recognizer.spot(["Christmas", "Mediterranean"], inFile: audio)

On an iPhone 13 it runs at 604× real time on a 6-second clip and 415× on a 72-second recording, with the same transcripts as the Mac.

Swift Package in This Repository

To use the files directly, add the swift/ folder as a local Swift Package:

import Foundation
import ParakeetTdtCtc110M

// Load model (folder with the .mlpackage files)
let model = try ParakeetTdtCtc110M(directory: URL(fileURLWithPath: "parakeet-ctc-110m"), ctcOnly: true)

// Transcribe
let result = try model.transcribe(file: URL(fileURLWithPath: "audio.wav"), includeCtcLogProbs: true)
print(result.text)

// CTC log-probabilities for keyword spotting: [frame][1025], blank = 1024
let logProbs = result.ctcLogProbs

Command Line

cd parakeet-ctc-110m
swift run -c release --package-path swift parakeet-110m . audio.wav --ctc

Model Conversion

To convert from the original NeMo model:

# Build the TDT + CTC CoreML models from NVIDIA's checkpoint
hf download nvidia/parakeet-tdt_ctc-110m parakeet-tdt_ctc-110m.nemo \
  --revision 431a349f3051ab85c22b9b7a2741b5fe77065665 --local-dir nvidia
hf download BuilderHelm/Parakeet-TDT-CTC-110M-CoreML --revision v1.1.0 --include "conversion/*" --local-dir tdt-ctc
python3.11 -m pip install -r tdt-ctc/conversion/requirements.txt
python3.11 tdt-ctc/conversion/build.py --nemo nvidia/parakeet-tdt_ctc-110m.nemo --out tdt-ctc-models

# Derive the CTC-only model
python3.11 conversion/export_ctc_encoder.py --source tdt-ctc-models --out parakeet-ctc-110m

This will:

1. Check the checkpoint against its pinned SHA-256
2. Convert the preprocessor and encoder to CoreML with 4, 8, 15 and 20-second windows and int8 encoder weights
3. Remove the unused TDT output from the encoder, keeping the weight file byte-identical

File Structure

./
├── Preprocessor.mlpackage    # Audio → log-mel spectrogram
├── Encoder.mlpackage         # Log-mel → CTC log-probs
├── vocab.json                # Token vocabulary (1,024 tokens)
├── config.json               # Model configuration
├── conversion/               # CTC-only encoder derivation
├── evaluation/               # Benchmark report and scripts
├── releases/                 # ZIP archive
└── swift/                    # Swift library and CLI

Decoding Modes

Transcription (Greedy CTC)

• One encoder pass, then a greedy CTC collapse
• 1,071× real-time on short clips, 11 ms per clip
• 8.25% WER on Common Voice, 2.95% on LibriSpeech clean
• Best for fast dictation and batch transcription

Keyword Spotting (CTC Log-Probabilities)

• Per-frame log-probabilities over 1,024 tokens plus blank
• 89.8% keyword recall at one false alarm per clip on Earnings22
• Best for detecting specific words, names and phrases

Long Audio

• Any length: audio above 20 seconds is split into 20-second windows
• 4-second overlap, or 6 seconds from 60 seconds; windows join on a frame that is most likely silence and where both windows agree on the words, so no word is repeated at a join
• Mel and decoding on the CPU overlap with the encoder on the Neural Engine: 998× real-time on long audio

Custom Vocabulary / Keyword Spotting

BuilderHelmAudio includes this as recognizer.spot(_:inFile:) (shown above). To score your terms against ctcLogProbs yourself, this scorer tries every way vocab.json can spell the phrase, so no tokenizer is needed:

import Foundation

/// How likely `phrase` was spoken anywhere in the clip: the best CTC path over every way
/// vocab.json can spell it, as average log-probability per character. Closer to 0 = more likely.
func keywordScore(_ phrase: String, logProbs: [[Float]], pieces: [String], blank: Int = 1024) -> Float {
    let text = Array(phrase.split(separator: " ").map { "▁" + $0 }.joined())
    let ids = Dictionary(pieces.enumerated().map { ($1, $0) }, uniquingKeysWith: { first, _ in first })
    var arcs: [(from: Int, to: Int, id: Int)] = []          // every piece that spells text[from..<to]
    for i in text.indices { for j in (i + 1)...text.count { if let id = ids[String(text[i..<j])] { arcs.append((i, j, id)) } } }
    var onArc = [Float](repeating: -.infinity, count: arcs.count)   // path currently emitting that piece
    var onBlank = [Float](repeating: -.infinity, count: text.count) // path on a blank after text[..<k]
    var best = -Float.infinity
    for row in logProbs {
        var nextArc = onArc, nextBlank = onBlank
        for (a, arc) in arcs.enumerated() {
            var previous = max(onArc[a], arc.from == 0 ? 0 : onBlank[arc.from])   // a keyword can start on any frame
            for (b, other) in arcs.enumerated() where other.to == arc.from && other.id != arc.id { previous = max(previous, onArc[b]) }
            nextArc[a] = previous + row[arc.id]
        }
        for k in 1..<text.count {
            var previous = onBlank[k]
            for (b, other) in arcs.enumerated() where other.to == k { previous = max(previous, onArc[b]) }
            nextBlank[k] = previous + row[blank]
        }
        onArc = nextArc; onBlank = nextBlank
        for (a, arc) in arcs.enumerated() where arc.to == text.count { best = max(best, onArc[a]) }
    }
    return best / Float(text.count)
}

let pieces = try JSONDecoder().decode([String].self, from: Data(contentsOf: URL(fileURLWithPath: "parakeet-ctc-110m/vocab.json")))
for keyword in ["Christmas", "Mediterranean"] {
    let score = keywordScore(keyword, logProbs: result.ctcLogProbs, pieces: pieces)
    if score > -1 { print("Found keyword: \(keyword) (\(score))") }
}

On a 5-minute LibriSpeech recording, spoken words scored between −0.003 and −0.05 and absent words −2.9 or lower; 12 keywords took 21 ms. Keywords are case-sensitive; tune the threshold on your own audio.

Keyword recall on Earnings22 (772 clips, about 25 candidate terms per clip), measured with our evaluation spotter, the same for both models:

False alarms per clip	This model	FluidInference CTC 110M
0.1	46.3%	1.1%
0.5	85.4%	2.8%
1.0	89.8%	5.0%

With only the spoken terms as candidates, recall is 97.6% (this model) and 99.2% (FluidInference).

Limitations

• English only
• First load compiles the models with CoreML; later loads use the cache
• Tested on Apple M5 (macOS 27.2) and iPhone 13 (iOS 27.2); other devices not yet tested
• Settings were tuned on parts of the test sets

License

Model weights and tokenizer: CC-BY-4.0, same as the original NVIDIA model. New code: MIT. See NOTICE.md.

Citation

If you use this model, please cite the original NVIDIA work:

@misc{nvidia_parakeet_tdt_ctc_110m,
  title     = {Parakeet-TDT-CTC-110M},
  author    = {NVIDIA},
  year      = {2024},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/nvidia/parakeet-tdt_ctc-110m}
}

Acknowledgments

• Original model by NVIDIA NeMo
• CoreML conversion by BuilderHelm AI