nahcmon/fluid-speech-pro-swiftlet-4bit

Fluid Speech Pro — Swiftlet 4-bit

A 4-bit build of Fluid Speech Pro (FastConformer-TDT, 25 European languages) for Swiftlet, a small speech-to-text engine for Apple silicon Macs: Metal kernels for the encoder, NEON for the decoder.

• About 2× faster than the original Core ML model on short clips, 122× real time overall (M1 Pro)
• Ready in 0.15 s: no Neural Engine compile on first launch
• 344 MB on disk instead of 622 MB, with the same overall accuracy (6.75% vs 6.72% WER)

These files only work with Swiftlet. For Core ML, use the original release.

Run it

You need an Apple silicon Mac with macOS 15 or later and Xcode 16 or the command line tools (xcode-select --install).

git clone https://github.com/nahcmon/Swiftlet && cd Swiftlet
swift build -c release
.build/release/swiftlet download              # this repository, 361 MB, SHA-256 checked
.build/release/swiftlet transcribe memo.m4a   # any audio file, any sample rate

In a Swift app (Xcode: File › Add Package Dependencies… › https://github.com/nahcmon/Swiftlet):

import Swiftlet

let folder = try await ModelDownloader.download()
let engine = try SpeechEngine(directory: folder)   // ~0.15 s, keep it loaded
print(try engine.transcribe(contentsOf: URL(fileURLWithPath: "memo.m4a")))

The Swiftlet README has the full guide: raw samples, live dictation, sandboxed apps.

Files

File	What it is
encoder.bin	24-layer FastConformer encoder. Large matrices as a 4-bit palette (16 fp16 values per output row) in the tile layout Swiftlet's Metal GEMM kernels read directly; everything else fp16
decoder.bin	TDT prediction network (2-layer LSTM) and joint network, fp16
Preprocessor.mlmodelc	Core ML log-mel front end, unchanged from upstream, runs on the CPU
parakeet_v3_vocab.json	SentencePiece vocabulary (8192 tokens)

Requires an Apple silicon Mac with macOS 15 or later.

Benchmarks

Measured on an M1 Pro (16 GB) with 234 clips, 46 minutes of English and German audio (best of two runs per clip), against the original fp16 Core ML release of Fluid Speech Pro running on the Neural Engine.

Speed

Audio length	Clips	Original (Core ML, fp16)	Swiftlet 4-bit	Speedup
0–3 s	19	58 ms	26 ms	2.2×
3–6 s	60	70 ms	42 ms	1.7×
6–10 s	59	85 ms	61 ms	1.4×
10–15 s	51	108 ms	99 ms	1.1×
15–30 s	40	183 ms	145 ms	1.3×
2-minute file	1	1.18 s	0.95 s	1.2×
3-minute file	1	1.58 s	1.41 s	1.1×

	Original	Swiftlet 4-bit
All 46 min of audio	29.2 s	22.9 s
Real-time factor	0.0105 (95× real time)	0.0082 (122× real time)
Model load, first launch	20.6 s (Neural Engine compile)	0.15 s
Model load, later	0.23 s	0.13 s

Times are medians per clip, from audio in memory to text.

Accuracy (word error rate, lower is better)

Audio	Original	Swiftlet 4-bit
All clips	6.72%	6.75%
English, clean	2.66%	3.11%
English, other speakers and accents	4.95%	4.69%
English, quiet	2.84%	2.58%
English, noisy	5.34%	5.77%
German, read	6.32%	7.94%
German, phone quality	20.23%	19.38%
Jargon	5.03%	5.03%
Long files (2–3 min)	6.18%	5.13%

175 of 234 transcripts are word-for-word identical to the original. The differences come from the 4-bit encoder weights; German read speech loses the most.

Memory and disk

	Original	Swiftlet 4-bit
Process memory, peak (5 files incl. 2–3 min)	104 MB RSS / 84 MB footprint	92 MB RSS / 92 MB footprint
Weights	held by the Neural Engine service	307 MB file-backed mapping (macOS can page it out and back in)
Size on disk	622 MB	344 MB

How it was made

The upstream encoder's int8 weights were re-quantized to a 4-bit palette per output row (1-D k-means), then everything was re-packed into Swiftlet's own format. Both steps are reproducible with Tools/palettize_encoder.py and Tools/export_model.py.

License and attribution

CC BY 4.0, like the model it is derived from. See LICENSE and NOTICE.

Fluid Speech Pro by FluidVoice. Built on NVIDIA Parakeet TDT 0.6B v3 and Moondream Parakeet Ultra. This repository re-quantizes and re-packs that model; it is not affiliated with or endorsed by FluidVoice, NVIDIA or Moondream.