README.md

---
license: cc-by-4.0
language:
- en
library_name: coreml
pipeline_tag: automatic-speech-recognition
base_model:
- ConwayResearch/Underdog-Bark-3.0
- ConwayResearch/Underdog-Bark-3.0-Clean
tags:
- coreml
- apple-neural-engine
- speech-to-text
- asr
- parakeet
- underdog
- fluidaudio
- on-device
---

# Underdog Bark 3.0 · Core ML

Full Core ML pipelines (preprocessor, encoder, decoder, joint) for
[Underdog Bark 3.0](https://huggingface.co/ConwayResearch/Underdog-Bark-3.0) and
[Underdog Bark 3.0 Clean](https://huggingface.co/ConwayResearch/Underdog-Bark-3.0-Clean) by Conway Research, laid
out in the [FluidAudio](https://github.com/FluidInference/FluidAudio) Parakeet TDT v3 component layout so they run
end to end in Swift with the encoder on the Apple Neural Engine.

Underdog already ships a Core ML encoder for Bark 3.0 (with the preprocessor and decoder in Python), and Bark 3.0 Pro
as a full Core ML pipeline. This repo adds the rest of the pipeline for the small models, plus Bark 3.0 Clean, which
had no Core ML build.

| Folder | Model | Output |
|---|---|---|
| [`bark-3.0/`](bark-3.0) | Underdog Bark 3.0 | Verbatim: keeps "um", "uh", repeats |
| [`bark-3.0-clean/`](bark-3.0-clean) | Underdog Bark 3.0 Clean | Cleaned: drops fillers, false starts and repeated words |

English only. Tokenizer and output conventions (punctuation, casing, numerals) are Parakeet TDT v3's.

## Files (each folder)

| File | What it is |
|---|---|
| `Encoder.mlmodelc` | FastConformer encoder, 15 s window, **Neural Engine default** (sparse mask + 6-bit palette per 8 rows; 399 MB Bark 3.0, 410 MB Clean) |
| `Encoder_lut3.mlmodelc` | Same weights as a dense 3-bit palette per row (253 MB): **use this one on the GPU** (`.cpuAndGPU`) |
| `Decoder.mlmodelc` | 2-layer LSTM prediction network |
| `JointDecisionv3.mlmodelc` | TDT joint (token + duration), v3 interface |
| `Preprocessor.mlmodelc` | Parakeet TDT v3 mel front-end (Bark uses the original's) |
| `parakeet_vocab.json` | Parakeet TDT v3 vocabulary (8192 tokens) |
| `NOTICE` | Upstream attribution |

### How the encoder is encoded

Bark stores each encoder linear as per-row levels: 260 of the 264 matrices take five values per output row
`{0, ±a, ±b}`, and layer 23's four feed-forward matrices take seven. Core ML has no entropy-coded weight format, so
the weights are re-encoded as Core ML palettes (`constexpr_lut_to_sparse` + `constexpr_sparse_to_dense`, iOS 18):
the palette of every group of 8 output rows holds those rows' values, and each weight stores its index. Nothing is
re-quantized: every palettized weight reconstructs **bit-exactly** to the value in the upstream `model.bark` (240
matrices; the 24 positional projections act on the fixed positional table and are folded into constants at export).

Grouping 8 rows per palette matters on the Neural Engine, whose palette cost scales with the number of palettes:
per-row palettes run about 3× slower there. The GPU does not care, which is why `Encoder_lut3.mlmodelc` (per-row,
smallest dense file) is the GPU build; the sparse `Encoder.mlmodelc` is rebuilt on every GPU load (minutes), so do not
use it with `.cpuAndGPU`.

## Benchmarks

Apple M5 Pro, macOS 27, FluidAudio `asr-benchmark`, encoder on the Neural Engine, **full** LibriSpeech test sets,
corpus WER (total edits / total reference words, lowercase, punctuation stripped), RTFx = audio / processing time.

| Model | test-clean WER | test-other WER | RTFx clean / other |
|---|--:|--:|--:|
| Bark 3.0 (this repo) | — | 4.96 % | — / 140× |
| Bark 3.0 Clean (this repo) | 2.37 %¹ | 4.66 % | 155×¹ / 143× |
| Bark 3.0 Clean, upstream MLX (`bark_mlx.py`, fp16), same files and scorer | 2.39 % | — | 55× / — |
| Parakeet TDT v3 (FluidInference, reference) | 2.27 % | 4.12 % | 150× / 140× |

Long-form: a one-hour Earnings-22 recording (four concatenated calls) transcribes in **6.5 s (558×)** with Bark 3.0 and
7.0 s (514×) with Clean, models warm; Clean writes 7,787 words to Bark 3.0's 8,223 (fillers and repeats removed).

¹ Measured with a dense 6-bit-palette build of the same exact encoder weights; the shipped sparse `Encoder.mlmodelc`
matched it on test-other (4.66 % both, 143× vs 142×).

Notes:
- LibriSpeech references are verbatim and Clean removes words on purpose, so read speech is not where Clean shines;
  see the upstream cards for AMI, DisfluencySpeech and the other Open ASR sets.
- Bark 3.0 is evaluated upstream with 0.25 s of leading silence. Through FluidAudio's sliding-window pipeline it makes
  no difference (test-other 5.02 % with it vs 4.96 % without), so none is added.
- Bark models emit `<unk>` (token 0) where their training text had a quotation mark. Upstream decoders skip special
  tokens; FluidAudio drops it from the transcript as of [#1054](https://github.com/FluidInference/FluidAudio/pull/1054).
  On older releases strip it from `result.text` (`replacingOccurrences(of: "<unk>", with: "")`); the WER numbers above
  are scored without it.

## Usage (Swift, FluidAudio)

With [FluidAudio](https://github.com/FluidInference/FluidAudio) ([#1054](https://github.com/FluidInference/FluidAudio/pull/1054)
adds `AsrModelVersion.bark3` / `.bark3Clean`, which download the right folder):

```swift
import FluidAudio

let models = try await AsrModels.downloadAndLoad(version: .bark3)   // or .bark3Clean
let asr = AsrManager(config: .default)
try await asr.loadModels(models)
var state = TdtDecoderState.make(decoderLayers: await asr.decoderLayerCount)
let result = try await asr.transcribe(URL(fileURLWithPath: "meeting.wav"), decoderState: &state)
print(result.text)
```

```bash
swift run -c release fluidaudiocli transcribe meeting.wav --model-version bark3   # or bark3-clean
```

On a FluidAudio release without #1054, download one folder and load it with the v3 layout:

```bash
hf download FluidInference/bark-3.0-coreml --include "bark-3.0/*" --local-dir bark
```

```swift
let models = try AsrModels.loadLocal(from: URL(fileURLWithPath: "bark/bark-3.0"), version: .ultra)
```

The first load compiles the encoder for the Neural Engine (up to a minute); later loads are cached by Core ML.
Requires iOS 18 / macOS 15 or later. Not yet tested on iPhone.

## Credits and license

- **Underdog Bark 3.0 and Bark 3.0 Clean** by Conway Research ([Underdog](https://underdog.ai)), CC-BY-4.0.
- Derived from **nvidia/parakeet-tdt-0.6b-v3** (© NVIDIA Corporation, CC-BY-4.0) through
  **moondream/parakeet-ultra** (© Moondream, CC-BY-4.0).
- This repo: Core ML conversion and re-encoding by Fluid Inference. The encoder's linear layers are bit-exact to the
  upstream weights; the remaining tensors (subsampling, depthwise convs, norms, decoder, joint) are the upstream values
  converted to Core ML at fp16. `NOTICE` in each folder carries the upstream attribution. License: [CC-BY-4.0](LICENSE).

## Changelog

| Date | Files | Change | Issue | mobius PR | FluidAudio PR | HF commit |
|---|---|---|---|---|---|---|
| 2026-10-21 | `bark-3.0/*`, `bark-3.0-clean/*` | Initial release | — | — | [#1054](https://github.com/FluidInference/FluidAudio/pull/1054) | [`ca21bed2`](https://huggingface.co/FluidInference/bark-3.0-coreml/commit/ca21bed2ccd9c3fbcd104a6014b9a5638a764bc8) |