INTEGRATION.md

# Integration contract

The bundle is `pal8/`. Ship it as an intact folder: its `manifest.json` lists every file's relative path, size and SHA-256. Check them once after installing or downloading, not on every launch. Its `adapter_id` is `parakeet-tdt-v3-coreai-pal8-fp16-v1`.

## Components

Each `.aimodel` has one function, `main`. Use Core AI's default specialization.

| Component | Inputs | Outputs | Precision |
| --- | --- | --- | --- |
| `frontend` | `waveform` float32 `[1, 240000]`, `valid_samples` int32 `[1]` | `mel` float32 `[1, 1500, 128]` | FP32 |
| `encoder` | `mel` **float16** `[1, 1500, 128]`, `valid_frames` int32 `[1]` | `encoded` float16 `[1, 188, 640]` | 8-bit palettized weights, FP16 compute |
| `predictor` | `token` int32 `[1, 1]`, `hidden` float32 `[2, 1, 640]`, `cell` float32 `[2, 1, 640]` | `prediction` `[1, 640]`, `next_hidden`, `next_cell` (float32) | FP16 weights, FP32 interface |
| `joint` | `encoder` float32 `[1, 1, 640]`, `predictor` float32 `[1, 1, 640]` | `logits` float32 `[1, 8198]` | FP16 weights, FP32 interface |

**Read each output by its actual tensor type.** Take it from the function's output descriptor, not from the manifest's precision label: the predictor and joint store 16-bit weights but return 32-bit tensors.

## One window: up to 15 seconds

1. **The frontend.** Put up to 240,000 samples of mono 16 kHz Float32 PCM at the start of a zero-filled `[1, 240000]` array, and pass the real count as `valid_samples`.
   - The frontend applies pre-emphasis 0.97.
   - Then a 512-point FFT over a 400-sample symmetric Hann window (centered in 512) with hop 160.
   - Then 128 Slaney mel filters, the log of power plus 2⁻²⁴, and per-channel normalization over the valid frames.
2. **The encoder.** Convert `mel` to float16 and pass `valid_frames = samples / 160`.
3. **The valid encoder steps** are `(samples / 160 + 7) / 8`, in integer division. Each step is 80 ms (1,280 samples).
4. **TDT greedy decoding:**
   - Start the predictor from the blank token (8192) with zero hidden and cell states.
   - At each step, run the joint on that step's `encoded` row and the current `prediction`.
   - The token is the argmax of logits `0…8192`. The duration is the argmax of logits `8193…8197`, which stand for durations 0–4.
   - **A non-blank token:** keep it, run the predictor on it to update the prediction and states, then advance by its duration.
   - **After 10 non-blank tokens in a row with duration 0,** advance one step anyway, and count it as a forced advance.
   - **Blank:** advance by `max(1, duration)`, and reset the zero-duration count.

## Longer recordings, up to 35 seconds

Recordings over 15 seconds use overlapping windows, each owning only its middle:
- **Output interval:** 496 mel frames (4.96 s).
- **Left context:** 496 frames.
- **Right context:** 200 frames (2 s).

For each target interval `[t, t + 79360)` samples, encode `[max(0, t − 79360), min(n, t + 79360 + 32000))`. Keep only tokens whose step position, `window start + step × 1280`, falls in the target interval. Predictor state resets for each window.

Token positions are coarse encoder-step estimates, not word alignments. Recordings shorter than 320 samples (20 ms) return empty text.

## Text

`tokenizer.json` is a Metaspace SentencePiece vocabulary with 8,192 pieces:
1. Drop special tokens (those marked `special` in `added_tokens`).
2. Join the pieces.
3. Replace `▁` with a space.
4. Trim.

## Validation and limits

- **Mac:** [pal8/validation.json](pal8/validation.json) records the checks. Every public test recording matched the original FP32 model's tokens exactly.
- **iPhone:** on an iPhone 17 Pro Max (iOS 27), it gave the same transcripts as the FP16 bundle on all three test clips ([evidence/iphone-fixture.json](evidence/iphone-fixture.json)). Check parity on each target device before relying on it: there are open reports of FP16 Core AI graphs giving wrong outputs on some iPhones (apple/coreai-torch #115, #116).
- **Configurations:** don't substitute `.neuralEngine` or CPU-only specialization without validating them.