INTEGRATION.md

Integration contract

The bundle is pal8/. Ship it as an intact folder: its manifest.json lists every file's relative path, size and SHA-256. Check them once after installing or downloading, not on every launch. Its adapter_id is parakeet-tdt-v3-coreai-pal8-fp16-v1.

Components

Each .aimodel has one function, main. Use Core AI's default specialization.

Component	Inputs	Outputs	Precision
frontend	waveform float32 [1, 240000], valid_samples int32 [1]	mel float32 [1, 1500, 128]	FP32
encoder	mel float16 [1, 1500, 128], valid_frames int32 [1]	encoded float16 [1, 188, 640]	8-bit palettized weights, FP16 compute
predictor	token int32 [1, 1], hidden float32 [2, 1, 640], cell float32 [2, 1, 640]	prediction [1, 640], next_hidden, next_cell (float32)	FP16 weights, FP32 interface
joint	encoder float32 [1, 1, 640], predictor float32 [1, 1, 640]	logits float32 [1, 8198]	FP16 weights, FP32 interface

Read each output by its actual tensor type. Take it from the function's output descriptor, not from the manifest's precision label: the predictor and joint store 16-bit weights but return 32-bit tensors.

One window: up to 15 seconds

1. The frontend. Put up to 240,000 samples of mono 16 kHz Float32 PCM at the start of a zero-filled [1, 240000] array, and pass the real count as valid_samples.
   • The frontend applies pre-emphasis 0.97.
   • Then a 512-point FFT over a 400-sample symmetric Hann window (centered in 512) with hop 160.
   • Then 128 Slaney mel filters, the log of power plus 2⁻²⁴, and per-channel normalization over the valid frames.
2. The encoder. Convert mel to float16 and pass valid_frames = samples / 160.
3. The valid encoder steps are (samples / 160 + 7) / 8, in integer division. Each step is 80 ms (1,280 samples).
4. TDT greedy decoding:
   • Start the predictor from the blank token (8192) with zero hidden and cell states.
   • At each step, run the joint on that step's encoded row and the current prediction.
   • The token is the argmax of logits 0…8192. The duration is the argmax of logits 8193…8197, which stand for durations 0–4.
   • A non-blank token: keep it, run the predictor on it to update the prediction and states, then advance by its duration.
   • After 10 non-blank tokens in a row with duration 0, advance one step anyway, and count it as a forced advance.
   • Blank: advance by max(1, duration), and reset the zero-duration count.

Longer recordings, up to 35 seconds

Recordings over 15 seconds use overlapping windows, each owning only its middle:

• Output interval: 496 mel frames (4.96 s).
• Left context: 496 frames.
• Right context: 200 frames (2 s).

For each target interval [t, t + 79360) samples, encode [max(0, t − 79360), min(n, t + 79360 + 32000)). Keep only tokens whose step position, window start + step × 1280, falls in the target interval. Predictor state resets for each window.

Token positions are coarse encoder-step estimates, not word alignments. Recordings shorter than 320 samples (20 ms) return empty text.

Text

tokenizer.json is a Metaspace SentencePiece vocabulary with 8,192 pieces:

1. Drop special tokens (those marked special in added_tokens).
2. Join the pieces.
3. Replace ▁ with a space.
4. Trim.

Validation and limits

• Mac: pal8/validation.json records the checks. Every public test recording matched the original FP32 model's tokens exactly.
• iPhone: on an iPhone 17 Pro Max (iOS 27), it gave the same transcripts as the FP16 bundle on all three test clips (evidence/iphone-fixture.json). Check parity on each target device before relying on it: there are open reports of FP16 Core AI graphs giving wrong outputs on some iPhones (apple/coreai-torch #115, #116).
• Configurations: don't substitute .neuralEngine or CPU-only specialization without validating them.