parakeet-redux for LiteRT

Speech-to-text in 25 languages on an Android phone: moondream/parakeet-redux (NVIDIA's parakeet-tdt-0.6b-v3 with a ternary encoder) converted to LiteRT with litert-torch 0.9.4 and ai-edge-quantizer 0.10.0. On a Galaxy S26 it transcribes 99× faster than real time on the NPU, 52× on the GPU and 29× on the CPU (655 LibriSpeech test-clean utterances, one after another; LiteRT 2.3.0, measured 2026-10-21). Its word error rate on all of LibriSpeech test-clean is 1.977 % on the NPU, against 1.968 % for our float32 PyTorch run of the same windows.

Files

file	bytes	what	backend
parakeet_redux_encoder_10s_i4.tflite	345,895,584	encoder for windows up to 10 s; 240 of the 264 ternary linear layers as 4-bit integers (−1, 0 or +1 per weight, one scale per 128 weights); the 24 relative-position projections are folded into float32 constants with the position table; the rest float32	CPU
parakeet_redux_encoder_20s_i4.tflite	370,471,584	the same, up to 20 s	CPU
parakeet_redux_encoder_30s_i4.tflite	395,047,616	the same, up to 30 s	CPU
parakeet_redux_encoder_10s_f16.tflite	1,194,680,560	encoder for windows up to 10 s; float16 weights in the linear and convolution layers, LayerNorm inputs rescaled so that float16 arithmetic does not overflow	GPU, NPU
parakeet_redux_encoder_20s_f16.tflite	1,219,256,560	the same, up to 20 s	GPU, NPU
parakeet_redux_encoder_30s_f16.tflite	1,243,832,560	the same, up to 30 s	GPU, NPU
parakeet_redux_mel_10s.tflite	1,608,068	log-mel front end, 10 s of 16 kHz audio	CPU
parakeet_redux_mel_20s.tflite	2,252,068	the same, 20 s	CPU
parakeet_redux_mel_30s.tflite	2,896,076	the same, 30 s	CPU
parakeet_redux_decoder_i8.tflite	17,866,768	TDT prediction network (predict) and joint network (joint), int8 weights	CPU
tokenizer.json	1,159,960	the original's tokenizer (byte-identical to nvidia/parakeet-tdt-0.6b-v3's)	host
redux_asr_runner_b.cc, build_android.sh		a command-line runner for Android (LiteRT C API) and its build script
tdt_text.py		turns the runner's token ids into text	host

Each encoder file holds the whole encoder; the three window lengths only save the work of padding short audio. Take the i4 set for the CPU (1.14 GB with the mel and decoder files) or the f16 set for the GPU or the NPU (3.68 GB with them).

How it works

audio (16 kHz mono) → 30 s windows → mel front end → encoder → greedy TDT decoding → text

• Windows. The audio is cut into consecutive windows of 479,840 samples (30 s); the last one is shorter. A window under 320 samples (two mel frames) is dropped and has no text. Each window is transcribed on its own, and the window texts are joined with one space. The model's voice-activity head is not converted, so long audio is cut at fixed 30 s points, not at pauses, and a word that crosses a cut can be split or lost.
• Buckets. Each window goes to the smallest graph pair that holds it: up to 160,000 samples → the 10 s mel and encoder, up to 320,000 → 20 s, else 30 s. The shorter graphs only skip the work on padding: in float32 they gave the same tokens as the 30 s graphs on all 83 windows of our 82-utterance test subset.
• Mel front end (parakeet_redux_mel_*, inputs audio [1, samples] float32 and n_valid [1] int32): pre-emphasis, STFT (512 points, 400-sample Hann window, hop 160), 128 Slaney mel bins, log and per-window normalization over the valid frames, as transformers' ParakeetFeatureExtractor. Outputs features [1, 128, T], mel_mask [1, T] and frame_mask [1, T / 8] (T = 1,000 / 2,000 / 3,000).
• Encoder (parakeet_redux_encoder_*, inputs features and mel_mask as the mel graph gives them): output enc [1, T / 8, 640]; rows past the valid frames are undefined.
• Decoding (on the host, parakeet_redux_decoder_i8): predict (token [1, 1] int32, h and c [2, 1, 640]) → dec_out [1, 640] and the new h, c; joint (one enc row [1, 640], dec_out) → 8,198 logits = 8,193 token logits (8192 = blank) + 5 duration logits (0–4 frames). Start from blank with zero state; at each step take the arg-max token and duration; a blank with duration 0 moves one frame; a non-blank token is emitted and fed to predict; ten tokens on one frame move one frame; token 3 ends the window; stop at the last valid frame (the sum of frame_mask) or after 3,750 steps.
• Text. Decode each window's token ids with tokenizer.json, special tokens skipped and repeated ids kept (Tokenizer.decode(ids, skip_special_tokens=True)). Id 3 (<|endoftext|>) is not flagged special in tokenizer.json, so a window that ended on it would print the literal token; no window did in our runs.

Use it

1. Download

pip install -U huggingface_hub
hf download litert-community/parakeet-redux --local-dir parakeet-redux
cd parakeet-redux

2. Your audio

Both runners below read 16 kHz mono 16-bit WAV, and the Android runner reads a folder with an index file. The block below writes those WAVs and the index from any format ffmpeg reads; replace sample1.wav sample2.wav with your files.

mkdir -p audio && : > audio/index.tsv && i=0
for f in sample1.wav sample2.wav; do
  id=$(printf %03d $i)
  ffmpeg -loglevel error -y -i "$f" -ac 1 -ar 16000 -c:a pcm_s16le "audio/$id.wav"
  printf '%s\t%s\t%s.wav\n' "$id" "$(basename "$f")" "$id" >> audio/index.tsv
  i=$((i + 1))
done

3a. On a desktop (Python, CPU)

Save this as transcribe.py in the parakeet-redux folder:

import sys
import time
import wave

import numpy as np
from ai_edge_litert.interpreter import Interpreter
from tokenizers import Tokenizer

BUCKETS = (10, 20, 30)  # seconds: each 30 s window goes to the smallest graph that holds it
mel = {b: Interpreter(f"parakeet_redux_mel_{b}s.tflite", num_threads=4).get_signature_runner() for b in BUCKETS}
enc = {b: Interpreter(f"parakeet_redux_encoder_{b}s_i4.tflite", num_threads=4).get_signature_runner() for b in BUCKETS}
dec = Interpreter("parakeet_redux_decoder_i8.tflite", num_threads=1)
predict, joint = dec.get_signature_runner("predict"), dec.get_signature_runner("joint")
tokenizer = Tokenizer.from_file("tokenizer.json")
BLANK, EOS, WINDOW = 8192, 3, 479840


def read_wav(path):  # 16 kHz mono 16-bit PCM
    with wave.open(path) as w:
        assert (w.getframerate(), w.getnchannels(), w.getsampwidth()) == (16000, 1, 2), path
        return np.frombuffer(w.readframes(w.getnframes()), "<i2").astype(np.float32) / 32768


def transcribe_window(x):
    n = len(x)
    b = next(b for b in BUCKETS if n <= b * 16000)
    audio = np.zeros((1, b * 16000), np.float32)
    audio[0, :n] = x
    m = mel[b](audio=audio, n_valid=np.array([n], np.int32))
    e = enc[b](features=m["features"], mel_mask=m["mel_mask"])["enc"][0]
    frames = int(m["frame_mask"].sum())
    state = predict(token=np.array([[BLANK]], np.int32), h=np.zeros((2, 1, 640), np.float32), c=np.zeros((2, 1, 640), np.float32))
    ids, t, steps, run = [], 0, 0, 0
    while t < frames and steps < 3750:
        steps += 1
        logits = joint(enc_frame=e[t:t + 1], dec_out=state["dec_out"])["logits"][0]
        token, d = int(np.argmax(logits[:8193])), int(np.argmax(logits[8193:]))
        if token == BLANK and d == 0:
            d = 1
        if token != BLANK:
            ids.append(token)
            if token == EOS:
                break
            state = predict(token=np.array([[token]], np.int32), h=state["h_out"], c=state["c_out"])
        if d == 0:
            run += 1
            if run >= 10:
                d = 1
        if d > 0:
            run = 0
        t += d
    return tokenizer.decode(ids, skip_special_tokens=True)


for path in sys.argv[1:]:
    x = read_wav(path)
    t0 = time.perf_counter()
    texts = [transcribe_window(x[s:s + WINDOW]) for s in range(0, len(x), WINDOW) if len(x[s:s + WINDOW]) >= 320]
    print(f"{path}\t{len(x) / 16000:.1f} s audio\t{time.perf_counter() - t0:.2f} s\t" + " ".join(t for t in texts if t))

pip install ai-edge-litert==2.3.0 tokenizers numpy
python transcribe.py audio/*.wav

3b. On an Android phone (native runner)

Build the runner with the Android NDK r29 (we built it on macOS; the path below is Android Studio's default there). The script downloads the LiteRT 2.3.0 C headers (litert_cc_sdk.zip from LiteRT's GitHub release) and the runtime libraries (Google's Maven), and checks their sha256.

NDK=$HOME/Library/Android/sdk/ndk/29.0.13113456 bash build_android.sh

Copy the runner, the CPU files and your audio to the phone, run it, and turn its output into text:

adb shell mkdir -p /data/local/tmp/parakeet-redux
adb push redux_asr_runner_b libs/libLiteRt.so parakeet_redux_mel_*.tflite parakeet_redux_encoder_*_i4.tflite parakeet_redux_decoder_i8.tflite audio /data/local/tmp/parakeet-redux/
adb shell chmod 755 /data/local/tmp/parakeet-redux/redux_asr_runner_b

adb shell 'cd /data/local/tmp/parakeet-redux && LD_LIBRARY_PATH=. ./redux_asr_runner_b --accel cpu --libdir . \
  --model parakeet_redux_encoder_30s_i4.tflite --mel parakeet_redux_mel_30s.tflite \
  --model10 parakeet_redux_encoder_10s_i4.tflite --mel10 parakeet_redux_mel_10s.tflite \
  --model20 parakeet_redux_encoder_20s_i4.tflite --mel20 parakeet_redux_mel_20s.tflite \
  --dec parakeet_redux_decoder_i8.tflite --threads 8 --mel-threads 2 --dec-threads 1 \
  --fixtures audio --count $(grep -c . audio/index.tsv) --rounds 0 --dump none --out out' > out.jsonl

pip install tokenizers
python tdt_text.py tokenizer.json out.jsonl

Each line of out.jsonl that starts with {"kind":"utt" holds one file's token ids (tok), their encoder frame (frame = window index × 375 + the frame within the window) and duration (dur), and the stage times in ms; the burst_summary line gives rtf_burst (processing time / audio time, the model load excluded). --threads 8 was the fastest of 2, 4, 6 and 8 on the Galaxy S26's CPU; try other values on other phones. The CPU run prints GPU accelerator could not be loaded and registered on stderr: the GPU library is not on the phone for this run, and nothing else changes.

GPU. Push libs/libLiteRtClGlAccelerator.so and the f16 encoders, and run the CPU command with --accel gpu --gpu-precision fp16 and the parakeet_redux_encoder_*_f16.tflite files, without --threads 8 (the encoder no longer runs on the CPU):

adb push libs/libLiteRtClGlAccelerator.so parakeet_redux_encoder_*_f16.tflite /data/local/tmp/parakeet-redux/
adb shell 'cd /data/local/tmp/parakeet-redux && LD_LIBRARY_PATH=. ./redux_asr_runner_b --accel gpu --gpu-precision fp16 --libdir . \
  --model parakeet_redux_encoder_30s_f16.tflite --mel parakeet_redux_mel_30s.tflite \
  --model10 parakeet_redux_encoder_10s_f16.tflite --mel10 parakeet_redux_mel_10s.tflite \
  --model20 parakeet_redux_encoder_20s_f16.tflite --mel20 parakeet_redux_mel_20s.tflite \
  --dec parakeet_redux_decoder_i8.tflite --mel-threads 2 --dec-threads 1 \
  --fixtures audio --count $(grep -c . audio/index.tsv) --rounds 0 --dump none --out out' > out.jsonl

NPU (Galaxy S26, Qualcomm Hexagon). The runner compiles each f16 encoder for the Hexagon NPU on the phone the first time (LiteRT's Qualcomm compiler plugin, on-device JIT) and keeps the result in --cache-dir. It needs LiteRT's Qualcomm plugin libraries (libLiteRtDispatch_Qualcomm.so and libLiteRtCompilerPlugin_Qualcomm.so, from the litert-npu-runtime-qualcomm package that build_android.sh fetches) and the QNN libraries of the Qualcomm AI Runtime SDK (QAIRT) 2.50.0.260828, which you download from Qualcomm and copy into a folder qnn/: lib/aarch64-android/ libQnnHtp.so, libQnnHtpPrepare.so, libQnnHtpV81Stub.so, libQnnHtpV81CalculatorStub.so, libQnnSystem.so, libQnnIr.so, libQnnSaver.so, and lib/hexagon-v81/unsigned/libQnnHtpV81Skel.so. --window-samples is the window each graph holds: 479,840 samples for the 30 s graph, 320,000 and 160,000 for the others. Compile one encoder per process first (one compile alone takes the runner to 5.3–6.2 GB, see Notes), then run with all three from the cache:

adb push libs/libLiteRtDispatch_Qualcomm.so libs/libLiteRtCompilerPlugin_Qualcomm.so qnn/. parakeet_redux_encoder_*_f16.tflite /data/local/tmp/parakeet-redux/
NPU_ENV='LD_LIBRARY_PATH=. ADSP_LIBRARY_PATH="/data/local/tmp/parakeet-redux;/vendor/dsp/cdsp;/vendor/lib/rfsa/adsp;/system/lib/rfsa/adsp;/dsp"'
compiled=0
for b in 30 20 10; do
  adb shell "cd /data/local/tmp/parakeet-redux && $NPU_ENV ./redux_asr_runner_b --accel npu+cpu --libdir . --cache-dir cache \
    --model parakeet_redux_encoder_${b}s_f16.tflite --mel parakeet_redux_mel_${b}s.tflite --window-samples $((b * 16000 - (b / 30) * 160)) \
    --dec parakeet_redux_decoder_i8.tflite --fixtures audio --count 1 --rounds 0 --dump none --out out_compile" > compile_$b.log &&
    grep -q '^done wall_ms=' compile_$b.log || { echo "the ${b} s encoder did not compile (see compile_$b.log): stopped" >&2; break; }
  compiled=$((compiled + 1))
done
[ "$compiled" = 3 ] && adb shell "cd /data/local/tmp/parakeet-redux && $NPU_ENV ./redux_asr_runner_b --accel npu+cpu --libdir . --cache-dir cache \
  --model parakeet_redux_encoder_30s_f16.tflite --mel parakeet_redux_mel_30s.tflite \
  --model10 parakeet_redux_encoder_10s_f16.tflite --mel10 parakeet_redux_mel_10s.tflite \
  --model20 parakeet_redux_encoder_20s_f16.tflite --mel20 parakeet_redux_mel_20s.tflite \
  --dec parakeet_redux_decoder_i8.tflite --mel-threads 2 --dec-threads 1 \
  --fixtures audio --count \$(grep -c . audio/index.tsv) --rounds 0 --dump none --out out" > out.jsonl

To remove everything from the phone: adb shell rm -rf /data/local/tmp/parakeet-redux.

Accuracy

Word error rate (%) of the NPU form on a Galaxy S26 (the f16 encoders on the NPU, the int8 decoder on the CPU), against the PyTorch reference. The reference is transformers 5.20.2 ParakeetForTDT in float32 with the ternary weights expanded, the same 30 s windows with a valid-length mask, and the same greedy TDT decoding. LibriSpeech test-clean: every utterance; FLEURS: every utterance of the test split, scored against the Open ASR Leaderboard's text column.

set	utterances	WER, Galaxy S26 NPU	WER, PyTorch reference	WER (jiwer), Galaxy S26 NPU	WER (jiwer), PyTorch reference
LibriSpeech test-clean	2,620	1.977	1.968	2.406	2.389
FLEURS German	862	5.736	5.632	6.820	6.692
FLEURS French	676	7.626	7.446	8.290	8.152
FLEURS Russian	775	6.737	6.717	7.273	7.253

The first pair of columns is scored as the Open ASR Leaderboard scores (its English or multilingual normalizer, compound merging, kaldialign), the second is plain corpus WER (jiwer) on the normalized text, without the compound merging. Every backend on the 655-utterance LibriSpeech subset of the performance table, same scoring (reference 1.952 / 2.531):

phone, backend	WER	WER (jiwer)
Galaxy S26, NPU	1.975	2.554
Galaxy S26, GPU	1.959	2.539
Galaxy S26, CPU	1.967	2.546
Pixel 8a, CPU	1.959	2.539

Performance

phone	backend	RTF	× real time	encoder per call, 30 / 20 / 10 s window (ms)	mel, 30 s (ms)	TDT decoding of a 30 s window (ms)	peak memory
Galaxy S26	NPU (Hexagon, f16)	0.010126	99×	155.5 / 79.2 / 41.9	3.7	20.6	4.80 GB (4,689,976 kB)
Galaxy S26	GPU (Adreno, f16, fp16 arithmetic)	0.019194	52×	155.7 / 105.4 / 60.4	3.5	17.3	7.83 GB (7,647,308 kB)
Galaxy S26	CPU (i4, 8 threads)	0.034982	29×	552.6 / 427.3 / 195.1	5.8	33.8	2.89 GB (2,820,008 kB)
Pixel 8a	CPU (i4, 8 threads)	0.112011	9×	—	—	—	2.84 GB (2,769,932 kB)

• RTF = the summed processing time (mel + encoder + decoding, the model load excluded) / the audio time, over 655 LibriSpeech test-clean utterances (4,875.8 s of audio) run one after another in one process after one warm-up utterance, with redux_asr_runner_b as in "Use it" (the mel graphs at 2 threads, the decoder at 1). The windows went 504 / 131 / 23 to the 10 / 20 / 30 s graphs.
• Per call: the median of 20 calls after 3 warm-up calls, input write and output read included, on LibriSpeech test-clean 8230-279154-0043 (24.5 s: its 30 s window with 306 valid encoder frames, and its first 20 s and 10 s), each run started from a phone at rest (thermal status 0, no CPU frequency cap). On the Galaxy S26's CPU the phone capped the CPU clocks within seconds (to 1.79 / 1.75 GHz), so the CPU numbers are sustained ones. The NPU and GPU calls also saw a shallow CPU cap within seconds.
• Peak memory: the runner's VmHWM during the 655-utterance run (GB = kB × 1024 / 10⁹).
• Pixel 8a: the CPU command of "Use it" with the 655 utterances as its input folder (and --dump full); per-call times for this form were not measured on this phone. The run started at rest, and the phone capped its three CPU clusters during it (to 1.425 / 1.065 / 1.164 GHz).
• Galaxy S26 (SM-S942Q, Android 16) and Pixel 8a (Tensor G3, Android 16); LiteRT 2.3.0 from Google's Maven; on the NPU the QNN libraries of QAIRT 2.50.0.260828 for Hexagon v81; 2026-10-21.

Notes

• First NPU run. On a Galaxy S26 the on-device compile took 65–119 s for the 30 s and 20 s encoders and 169–210 s for the 10 s encoder (one encoder per process; five runs, 15 compiles), and each compiled encoder takes 1.2 GB in the cache folder; a later run loads each of the three from the cache in 0.9–3.1 s. While one encoder compiles, the runner's peak memory is 5.3–6.2 GB (VmHWM 5,134,964–6,056,380 kB over the 15 compiles). In one of our runs, started right after the GPU block above, the phone's available memory fell to 0.80 GB (776,964 kB) while the 30 s encoder compiled, and our harness stopped the run (it stops at the first reading under 1,000,000 kB); a run without the GPU step before it compiled all three with at least 2.64 GB (2,579,600 kB) available. Start the first NPU run fresh, with other apps closed.
• GPU and fp16. In the original encoder the squared deviation inside the first LayerNorm reaches 6.8e7 over the 55 windows of our range check, beyond float16's largest value 65,504; with float16 arithmetic (the GPU's default) it returned only blanks on a Mac GPU in our test. The f16 encoders scale each of the 120 LayerNorm inputs by 2^-k (k = 3–9 per norm) and its epsilon by 4^-k, which leaves the float32 result bit-identical and keeps the squares in range; the GPU and the NPU run these files.
• Pixel 8a GPU. The three f16 encoders do not run together on the Pixel 8a's GPU in our test: our harness stops a run at the first memory reading under 400,000 kB available, and the second encoder's GPU setup read 170,696 kB. One 30 s f16 encoder alone runs, and was slower than the CPU form in our runs (both with the float32 decoder at 4 threads, not the table's CPU form: RTF 0.161 against 0.152).
• int8 decoder. The decoder's weights are int8 with float32 activations (dynamic range). Against the float32 decoder (Galaxy S26, NPU encoders) it gives the same token sequence on 630 of 658 windows of the 655-utterance subset, and the WER stays within 0.03 points of the reference.
• Converted from the weights of transformers 5.20.2's ParakeetForTDT (the ternary weights expanded), rewritten as plain PyTorch graphs (masks as float arithmetic, the LSTM step as explicit gates) and exported with litert-torch 0.9.4; the steps and scripts are in hf-to-litertlm (REPRODUCE.md, parakeet-redux).

License

The model files are under CC-BY-4.0, as moondream/parakeet-redux and NVIDIA's parakeet-tdt-0.6b-v3 are. The runner and the scripts (redux_asr_runner_b.cc, build_android.sh, tdt_text.py) are under the Apache License 2.0.