spybyscript/parakeet-tdt-litert

Parakeet-TDT-0.6B-v3 — INT8 LiteRT

Community conversion of NVIDIA Parakeet-TDT-0.6B-v3, with a dynamic INT8 encoder and the original FP16 decoder/joint. No additional training. This is not an official NVIDIA or Google release.

The repository root now contains the INT8 release. The previous FP16 release, including its original ZIP and documentation, is preserved byte-for-byte in legacy/. See migration and integration instructions.

Files and download

File	Size, decimal MB	Purpose
encoder_T1500.tflite	628.6	Dynamic INT8 encoder; one fixed 15-second input bucket
decoder_step.tflite	23.7	Original FP16 LSTM prediction network
joint_step.tflite	12.7	Original FP16 token/duration joint network
tokenizer.model	0.4	Unchanged SentencePiece tokenizer
manifest.json	—	Tensor contracts, profile, component hashes and provenance
validation.json	—	Measured quality and memory results, with limitations
mapped_model_file.h	—	Optional native source-page memory hint; Linux-tested example

The four runtime assets total approximately 665 MB on disk. Download them individually, or use the updated ZIP bundle. Root SHA256SUMS includes the ZIP checksum; the ZIP contains component checksums in its manifest.

To avoid downloading the legacy release and duplicate ZIPs, select root files explicitly:

from huggingface_hub import snapshot_download

snapshot_download(
    "spybyscript/parakeet-tdt-litert",
    local_dir="parakeet-int8",
    allow_patterns=[
        "encoder_T1500.tflite", "decoder_step.tflite", "joint_step.tflite",
        "tokenizer.model", "manifest.json", "validation.json",
        "README.md", "INTEGRATION.md", "NOTICE.md", "mapped_model_file.h",
    ],
)

Pin a repository revision in production. The root encoder filename is unchanged, but its content and precision have changed; invalidate caches using the manifest hashes or release ID 20260914-int8-t1500-v1.

Measured memory

Linux CPU configuration	Process peak	Resident memory with all models retained
Previous FP16 encoder	4.30 GB	—
INT8, ordinary runtime	1.60 GB	—
INT8, verified native source-page hint after warmup	1.46 GB	0.954 GB RSS / 0.947 GB PSS

All figures use decimal GB and include encoder, decoder and joint in an isolated ASR process. They exclude the Android UI, capture frontend and other ML models. Runtime: LiteRT 2.1.2/XNNPACK, four encoder threads and one decoder/joint thread. Android, GPU and NPU memory/performance have not been validated for this release.

The approximately 954 MB result requires the optional file-mapping optimization described in INTEGRATION.md. It is not automatic when downloading this model and is not a Java CompiledModel setting. Startup memory remains separately reported; clean source pages can remain in the OS file cache until reclaimed.

Recognition validation

On 33 English synthetic clips with 745 source-text reference words, INT8 scored 4.30% word error, versus 4.03% for the prior FP16 baseline. These references were not independently transcribed by humans. INT8 matched 29/33 complete token sequences and 31/33 normalized word sequences; it is not an exact-parity replacement for FP16.

Four additional human long recordings totaled 159.758 seconds. They lack complete reference transcripts, so no human-reference WER is claimed. The native source-page hint preserved all 37/37 token sequences versus ordinary INT8. That parity claim concerns the memory hint, not quantization versus FP16.

Those four long recordings took about 12.18 seconds of ordinary INT8 compute, or 11.42 seconds with the hint, versus 31.94 seconds for FP16. Frontend calculation, loading and warmup are excluded; other work was running on the machine. These timings are observations, not a controlled speed ranking or phone benchmark. The base model is multilingual; the current quantization evaluation was English only. Private audio, transcripts and token sequences are not distributed.

Buffered live use

This is a component bundle with a host-side TDT decoder, not a single audio-to-text graph. It can process a continuous audio source using overlapping windows. The tested profile commits 4.96 seconds, with 4.96 seconds of left context and 2 seconds of right context, padded to the 15-second capacity.

Expect approximately 7 seconds of initial buffering and updates every 5 seconds, plus compute. The 15-second input capacity does not mean waiting 15 seconds for every result. This is buffered processing of an offline-trained encoder; it does not have Nemotron's cache-aware encoder or frequent immediate partials. Boundary repetitions and recognition errors remain possible. Neither VAD nor speaker diarization is included.

Precision and provenance

The encoder has 193 fully connected, 50 Conv2D and 27 depthwise-convolution operations with INT8 weights. Dynamic integer kernels consume floating-point activations/interfaces; this is not a fully integer end-to-end graph. The unchanged decoder/joint use FP16 weight storage and floating-point interfaces. Start with CPU/XNNPACK and test other backends independently.

Conversion used the existing staged Parakeet program, PyTorch 2.11.0, litert-torch 0.9.1 and AI Edge Quantizer 0.7.0. The staged source-program SHA256 and runtime component hashes are in the manifest. INT4 alternatives were investigated locally but are not included in this default release because they changed more long-form words and omitted some phrases.

License and attribution

Model weights derive from NVIDIA Parakeet-TDT-0.6B-v3 under CC BY 4.0. Conversion and quantization by spybyscript; see NOTICE.md. Consult the upstream card for base-model capabilities and training information. Upstream accuracy benchmarks are not measurements of this conversion.