Automatic Speech Recognition
LiteRT
LiteRT
speech
audio
parakeet
tdt
int8
android
on-device
buffered-streaming
Instructions to use spybyscript/parakeet-tdt-litert with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use spybyscript/parakeet-tdt-litert with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Publish INT8 T1500 release and preserve previous FP16 bundle under legacy
Browse filesArchive every prior release file including its ZIP byte-for-byte. Publish the validated INT8 encoder, unchanged decoder/joint/tokenizer, updated ZIP, measured memory/quality evidence and native integration instructions. Android memory and delegate validation remain pending.
- INTEGRATION.md +55 -0
- NOTICE.md +1 -0
- README.md +51 -161
- SHA256SUMS +14 -0
- encoder_T1500.tflite +2 -2
- legacy/.gitattributes +35 -0
- legacy/README.md +206 -0
- legacy/archive.json +55 -0
- legacy/decoder_step.tflite +3 -0
- legacy/encoder_T1500.tflite +3 -0
- legacy/joint_step.tflite +3 -0
- legacy/manifest.json +345 -0
- legacy/parakeet-tdt-0.6b-v3-litert.zip +3 -0
- legacy/tokenizer.model +3 -0
- manifest.json +88 -260
- mapped_model_file.h +64 -0
- parakeet-tdt-0.6b-v3-litert.zip +2 -2
- publication.json +11 -0
- validation.json +51 -0
INTEGRATION.md
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Integrating the INT8 Parakeet bundle
|
| 2 |
+
|
| 3 |
+
## Migrating from the FP16 release
|
| 4 |
+
|
| 5 |
+
Use the root INT8 encoder with root decoder, joint and tokenizer. Their filenames remain stable. Decoder/joint/tokenizer bytes match the previous release. Check `release_id` and SHA256 in the manifest when invalidating downloaded-file caches. The previous complete release is in `legacy/`; its documentation describes that historical release, not the current INT8 memory measurements.
|
| 6 |
+
|
| 7 |
+
The 15-second shape and TDT token/duration contracts are retained. Use the actual tensor names and signature metadata rather than relying on positional ordering across exports. Keep all three interpreters alive and reuse buffers; do not load all precision variants concurrently.
|
| 8 |
+
|
| 9 |
+
## Encoder
|
| 10 |
+
|
| 11 |
+
Signature: `serving_default`.
|
| 12 |
+
|
| 13 |
+
| Role | Actual tensor name | Type | Shape |
|
| 14 |
+
|---|---|---|---|
|
| 15 |
+
| Normalized log-mel input | `serving_default_args_0` | float32 | `[1,128,1500]` |
|
| 16 |
+
| Actual valid mel length | `serving_default_args_1` | int32 | `[1]` |
|
| 17 |
+
| Encoded sequence | `serving_default_output_0_output` | float32 | `[1,1024,188]` |
|
| 18 |
+
| Valid encoder length | `serving_default_output_1_output` | int32 | `[1]` |
|
| 19 |
+
|
| 20 |
+
Anchor actual audio/features at position zero and zero-pad only at the tail. Pass the true valid length. Use the returned encoded length rather than treating all 188 output positions as valid. Handle empty/no-speech input in the host; this bundle does not include VAD.
|
| 21 |
+
|
| 22 |
+
## Frontend
|
| 23 |
+
|
| 24 |
+
Match the evaluated NeMo frontend: 16 kHz mono, preemphasis 0.97, FFT 512, nonperiodic Hann window 400 samples, hop 160, 128 Slaney-normalized mel filters, power spectrum, and natural log with additive guard `2**-24`. Inference dither is disabled. Normalize each mel channel over valid time frames using the mean and sample standard deviation plus `1e-5`; mask invalid frames to zero afterward. Valid mel length is the reference frontend's floor of sample count / 160, not simply every STFT output column.
|
| 25 |
+
|
| 26 |
+
For the long-recording validation, frontend features were computed independently for each overlapping audio window. Do not normalize across future, unavailable audio for a live implementation. Match centering/padding, mel filter construction and lengths against NeMo before attributing differences to quantization. The legacy README's `log(mel + 1e-5)` description was not the exact evaluated NeMo log guard; the values above record the actual reference. No frontend graph or new frontend implementation is included here.
|
| 27 |
+
|
| 28 |
+
## Prediction network and joint
|
| 29 |
+
|
| 30 |
+
Decoder inputs: token int64 `[1,1]`, hidden and cell float32 `[2,1,640]`. Outputs: prediction float32 `[1,1,640]`, updated hidden/cell of the same state shape.
|
| 31 |
+
|
| 32 |
+
Joint inputs: encoded frame float32 `[1,1024,1]`, transposed prediction float32 `[1,640,1]`. Output: float32 `[1,1,1,8198]`. Split logits into `[0:8193]` for tokens and `[8193:8198]` for durations `[0,1,2,3,4]`. Vocabulary contains 8192 pieces; blank ID is **8192**.
|
| 33 |
+
|
| 34 |
+
Initialize hidden/cell to zero and prime the decoder with blank. On non-blank emission, append the token, update decoder state, and advance by the predicted duration. On blank, advance by at least one frame without updating prediction state. Guard repeated zero-duration emissions, as in the validated host loop. Reset independent-window decoder state; do not pretend this offline encoder has a cache. Do not blindly collapse repeated text or token pieces.
|
| 35 |
+
|
| 36 |
+
## Continuous input
|
| 37 |
+
|
| 38 |
+
Tested profile: 496 new mel frames per commit, 496 left-context frames, 200 right-context frames. Each mel frame is 10 ms; encoder steps are 80 ms. Re-encode the overlapping window and commit only tokens in its target interval. No left zero-padding at the beginning of the recording. The profile is approximately 5-second updates after approximately 7 seconds of initial buffering, plus compute. Audio/frame timestamps are emission estimates, not verified word alignments. Boundary stitching remains imperfect.
|
| 39 |
+
|
| 40 |
+
## Memory and optional native mapping hint
|
| 41 |
+
|
| 42 |
+
The default INT8 runtime measured about 1.60 GB process peak on Linux. The optional verified source-page hint measured about 1.46 GB initialization peak and 0.954 GB retained RSS after the complete evaluation. The latter includes all three model objects; it is not a post-teardown measurement. Full English and long-recording token sequences were identical with and without the hint.
|
| 43 |
+
|
| 44 |
+
`mapped_model_file.h` is a Linux/Android native example, compiled and tested on Linux only. It owns an immutable file-backed mapping. If the chosen model API explicitly supports an externally owned read-only model buffer:
|
| 45 |
+
|
| 46 |
+
1. Construct the mapping owner before the model/interpreter and keep it alive until after they are destroyed.
|
| 47 |
+
2. Prepare the CPU delegate and run representative encoder warmup.
|
| 48 |
+
3. Call `DropVerifiedSourcePages()` once. It verifies mapped bytes against the file before `MADV_DONTNEED`.
|
| 49 |
+
4. Keep the mapping and file valid. Required pages can fault back in; this does not unmap the model.
|
| 50 |
+
|
| 51 |
+
This discards redundant resident source pages after weights have been packed. It does not shrink the file, guarantee immediate removal from the global OS file cache, or eliminate startup allocations. Some runtimes may continue reading source weights and refault more pages.
|
| 52 |
+
|
| 53 |
+
Never apply the hint to an anonymous or writable model ByteBuffer. Do not advise an unrelated second mapping and assume it affects the interpreter. Do not call it while inference is running. A Java file-path-only `CompiledModel` API may not expose the necessary mapping, so this is not a drop-in Java option. Ordinary INT8 remains usable without this optimization.
|
| 54 |
+
|
| 55 |
+
On Android, measure total application PSS/RSS, initialization overlap, sustained latency, page faults and thermals. The local ASR figures exclude the UI, microphone frontend, text/audio emotion models and a second ASR model. Keeping Nemotron and Parakeet resident together is not covered by this memory result. GPU/NPU delegates and INT8 coverage must be evaluated separately; retain CPU decoder/joint initially.
|
NOTICE.md
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
Derived from NVIDIA Parakeet-TDT-0.6B-v3, licensed CC BY 4.0: https://creativecommons.org/licenses/by/4.0/ . Original model and attribution: https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3 . Modified by community LiteRT export and post-training quantization; no additional training.
|
README.md
CHANGED
|
@@ -9,198 +9,88 @@ language:
|
|
| 9 |
- pt
|
| 10 |
library_name: litert
|
| 11 |
base_model: nvidia/parakeet-tdt-0.6b-v3
|
|
|
|
| 12 |
tags:
|
| 13 |
-
- automatic-speech-recognition
|
| 14 |
- speech
|
| 15 |
- audio
|
| 16 |
- parakeet
|
| 17 |
- tdt
|
| 18 |
-
- litert
|
| 19 |
- tflite
|
| 20 |
-
-
|
| 21 |
-
- mobile
|
| 22 |
- android
|
| 23 |
-
-
|
| 24 |
-
|
| 25 |
---
|
|
|
|
| 26 |
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
LiteRT (TFLite) port of
|
| 30 |
-
[`nvidia/parakeet-tdt-0.6b-v3`](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3),
|
| 31 |
-
packaged for on-device inference (Android / Mac / embedded) without a Python
|
| 32 |
-
or NeMo runtime dependency.
|
| 33 |
-
|
| 34 |
-
For **model capabilities, languages, training data, license, and benchmarks**,
|
| 35 |
-
see the upstream model card. This card only documents what's specific to the
|
| 36 |
-
LiteRT port.
|
| 37 |
-
|
| 38 |
-
## What's in this bundle
|
| 39 |
-
|
| 40 |
-
| File | Size | Purpose |
|
| 41 |
-
|---|---|---|
|
| 42 |
-
| `encoder_T1500.tflite` | 1.15 GB | FP16 encoder, fixed `T_mel = 1500` (15 s window) |
|
| 43 |
-
| `decoder_step.tflite` | 23 MB | Single-step LSTM prediction network |
|
| 44 |
-
| `joint_step.tflite` | 12 MB | TDT joint network (token + duration logits) |
|
| 45 |
-
| `tokenizer.model` | 353 KB | SentencePiece BPE tokenizer (vocab=8192) |
|
| 46 |
-
| `manifest.json` | — | All metadata the runtime needs |
|
| 47 |
-
|
| 48 |
-
Total: **~1.18 GB** (FP16). FP32 reference is ~2.37 GB.
|
| 49 |
-
|
| 50 |
-
## Encoder I/O contract
|
| 51 |
-
|
| 52 |
-
```
|
| 53 |
-
inputs:
|
| 54 |
-
audio_signal : float32 [1, 128, 1500] # log-mel features (NeMo preproc)
|
| 55 |
-
length : int32 [1] # actual mel frames used (≤ 1500)
|
| 56 |
-
outputs:
|
| 57 |
-
encoded : float32 [1, 1024, 188] # 188 = (1500 - 4) // 8
|
| 58 |
-
encoded_lengths : int32 [1]
|
| 59 |
-
```
|
| 60 |
-
|
| 61 |
-
Pad shorter inputs with zeros at the **tail** (the encoder was trained with
|
| 62 |
-
audio anchored at position 0; left-padding causes hallucinations) and pass
|
| 63 |
-
the true length.
|
| 64 |
|
| 65 |
-
The
|
| 66 |
-
encoder in a sliding-window streaming loop — see "Streaming usage" below.
|
| 67 |
|
| 68 |
-
|
| 69 |
-
NPU accelerator) reject int64 tensors entirely. With int64 length, every
|
| 70 |
-
internal CAST node touching it falls back to CPU, and `CompiledModel.create()`
|
| 71 |
-
fails outright on Android with the GPU backend. This bundle is exported with
|
| 72 |
-
int32 length end-to-end (input → internal mask arange/comparisons → output
|
| 73 |
-
`encoded_lengths`). int32 covers > 2 billion mel frames (~5 hours of audio),
|
| 74 |
-
so no practical range loss.
|
| 75 |
|
| 76 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
|
| 78 |
-
|
| 79 |
-
(300/500/700/1500) sharing weights inside one `.tflite`. The disk savings
|
| 80 |
-
were real (~1.2 GB instead of 4.8 GB for 4 separate files), but on Android
|
| 81 |
-
the LiteRT `CompiledModel.create()` API prepares **every** signature's
|
| 82 |
-
subgraph at load time — each one going through the full delegate-partition
|
| 83 |
-
pass. With 4 signatures × ~7 s of XNNPACK / GPU partition prep, app cold
|
| 84 |
-
start was ~28 s.
|
| 85 |
|
| 86 |
-
|
| 87 |
-
multiple bucket sizes for latency reasons, ship them as separate `.tflite`
|
| 88 |
-
files (TFLite has no cross-file weight sharing) and load on demand.
|
| 89 |
|
| 90 |
-
|
|
|
|
| 91 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 92 |
```
|
| 93 |
-
decoder_step:
|
| 94 |
-
inputs: token int64 [1,1], h float32 [2,1,640], c float32 [2,1,640]
|
| 95 |
-
outputs: g float32 [1,1,640], h float32 [2,1,640], c float32 [2,1,640]
|
| 96 |
-
|
| 97 |
-
joint_step:
|
| 98 |
-
inputs: enc_frame float32 [1,1024,1], pred_frame float32 [1,640,1]
|
| 99 |
-
outputs: logits float32 [1,1,1,8198]
|
| 100 |
-
# logits[..., 0:8193] → token logits (8192 BPE + 1 blank)
|
| 101 |
-
# logits[..., 8193:8198] → duration logits over [0,1,2,3,4]
|
| 102 |
-
```
|
| 103 |
-
|
| 104 |
-
`decoder_step.token` is `int64` because it's an embedding lookup; that op
|
| 105 |
-
runs on CPU regardless of delegate, so int64 there is harmless.
|
| 106 |
-
|
| 107 |
-
Greedy TDT decoding (per encoder frame):
|
| 108 |
|
| 109 |
-
|
| 110 |
-
2. `token = argmax(token_logits)`; `dur = durations[argmax(duration_logits)] ∈ {0,1,2,3,4}`.
|
| 111 |
-
3. If `token != blank_id (8192)`: emit token, advance `dur` encoder frames,
|
| 112 |
-
re-prime decoder with the emitted token (h, c update).
|
| 113 |
-
4. Else: advance `max(dur, 1)` encoder frames; do not advance the decoder.
|
| 114 |
-
5. Repeat until `enc_lengths` is exhausted.
|
| 115 |
|
| 116 |
-
|
| 117 |
-
pathological `dur=0` decode loop.
|
| 118 |
|
| 119 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 120 |
|
| 121 |
-
LiteRT
|
| 122 |
-
them. Match NeMo's preprocessor exactly:
|
| 123 |
-
|
| 124 |
-
```
|
| 125 |
-
sample_rate : 16000 Hz (resample if needed)
|
| 126 |
-
n_fft : 512
|
| 127 |
-
hop_length : 160 → 100 mel frames / second
|
| 128 |
-
win_length : 400
|
| 129 |
-
n_mels : 128
|
| 130 |
-
preemph : 0.97
|
| 131 |
-
log : log(mel + 1e-5), per-feature normalized
|
| 132 |
-
mel_scale : slaney
|
| 133 |
-
```
|
| 134 |
|
| 135 |
-
|
| 136 |
|
| 137 |
-
##
|
| 138 |
|
| 139 |
-
|
| 140 |
-
context window that fits inside 15 s. A reference Python implementation is
|
| 141 |
-
in the upstream repo (`transcribe_litert_streaming.py`). Recommended config
|
| 142 |
-
for Android UX:
|
| 143 |
|
| 144 |
-
|
| 145 |
-
|---|---|---|
|
| 146 |
-
| `chunk_seconds` | 5 | committed per step |
|
| 147 |
-
| `left_context_seconds` | 5 | encoder bilateral context |
|
| 148 |
-
| `right_context_seconds` | 2 | end-to-end latency ≈ 7 s |
|
| 149 |
-
| `window total` | 12 s | (5 + 5 + 2) × 100 = 1200 mel ≤ 1500 |
|
| 150 |
-
| `carry_state` | false | offline-trained model; carrying LSTM state across chunks tends to hurt |
|
| 151 |
|
| 152 |
-
|
| 153 |
-
code-switching) with this config, ~22 % on clean offline ≤15 s English.
|
| 154 |
|
| 155 |
-
##
|
| 156 |
|
| 157 |
-
-
|
| 158 |
-
- Bit-identical token output vs the upstream FP32 model on a 99-clip eval
|
| 159 |
-
set.
|
| 160 |
|
| 161 |
-
|
| 162 |
|
| 163 |
-
|
| 164 |
|
| 165 |
-
|
| 166 |
-
2. **ExportedProgram → TFLite** via
|
| 167 |
-
[`litert-torch`](https://github.com/google-ai-edge/LiteRT) 0.8.0.
|
| 168 |
-
3. **FP32 → FP16** via `ai_edge_quantizer` `FLOAT_CASTING` algorithm on
|
| 169 |
-
FC / Conv / DepthwiseConv / ConvTranspose / EmbeddingLookup ops.
|
| 170 |
|
| 171 |
-
|
| 172 |
|
| 173 |
-
|
| 174 |
-
remove `.expand(...)` patterns rejected by the TFLite broadcast checker.
|
| 175 |
-
- `RelPositionMultiHeadAttentionLongformer._get_invalid_locations_mask` — to
|
| 176 |
-
build masks in `bool` instead of `uint8` (litert-torch has no uint8
|
| 177 |
-
lowering).
|
| 178 |
-
- `ConformerEncoder.{forward_internal,_create_masks}` and
|
| 179 |
-
`MaskedConvSequential.{forward,_create_mask}` — to keep the entire length
|
| 180 |
-
pipeline in `int32` instead of NeMo's default `int64`, so LiteRT's
|
| 181 |
-
GPU/NPU delegates can compile the graph without falling back to CPU.
|
| 182 |
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
1. **Audio at position 0.** The encoder expects audio anchored at the start
|
| 186 |
-
of its input window. Padding before the audio causes hallucinations.
|
| 187 |
-
2. **15 s max per call.** Use the streaming chunker for longer clips.
|
| 188 |
-
3. **No VAD or diarization.** Pair with an external VAD or a diarizer
|
| 189 |
-
(e.g. Sortformer) for speaker-attributed transcripts.
|
| 190 |
-
4. **Multilingual but no language token.** Code-switching works, but the
|
| 191 |
-
model doesn't emit a language ID. Run a separate classifier if you need it.
|
| 192 |
-
|
| 193 |
-
## License
|
| 194 |
-
|
| 195 |
-
Inherits the upstream `nvidia/parakeet-tdt-0.6b-v3` license (CC-BY-4.0).
|
| 196 |
-
|
| 197 |
-
## Citation
|
| 198 |
-
|
| 199 |
-
```bibtex
|
| 200 |
-
@misc{nvidia_parakeet_tdt_0_6b_v3,
|
| 201 |
-
title = {Parakeet-TDT-0.6B-v3},
|
| 202 |
-
author = {NVIDIA},
|
| 203 |
-
year = {2025},
|
| 204 |
-
url = {https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3},
|
| 205 |
-
}
|
| 206 |
-
```
|
|
|
|
| 9 |
- pt
|
| 10 |
library_name: litert
|
| 11 |
base_model: nvidia/parakeet-tdt-0.6b-v3
|
| 12 |
+
pipeline_tag: automatic-speech-recognition
|
| 13 |
tags:
|
|
|
|
| 14 |
- speech
|
| 15 |
- audio
|
| 16 |
- parakeet
|
| 17 |
- tdt
|
|
|
|
| 18 |
- tflite
|
| 19 |
+
- int8
|
|
|
|
| 20 |
- android
|
| 21 |
+
- on-device
|
| 22 |
+
- buffered-streaming
|
| 23 |
---
|
| 24 |
+
# Parakeet-TDT-0.6B-v3 — INT8 LiteRT
|
| 25 |
|
| 26 |
+
Community conversion of [NVIDIA Parakeet-TDT-0.6B-v3](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3), with a dynamic INT8 encoder and the original FP16 decoder/joint. No additional training. This is not an official NVIDIA or Google release.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
+
**The repository root now contains the INT8 release.** The previous FP16 release, including its original ZIP and documentation, is preserved byte-for-byte in [legacy/](https://huggingface.co/spybyscript/parakeet-tdt-litert/tree/main/legacy). See [migration and integration instructions](INTEGRATION.md).
|
|
|
|
| 29 |
|
| 30 |
+
## Files and download
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
|
| 32 |
+
| File | Size, decimal MB | Purpose |
|
| 33 |
+
|---|---:|---|
|
| 34 |
+
| [encoder_T1500.tflite](encoder_T1500.tflite) | 628.6 | Dynamic INT8 encoder; one fixed 15-second input bucket |
|
| 35 |
+
| [decoder_step.tflite](decoder_step.tflite) | 23.7 | Original FP16 LSTM prediction network |
|
| 36 |
+
| [joint_step.tflite](joint_step.tflite) | 12.7 | Original FP16 token/duration joint network |
|
| 37 |
+
| [tokenizer.model](tokenizer.model) | 0.4 | Unchanged SentencePiece tokenizer |
|
| 38 |
+
| [manifest.json](manifest.json) | — | Tensor contracts, profile, component hashes and provenance |
|
| 39 |
+
| [validation.json](validation.json) | — | Measured quality and memory results, with limitations |
|
| 40 |
+
| [mapped_model_file.h](mapped_model_file.h) | — | Optional native source-page memory hint; Linux-tested example |
|
| 41 |
|
| 42 |
+
The four runtime assets total approximately **665 MB on disk**. Download them individually, or use the updated [ZIP bundle](https://huggingface.co/spybyscript/parakeet-tdt-litert/resolve/main/parakeet-tdt-0.6b-v3-litert.zip). Root [SHA256SUMS](SHA256SUMS) includes the ZIP checksum; the ZIP contains component checksums in its manifest.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
|
| 44 |
+
To avoid downloading the legacy release and duplicate ZIPs, select root files explicitly:
|
|
|
|
|
|
|
| 45 |
|
| 46 |
+
```python
|
| 47 |
+
from huggingface_hub import snapshot_download
|
| 48 |
|
| 49 |
+
snapshot_download(
|
| 50 |
+
"spybyscript/parakeet-tdt-litert",
|
| 51 |
+
local_dir="parakeet-int8",
|
| 52 |
+
allow_patterns=[
|
| 53 |
+
"encoder_T1500.tflite", "decoder_step.tflite", "joint_step.tflite",
|
| 54 |
+
"tokenizer.model", "manifest.json", "validation.json",
|
| 55 |
+
"README.md", "INTEGRATION.md", "NOTICE.md", "mapped_model_file.h",
|
| 56 |
+
],
|
| 57 |
+
)
|
| 58 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
|
| 60 |
+
Pin a repository revision in production. The root encoder filename is unchanged, but its content and precision have changed; invalidate caches using the manifest hashes or release ID `20260914-int8-t1500-v1`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 61 |
|
| 62 |
+
## Measured memory
|
|
|
|
| 63 |
|
| 64 |
+
| Linux CPU configuration | Process peak | Resident memory with all models retained |
|
| 65 |
+
|---|---:|---:|
|
| 66 |
+
| Previous FP16 encoder | 4.30 GB | — |
|
| 67 |
+
| INT8, ordinary runtime | 1.60 GB | — |
|
| 68 |
+
| INT8, verified native source-page hint after warmup | 1.46 GB | 0.954 GB RSS / 0.947 GB PSS |
|
| 69 |
|
| 70 |
+
All figures use decimal GB and include encoder, decoder and joint in an isolated ASR process. They exclude the Android UI, capture frontend and other ML models. Runtime: LiteRT 2.1.2/XNNPACK, four encoder threads and one decoder/joint thread. **Android, GPU and NPU memory/performance have not been validated for this release.**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
|
| 72 |
+
The approximately 954 MB result requires the optional file-mapping optimization described in [INTEGRATION.md](INTEGRATION.md). It is not automatic when downloading this model and is not a Java `CompiledModel` setting. Startup memory remains separately reported; clean source pages can remain in the OS file cache until reclaimed.
|
| 73 |
|
| 74 |
+
## Recognition validation
|
| 75 |
|
| 76 |
+
On 33 English synthetic clips with 745 source-text reference words, INT8 scored **4.30% word error**, versus **4.03%** for the prior FP16 baseline. These references were not independently transcribed by humans. INT8 matched 29/33 complete token sequences and 31/33 normalized word sequences; it is not an exact-parity replacement for FP16.
|
|
|
|
|
|
|
|
|
|
| 77 |
|
| 78 |
+
Four additional human long recordings totaled 159.758 seconds. They lack complete reference transcripts, so no human-reference WER is claimed. The native source-page hint preserved **all 37/37 token sequences versus ordinary INT8**. That parity claim concerns the memory hint, not quantization versus FP16.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 79 |
|
| 80 |
+
Those four long recordings took about 12.18 seconds of ordinary INT8 compute, or 11.42 seconds with the hint, versus 31.94 seconds for FP16. Frontend calculation, loading and warmup are excluded; other work was running on the machine. These timings are observations, not a controlled speed ranking or phone benchmark. The base model is multilingual; the current quantization evaluation was English only. Private audio, transcripts and token sequences are not distributed.
|
|
|
|
| 81 |
|
| 82 |
+
## Buffered live use
|
| 83 |
|
| 84 |
+
This is a component bundle with a host-side TDT decoder, not a single audio-to-text graph. It can process a continuous audio source using overlapping windows. The tested profile commits 4.96 seconds, with 4.96 seconds of left context and 2 seconds of right context, padded to the 15-second capacity.
|
|
|
|
|
|
|
| 85 |
|
| 86 |
+
Expect approximately **7 seconds of initial buffering and updates every 5 seconds**, plus compute. The 15-second input capacity does not mean waiting 15 seconds for every result. This is buffered processing of an offline-trained encoder; it does not have Nemotron's cache-aware encoder or frequent immediate partials. Boundary repetitions and recognition errors remain possible. Neither VAD nor speaker diarization is included.
|
| 87 |
|
| 88 |
+
## Precision and provenance
|
| 89 |
|
| 90 |
+
The encoder has 193 fully connected, 50 Conv2D and 27 depthwise-convolution operations with INT8 weights. Dynamic integer kernels consume floating-point activations/interfaces; this is not a fully integer end-to-end graph. The unchanged decoder/joint use FP16 weight storage and floating-point interfaces. Start with CPU/XNNPACK and test other backends independently.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 91 |
|
| 92 |
+
Conversion used the existing staged Parakeet program, PyTorch 2.11.0, litert-torch 0.9.1 and AI Edge Quantizer 0.7.0. The staged source-program SHA256 and runtime component hashes are in the manifest. INT4 alternatives were investigated locally but are not included in this default release because they changed more long-form words and omitted some phrases.
|
| 93 |
|
| 94 |
+
## License and attribution
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 95 |
|
| 96 |
+
Model weights derive from NVIDIA Parakeet-TDT-0.6B-v3 under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). Conversion and quantization by spybyscript; see [NOTICE.md](NOTICE.md). Consult the upstream card for base-model capabilities and training information. Upstream accuracy benchmarks are not measurements of this conversion.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
SHA256SUMS
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
11ad7efa24975ee4b0c3c3a38ed18737f0658a5f75a0a96787b576a78a023361 .gitattributes
|
| 2 |
+
510b2919183633eb53418a6c0ef65cf6b72ac6e04845893e3a67f70d840a2ccb INTEGRATION.md
|
| 3 |
+
5bf6b396ccf5b900a009f166a33d2464ba0fdce0ec117ad7f002cf5dc66ee14e NOTICE.md
|
| 4 |
+
c0dac492e179384912d4a623522d349d7f212a0012d5bc1db54c52adfea3cef4 README.md
|
| 5 |
+
eb0bf3559a0b4cbdc3ca05b7e8ff948ee5ef158ce424667b62a85f6c769a9ce1 decoder_step.tflite
|
| 6 |
+
b26804686f98d68f7394baf3b13bdcea8e846ebf4ecf2903e3fc30fe3a3882a8 encoder_T1500.tflite
|
| 7 |
+
0e28c22fc426df9900ef4a1bd15760ec757e44f0fd1818e0afb51c4fe79031be joint_step.tflite
|
| 8 |
+
2c6b2684937eaa67b2b6e68a0ad4aff96c0a114e8a135435de554822b258f3d0 legacy/archive.json
|
| 9 |
+
12bc7c5c126db8f7f97b617eb151d8e5b14d0ee3d57244f952693faa2a8fde98 manifest.json
|
| 10 |
+
4226d273861baa0328a5dcf8ef5d9eba604074137111ba53116f709ba63399c0 mapped_model_file.h
|
| 11 |
+
cd49232e6756b62ba14049bb31bab799838e30170b235a6993888e57662fdc95 parakeet-tdt-0.6b-v3-litert.zip
|
| 12 |
+
8efdaeebf2ead197dc5a265e07be3b184cce31056d53a2315a6a3e535f4dbcd3 publication.json
|
| 13 |
+
eacec2b0a77f336d4a2ca4a25a7047575d3c2b74de47e997f4c205126ed3135e tokenizer.model
|
| 14 |
+
f869d60f54a5d1cfbe2f3c73e2a1f3845fccf64109decc6cdd12345a89862b16 validation.json
|
encoder_T1500.tflite
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b26804686f98d68f7394baf3b13bdcea8e846ebf4ecf2903e3fc30fe3a3882a8
|
| 3 |
+
size 628579360
|
legacy/.gitattributes
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
legacy/README.md
ADDED
|
@@ -0,0 +1,206 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-4.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
- es
|
| 6 |
+
- it
|
| 7 |
+
- de
|
| 8 |
+
- fr
|
| 9 |
+
- pt
|
| 10 |
+
library_name: litert
|
| 11 |
+
base_model: nvidia/parakeet-tdt-0.6b-v3
|
| 12 |
+
tags:
|
| 13 |
+
- automatic-speech-recognition
|
| 14 |
+
- speech
|
| 15 |
+
- audio
|
| 16 |
+
- parakeet
|
| 17 |
+
- tdt
|
| 18 |
+
- litert
|
| 19 |
+
- tflite
|
| 20 |
+
- on-device
|
| 21 |
+
- mobile
|
| 22 |
+
- android
|
| 23 |
+
- streaming
|
| 24 |
+
pipeline_tag: automatic-speech-recognition
|
| 25 |
+
---
|
| 26 |
+
|
| 27 |
+
# Parakeet-TDT-0.6B-v3 — LiteRT (TFLite) port
|
| 28 |
+
|
| 29 |
+
LiteRT (TFLite) port of
|
| 30 |
+
[`nvidia/parakeet-tdt-0.6b-v3`](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3),
|
| 31 |
+
packaged for on-device inference (Android / Mac / embedded) without a Python
|
| 32 |
+
or NeMo runtime dependency.
|
| 33 |
+
|
| 34 |
+
For **model capabilities, languages, training data, license, and benchmarks**,
|
| 35 |
+
see the upstream model card. This card only documents what's specific to the
|
| 36 |
+
LiteRT port.
|
| 37 |
+
|
| 38 |
+
## What's in this bundle
|
| 39 |
+
|
| 40 |
+
| File | Size | Purpose |
|
| 41 |
+
|---|---|---|
|
| 42 |
+
| `encoder_T1500.tflite` | 1.15 GB | FP16 encoder, fixed `T_mel = 1500` (15 s window) |
|
| 43 |
+
| `decoder_step.tflite` | 23 MB | Single-step LSTM prediction network |
|
| 44 |
+
| `joint_step.tflite` | 12 MB | TDT joint network (token + duration logits) |
|
| 45 |
+
| `tokenizer.model` | 353 KB | SentencePiece BPE tokenizer (vocab=8192) |
|
| 46 |
+
| `manifest.json` | — | All metadata the runtime needs |
|
| 47 |
+
|
| 48 |
+
Total: **~1.18 GB** (FP16). FP32 reference is ~2.37 GB.
|
| 49 |
+
|
| 50 |
+
## Encoder I/O contract
|
| 51 |
+
|
| 52 |
+
```
|
| 53 |
+
inputs:
|
| 54 |
+
audio_signal : float32 [1, 128, 1500] # log-mel features (NeMo preproc)
|
| 55 |
+
length : int32 [1] # actual mel frames used (≤ 1500)
|
| 56 |
+
outputs:
|
| 57 |
+
encoded : float32 [1, 1024, 188] # 188 = (1500 - 4) // 8
|
| 58 |
+
encoded_lengths : int32 [1]
|
| 59 |
+
```
|
| 60 |
+
|
| 61 |
+
Pad shorter inputs with zeros at the **tail** (the encoder was trained with
|
| 62 |
+
audio anchored at position 0; left-padding causes hallucinations) and pass
|
| 63 |
+
the true length.
|
| 64 |
+
|
| 65 |
+
The 1500-mel bucket covers ≤ 15 s of audio. For long-form input, run the
|
| 66 |
+
encoder in a sliding-window streaming loop — see "Streaming usage" below.
|
| 67 |
+
|
| 68 |
+
**Why int32, not int64.** LiteRT's GPU/NPU delegates (LiteRT-CL / OpenCL,
|
| 69 |
+
NPU accelerator) reject int64 tensors entirely. With int64 length, every
|
| 70 |
+
internal CAST node touching it falls back to CPU, and `CompiledModel.create()`
|
| 71 |
+
fails outright on Android with the GPU backend. This bundle is exported with
|
| 72 |
+
int32 length end-to-end (input → internal mask arange/comparisons → output
|
| 73 |
+
`encoded_lengths`). int32 covers > 2 billion mel frames (~5 hours of audio),
|
| 74 |
+
so no practical range loss.
|
| 75 |
+
|
| 76 |
+
## Why a single bucket and not multi-signature
|
| 77 |
+
|
| 78 |
+
An earlier revision shipped a multi-signature encoder with 4 buckets
|
| 79 |
+
(300/500/700/1500) sharing weights inside one `.tflite`. The disk savings
|
| 80 |
+
were real (~1.2 GB instead of 4.8 GB for 4 separate files), but on Android
|
| 81 |
+
the LiteRT `CompiledModel.create()` API prepares **every** signature's
|
| 82 |
+
subgraph at load time — each one going through the full delegate-partition
|
| 83 |
+
pass. With 4 signatures × ~7 s of XNNPACK / GPU partition prep, app cold
|
| 84 |
+
start was ~28 s.
|
| 85 |
+
|
| 86 |
+
A single-bucket file is one subgraph: ~7 s init, then ready. If you need
|
| 87 |
+
multiple bucket sizes for latency reasons, ship them as separate `.tflite`
|
| 88 |
+
files (TFLite has no cross-file weight sharing) and load on demand.
|
| 89 |
+
|
| 90 |
+
## Decoder + joint contract
|
| 91 |
+
|
| 92 |
+
```
|
| 93 |
+
decoder_step:
|
| 94 |
+
inputs: token int64 [1,1], h float32 [2,1,640], c float32 [2,1,640]
|
| 95 |
+
outputs: g float32 [1,1,640], h float32 [2,1,640], c float32 [2,1,640]
|
| 96 |
+
|
| 97 |
+
joint_step:
|
| 98 |
+
inputs: enc_frame float32 [1,1024,1], pred_frame float32 [1,640,1]
|
| 99 |
+
outputs: logits float32 [1,1,1,8198]
|
| 100 |
+
# logits[..., 0:8193] → token logits (8192 BPE + 1 blank)
|
| 101 |
+
# logits[..., 8193:8198] → duration logits over [0,1,2,3,4]
|
| 102 |
+
```
|
| 103 |
+
|
| 104 |
+
`decoder_step.token` is `int64` because it's an embedding lookup; that op
|
| 105 |
+
runs on CPU regardless of delegate, so int64 there is harmless.
|
| 106 |
+
|
| 107 |
+
Greedy TDT decoding (per encoder frame):
|
| 108 |
+
|
| 109 |
+
1. Run joint with current `enc_frame` and last predicted `pred_frame`.
|
| 110 |
+
2. `token = argmax(token_logits)`; `dur = durations[argmax(duration_logits)] ∈ {0,1,2,3,4}`.
|
| 111 |
+
3. If `token != blank_id (8192)`: emit token, advance `dur` encoder frames,
|
| 112 |
+
re-prime decoder with the emitted token (h, c update).
|
| 113 |
+
4. Else: advance `max(dur, 1)` encoder frames; do not advance the decoder.
|
| 114 |
+
5. Repeat until `enc_lengths` is exhausted.
|
| 115 |
+
|
| 116 |
+
Cap at ~10 non-blank emissions per encoder frame to guard against the
|
| 117 |
+
pathological `dur=0` decode loop.
|
| 118 |
+
|
| 119 |
+
## Audio preprocessing
|
| 120 |
+
|
| 121 |
+
LiteRT itself does not produce mel features — your runtime must compute
|
| 122 |
+
them. Match NeMo's preprocessor exactly:
|
| 123 |
+
|
| 124 |
+
```
|
| 125 |
+
sample_rate : 16000 Hz (resample if needed)
|
| 126 |
+
n_fft : 512
|
| 127 |
+
hop_length : 160 → 100 mel frames / second
|
| 128 |
+
win_length : 400
|
| 129 |
+
n_mels : 128
|
| 130 |
+
preemph : 0.97
|
| 131 |
+
log : log(mel + 1e-5), per-feature normalized
|
| 132 |
+
mel_scale : slaney
|
| 133 |
+
```
|
| 134 |
+
|
| 135 |
+
Encoder frame rate after the 8× subsampler: **12.5 fps** (1 enc frame = 80 ms).
|
| 136 |
+
|
| 137 |
+
## Streaming usage
|
| 138 |
+
|
| 139 |
+
This bundle supports chunked streaming inference using a left+chunk+right
|
| 140 |
+
context window that fits inside 15 s. A reference Python implementation is
|
| 141 |
+
in the upstream repo (`transcribe_litert_streaming.py`). Recommended config
|
| 142 |
+
for Android UX:
|
| 143 |
+
|
| 144 |
+
| Knob | Value | Reason |
|
| 145 |
+
|---|---|---|
|
| 146 |
+
| `chunk_seconds` | 5 | committed per step |
|
| 147 |
+
| `left_context_seconds` | 5 | encoder bilateral context |
|
| 148 |
+
| `right_context_seconds` | 2 | end-to-end latency ≈ 7 s |
|
| 149 |
+
| `window total` | 12 s | (5 + 5 + 2) × 100 = 1200 mel ≤ 1500 |
|
| 150 |
+
| `carry_state` | false | offline-trained model; carrying LSTM state across chunks tends to hurt |
|
| 151 |
+
|
| 152 |
+
We measured ~27 % WER on multilingual long-form audio (EN/ES/IT
|
| 153 |
+
code-switching) with this config, ~22 % on clean offline ≤15 s English.
|
| 154 |
+
|
| 155 |
+
## Quantization
|
| 156 |
+
|
| 157 |
+
- All `.tflite` weights are FP16. Activations remain FP32.
|
| 158 |
+
- Bit-identical token output vs the upstream FP32 model on a 99-clip eval
|
| 159 |
+
set.
|
| 160 |
+
|
| 161 |
+
## Conversion provenance
|
| 162 |
+
|
| 163 |
+
Built from upstream `nvidia/parakeet-tdt-0.6b-v3.nemo` via:
|
| 164 |
+
|
| 165 |
+
1. **NeMo → torch.export ExportedProgram** (per encoder/decoder/joint module).
|
| 166 |
+
2. **ExportedProgram → TFLite** via
|
| 167 |
+
[`litert-torch`](https://github.com/google-ai-edge/LiteRT) 0.8.0.
|
| 168 |
+
3. **FP32 → FP16** via `ai_edge_quantizer` `FLOAT_CASTING` algorithm on
|
| 169 |
+
FC / Conv / DepthwiseConv / ConvTranspose / EmbeddingLookup ops.
|
| 170 |
+
|
| 171 |
+
Several NeMo internals required export-time monkey-patches:
|
| 172 |
+
|
| 173 |
+
- `MaskedConvSequential.{forward,_create_mask}` and `apply_channel_mask` — to
|
| 174 |
+
remove `.expand(...)` patterns rejected by the TFLite broadcast checker.
|
| 175 |
+
- `RelPositionMultiHeadAttentionLongformer._get_invalid_locations_mask` — to
|
| 176 |
+
build masks in `bool` instead of `uint8` (litert-torch has no uint8
|
| 177 |
+
lowering).
|
| 178 |
+
- `ConformerEncoder.{forward_internal,_create_masks}` and
|
| 179 |
+
`MaskedConvSequential.{forward,_create_mask}` — to keep the entire length
|
| 180 |
+
pipeline in `int32` instead of NeMo's default `int64`, so LiteRT's
|
| 181 |
+
GPU/NPU delegates can compile the graph without falling back to CPU.
|
| 182 |
+
|
| 183 |
+
## Limitations
|
| 184 |
+
|
| 185 |
+
1. **Audio at position 0.** The encoder expects audio anchored at the start
|
| 186 |
+
of its input window. Padding before the audio causes hallucinations.
|
| 187 |
+
2. **15 s max per call.** Use the streaming chunker for longer clips.
|
| 188 |
+
3. **No VAD or diarization.** Pair with an external VAD or a diarizer
|
| 189 |
+
(e.g. Sortformer) for speaker-attributed transcripts.
|
| 190 |
+
4. **Multilingual but no language token.** Code-switching works, but the
|
| 191 |
+
model doesn't emit a language ID. Run a separate classifier if you need it.
|
| 192 |
+
|
| 193 |
+
## License
|
| 194 |
+
|
| 195 |
+
Inherits the upstream `nvidia/parakeet-tdt-0.6b-v3` license (CC-BY-4.0).
|
| 196 |
+
|
| 197 |
+
## Citation
|
| 198 |
+
|
| 199 |
+
```bibtex
|
| 200 |
+
@misc{nvidia_parakeet_tdt_0_6b_v3,
|
| 201 |
+
title = {Parakeet-TDT-0.6B-v3},
|
| 202 |
+
author = {NVIDIA},
|
| 203 |
+
year = {2025},
|
| 204 |
+
url = {https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3},
|
| 205 |
+
}
|
| 206 |
+
```
|
legacy/archive.json
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"source_repo": "spybyscript/parakeet-tdt-litert",
|
| 3 |
+
"source_revision": "c528ceb57bbc4e366993e50c79b22e0417a68484",
|
| 4 |
+
"layout": "Original release files preserved byte-for-byte under legacy/",
|
| 5 |
+
"files": [
|
| 6 |
+
{
|
| 7 |
+
"path": ".gitattributes",
|
| 8 |
+
"bytes": 1519,
|
| 9 |
+
"blob_id": "a6344aac8c09253b3b630fb776ae94478aa0275b",
|
| 10 |
+
"lfs_sha256": null
|
| 11 |
+
},
|
| 12 |
+
{
|
| 13 |
+
"path": "README.md",
|
| 14 |
+
"bytes": 7545,
|
| 15 |
+
"blob_id": "99bedadd0d487c3777086d131e2f4f06294fe1eb",
|
| 16 |
+
"lfs_sha256": null
|
| 17 |
+
},
|
| 18 |
+
{
|
| 19 |
+
"path": "decoder_step.tflite",
|
| 20 |
+
"bytes": 23650084,
|
| 21 |
+
"blob_id": "ef4d5b35b1e08b89be5a45d16632197369b95dd2",
|
| 22 |
+
"lfs_sha256": "eb0bf3559a0b4cbdc3ca05b7e8ff948ee5ef158ce424667b62a85f6c769a9ce1"
|
| 23 |
+
},
|
| 24 |
+
{
|
| 25 |
+
"path": "encoder_T1500.tflite",
|
| 26 |
+
"bytes": 1206789364,
|
| 27 |
+
"blob_id": "f97b8f24f9759ef4963a1d08dc057a2b31ebb032",
|
| 28 |
+
"lfs_sha256": "8f826f40ab8e50c0e05ff1532d1cedcaa9c79c34909f1c14d9ac86c0643db45b"
|
| 29 |
+
},
|
| 30 |
+
{
|
| 31 |
+
"path": "joint_step.tflite",
|
| 32 |
+
"bytes": 12664976,
|
| 33 |
+
"blob_id": "83121aa7742953278fe8d41a5319153725644348",
|
| 34 |
+
"lfs_sha256": "0e28c22fc426df9900ef4a1bd15760ec757e44f0fd1818e0afb51c4fe79031be"
|
| 35 |
+
},
|
| 36 |
+
{
|
| 37 |
+
"path": "manifest.json",
|
| 38 |
+
"bytes": 6294,
|
| 39 |
+
"blob_id": "6fe030289b5aad6912c07247c5e479e055c37150",
|
| 40 |
+
"lfs_sha256": null
|
| 41 |
+
},
|
| 42 |
+
{
|
| 43 |
+
"path": "parakeet-tdt-0.6b-v3-litert.zip",
|
| 44 |
+
"bytes": 1243526688,
|
| 45 |
+
"blob_id": "8a122cdd57d101647c43d375eb8fcd02af8fbb47",
|
| 46 |
+
"lfs_sha256": "da306f15d3835e939fe8585933709646c01621f4570f3186121cb435d50fd1ac"
|
| 47 |
+
},
|
| 48 |
+
{
|
| 49 |
+
"path": "tokenizer.model",
|
| 50 |
+
"bytes": 360916,
|
| 51 |
+
"blob_id": "60c11e1e1ac25a6514cb2d27cf8c1a7590ed2833",
|
| 52 |
+
"lfs_sha256": "eacec2b0a77f336d4a2ca4a25a7047575d3c2b74de47e997f4c205126ed3135e"
|
| 53 |
+
}
|
| 54 |
+
]
|
| 55 |
+
}
|
legacy/decoder_step.tflite
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:eb0bf3559a0b4cbdc3ca05b7e8ff948ee5ef158ce424667b62a85f6c769a9ce1
|
| 3 |
+
size 23650084
|
legacy/encoder_T1500.tflite
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8f826f40ab8e50c0e05ff1532d1cedcaa9c79c34909f1c14d9ac86c0643db45b
|
| 3 |
+
size 1206789364
|
legacy/joint_step.tflite
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0e28c22fc426df9900ef4a1bd15760ec757e44f0fd1818e0afb51c4fe79031be
|
| 3 |
+
size 12664976
|
legacy/manifest.json
ADDED
|
@@ -0,0 +1,345 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model": "nvidia/parakeet-tdt-0.6b-v3",
|
| 3 |
+
"torch_version": "2.11.0+cu130",
|
| 4 |
+
"model_class": "EncDecRNNTBPEModel",
|
| 5 |
+
"vocab_size": 8192,
|
| 6 |
+
"blank_id": 8192,
|
| 7 |
+
"durations": [
|
| 8 |
+
0,
|
| 9 |
+
1,
|
| 10 |
+
2,
|
| 11 |
+
3,
|
| 12 |
+
4
|
| 13 |
+
],
|
| 14 |
+
"num_durations": 5,
|
| 15 |
+
"joint_output_dim": 8198,
|
| 16 |
+
"joint_token_logits_slice": [
|
| 17 |
+
0,
|
| 18 |
+
8193
|
| 19 |
+
],
|
| 20 |
+
"joint_duration_logits_slice": [
|
| 21 |
+
8193,
|
| 22 |
+
8198
|
| 23 |
+
],
|
| 24 |
+
"encoder": {
|
| 25 |
+
"d_model": 1024,
|
| 26 |
+
"subsampling_factor": 8,
|
| 27 |
+
"n_layers": 24,
|
| 28 |
+
"n_heads": 8,
|
| 29 |
+
"feat_in": 128,
|
| 30 |
+
"attention_mode": "rel_pos",
|
| 31 |
+
"att_context_size": null,
|
| 32 |
+
"buckets": [
|
| 33 |
+
{
|
| 34 |
+
"n_mel_frames": 1500,
|
| 35 |
+
"n_encoder_frames": 187,
|
| 36 |
+
"input_shape": [
|
| 37 |
+
1,
|
| 38 |
+
128,
|
| 39 |
+
1500
|
| 40 |
+
],
|
| 41 |
+
"output_shape": [
|
| 42 |
+
1,
|
| 43 |
+
1024,
|
| 44 |
+
188
|
| 45 |
+
],
|
| 46 |
+
"artifact": "encoder_T1500.pt2",
|
| 47 |
+
"size_mb": 2366.85
|
| 48 |
+
}
|
| 49 |
+
],
|
| 50 |
+
"multisig": false
|
| 51 |
+
},
|
| 52 |
+
"decoder": {
|
| 53 |
+
"num_layers": 2,
|
| 54 |
+
"hidden": 640,
|
| 55 |
+
"embed_dim": 640
|
| 56 |
+
},
|
| 57 |
+
"joint": {
|
| 58 |
+
"d_enc": 1024,
|
| 59 |
+
"d_pred": 640,
|
| 60 |
+
"joint_dim": 640
|
| 61 |
+
},
|
| 62 |
+
"preprocessor": {
|
| 63 |
+
"sample_rate": 16000,
|
| 64 |
+
"n_fft": 512,
|
| 65 |
+
"win_length": 400,
|
| 66 |
+
"hop_length": 160,
|
| 67 |
+
"n_mels": 128,
|
| 68 |
+
"preemph": 0.97,
|
| 69 |
+
"log": true,
|
| 70 |
+
"frame_rate_hz_post_subsample": 12.5
|
| 71 |
+
},
|
| 72 |
+
"artifacts": {
|
| 73 |
+
"decoder_step": {
|
| 74 |
+
"filename": "decoder_step.pt2",
|
| 75 |
+
"size_mb": 45.07,
|
| 76 |
+
"input_shapes": {
|
| 77 |
+
"token": [
|
| 78 |
+
1,
|
| 79 |
+
1
|
| 80 |
+
],
|
| 81 |
+
"h": [
|
| 82 |
+
2,
|
| 83 |
+
1,
|
| 84 |
+
640
|
| 85 |
+
],
|
| 86 |
+
"c": [
|
| 87 |
+
2,
|
| 88 |
+
1,
|
| 89 |
+
640
|
| 90 |
+
]
|
| 91 |
+
},
|
| 92 |
+
"input_dtypes": {
|
| 93 |
+
"token": "int64",
|
| 94 |
+
"h": "float32",
|
| 95 |
+
"c": "float32"
|
| 96 |
+
},
|
| 97 |
+
"output_shapes": {
|
| 98 |
+
"g": [
|
| 99 |
+
1,
|
| 100 |
+
1,
|
| 101 |
+
640
|
| 102 |
+
],
|
| 103 |
+
"h": [
|
| 104 |
+
2,
|
| 105 |
+
1,
|
| 106 |
+
640
|
| 107 |
+
],
|
| 108 |
+
"c": [
|
| 109 |
+
2,
|
| 110 |
+
1,
|
| 111 |
+
640
|
| 112 |
+
]
|
| 113 |
+
}
|
| 114 |
+
},
|
| 115 |
+
"joint_step": {
|
| 116 |
+
"filename": "joint_step.pt2",
|
| 117 |
+
"size_mb": 24.14,
|
| 118 |
+
"input_shapes": {
|
| 119 |
+
"enc_frame": [
|
| 120 |
+
1,
|
| 121 |
+
1024,
|
| 122 |
+
1
|
| 123 |
+
],
|
| 124 |
+
"pred_frame": [
|
| 125 |
+
1,
|
| 126 |
+
640,
|
| 127 |
+
1
|
| 128 |
+
]
|
| 129 |
+
},
|
| 130 |
+
"output_shape": [
|
| 131 |
+
1,
|
| 132 |
+
1,
|
| 133 |
+
1,
|
| 134 |
+
8198
|
| 135 |
+
]
|
| 136 |
+
}
|
| 137 |
+
},
|
| 138 |
+
"tokenizer": {
|
| 139 |
+
"saved": true,
|
| 140 |
+
"method": "serialized_model_proto",
|
| 141 |
+
"vocab_size": 8192
|
| 142 |
+
},
|
| 143 |
+
"litert": {
|
| 144 |
+
"quant": "fp16",
|
| 145 |
+
"results": [
|
| 146 |
+
{
|
| 147 |
+
"graph": "encoder",
|
| 148 |
+
"source_artifact": "encoder_T1500.pt2",
|
| 149 |
+
"output_artifact": "encoder_T1500.tflite",
|
| 150 |
+
"size_mb": 1150.88,
|
| 151 |
+
"convert_seconds": 158.59,
|
| 152 |
+
"quant": "fp16",
|
| 153 |
+
"torch_output_shapes": [
|
| 154 |
+
[
|
| 155 |
+
1,
|
| 156 |
+
1024,
|
| 157 |
+
188
|
| 158 |
+
],
|
| 159 |
+
[
|
| 160 |
+
1
|
| 161 |
+
]
|
| 162 |
+
],
|
| 163 |
+
"parity": {
|
| 164 |
+
"ok": true,
|
| 165 |
+
"max_abs_diff": 0.0,
|
| 166 |
+
"per_output_diffs": [
|
| 167 |
+
[
|
| 168 |
+
"shape mismatch",
|
| 169 |
+
[
|
| 170 |
+
1
|
| 171 |
+
],
|
| 172 |
+
[
|
| 173 |
+
1,
|
| 174 |
+
1024,
|
| 175 |
+
188
|
| 176 |
+
]
|
| 177 |
+
],
|
| 178 |
+
[
|
| 179 |
+
"shape mismatch",
|
| 180 |
+
[
|
| 181 |
+
1,
|
| 182 |
+
1024,
|
| 183 |
+
188
|
| 184 |
+
],
|
| 185 |
+
[
|
| 186 |
+
1
|
| 187 |
+
]
|
| 188 |
+
]
|
| 189 |
+
],
|
| 190 |
+
"tflite_output_shapes": [
|
| 191 |
+
[
|
| 192 |
+
1
|
| 193 |
+
],
|
| 194 |
+
[
|
| 195 |
+
1,
|
| 196 |
+
1024,
|
| 197 |
+
188
|
| 198 |
+
]
|
| 199 |
+
],
|
| 200 |
+
"torch_output_shapes": [
|
| 201 |
+
[
|
| 202 |
+
1,
|
| 203 |
+
1024,
|
| 204 |
+
188
|
| 205 |
+
],
|
| 206 |
+
[
|
| 207 |
+
1
|
| 208 |
+
]
|
| 209 |
+
]
|
| 210 |
+
}
|
| 211 |
+
},
|
| 212 |
+
{
|
| 213 |
+
"graph": "decoder_step",
|
| 214 |
+
"source_artifact": "decoder_step.pt2",
|
| 215 |
+
"output_artifact": "decoder_step.tflite",
|
| 216 |
+
"size_mb": 22.55,
|
| 217 |
+
"convert_seconds": 1.92,
|
| 218 |
+
"quant": "fp16",
|
| 219 |
+
"torch_output_shapes": [
|
| 220 |
+
[
|
| 221 |
+
1,
|
| 222 |
+
1,
|
| 223 |
+
640
|
| 224 |
+
],
|
| 225 |
+
[
|
| 226 |
+
2,
|
| 227 |
+
1,
|
| 228 |
+
640
|
| 229 |
+
],
|
| 230 |
+
[
|
| 231 |
+
2,
|
| 232 |
+
1,
|
| 233 |
+
640
|
| 234 |
+
]
|
| 235 |
+
],
|
| 236 |
+
"parity": {
|
| 237 |
+
"ok": true,
|
| 238 |
+
"max_abs_diff": 0.0044100284576416016,
|
| 239 |
+
"per_output_diffs": [
|
| 240 |
+
[
|
| 241 |
+
"shape mismatch",
|
| 242 |
+
[
|
| 243 |
+
2,
|
| 244 |
+
1,
|
| 245 |
+
640
|
| 246 |
+
],
|
| 247 |
+
[
|
| 248 |
+
1,
|
| 249 |
+
1,
|
| 250 |
+
640
|
| 251 |
+
]
|
| 252 |
+
],
|
| 253 |
+
[
|
| 254 |
+
"shape mismatch",
|
| 255 |
+
[
|
| 256 |
+
1,
|
| 257 |
+
1,
|
| 258 |
+
640
|
| 259 |
+
],
|
| 260 |
+
[
|
| 261 |
+
2,
|
| 262 |
+
1,
|
| 263 |
+
640
|
| 264 |
+
]
|
| 265 |
+
],
|
| 266 |
+
0.0044100284576416016
|
| 267 |
+
],
|
| 268 |
+
"tflite_output_shapes": [
|
| 269 |
+
[
|
| 270 |
+
2,
|
| 271 |
+
1,
|
| 272 |
+
640
|
| 273 |
+
],
|
| 274 |
+
[
|
| 275 |
+
1,
|
| 276 |
+
1,
|
| 277 |
+
640
|
| 278 |
+
],
|
| 279 |
+
[
|
| 280 |
+
2,
|
| 281 |
+
1,
|
| 282 |
+
640
|
| 283 |
+
]
|
| 284 |
+
],
|
| 285 |
+
"torch_output_shapes": [
|
| 286 |
+
[
|
| 287 |
+
1,
|
| 288 |
+
1,
|
| 289 |
+
640
|
| 290 |
+
],
|
| 291 |
+
[
|
| 292 |
+
2,
|
| 293 |
+
1,
|
| 294 |
+
640
|
| 295 |
+
],
|
| 296 |
+
[
|
| 297 |
+
2,
|
| 298 |
+
1,
|
| 299 |
+
640
|
| 300 |
+
]
|
| 301 |
+
]
|
| 302 |
+
}
|
| 303 |
+
},
|
| 304 |
+
{
|
| 305 |
+
"graph": "joint_step",
|
| 306 |
+
"source_artifact": "joint_step.pt2",
|
| 307 |
+
"output_artifact": "joint_step.tflite",
|
| 308 |
+
"size_mb": 12.08,
|
| 309 |
+
"convert_seconds": 1.61,
|
| 310 |
+
"quant": "fp16",
|
| 311 |
+
"torch_output_shapes": [
|
| 312 |
+
[
|
| 313 |
+
1,
|
| 314 |
+
1,
|
| 315 |
+
1,
|
| 316 |
+
8198
|
| 317 |
+
]
|
| 318 |
+
],
|
| 319 |
+
"parity": {
|
| 320 |
+
"ok": true,
|
| 321 |
+
"max_abs_diff": 0.408447265625,
|
| 322 |
+
"per_output_diffs": [
|
| 323 |
+
0.408447265625
|
| 324 |
+
],
|
| 325 |
+
"tflite_output_shapes": [
|
| 326 |
+
[
|
| 327 |
+
1,
|
| 328 |
+
1,
|
| 329 |
+
1,
|
| 330 |
+
8198
|
| 331 |
+
]
|
| 332 |
+
],
|
| 333 |
+
"torch_output_shapes": [
|
| 334 |
+
[
|
| 335 |
+
1,
|
| 336 |
+
1,
|
| 337 |
+
1,
|
| 338 |
+
8198
|
| 339 |
+
]
|
| 340 |
+
]
|
| 341 |
+
}
|
| 342 |
+
}
|
| 343 |
+
]
|
| 344 |
+
}
|
| 345 |
+
}
|
legacy/parakeet-tdt-0.6b-v3-litert.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:da306f15d3835e939fe8585933709646c01621f4570f3186121cb435d50fd1ac
|
| 3 |
+
size 1243526688
|
legacy/tokenizer.model
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:eacec2b0a77f336d4a2ca4a25a7047575d3c2b74de47e997f4c205126ed3135e
|
| 3 |
+
size 360916
|
manifest.json
CHANGED
|
@@ -1,7 +1,5 @@
|
|
| 1 |
{
|
| 2 |
"model": "nvidia/parakeet-tdt-0.6b-v3",
|
| 3 |
-
"torch_version": "2.11.0+cu130",
|
| 4 |
-
"model_class": "EncDecRNNTBPEModel",
|
| 5 |
"vocab_size": 8192,
|
| 6 |
"blank_id": 8192,
|
| 7 |
"durations": [
|
|
@@ -32,7 +30,7 @@
|
|
| 32 |
"buckets": [
|
| 33 |
{
|
| 34 |
"n_mel_frames": 1500,
|
| 35 |
-
"n_encoder_frames":
|
| 36 |
"input_shape": [
|
| 37 |
1,
|
| 38 |
128,
|
|
@@ -43,8 +41,8 @@
|
|
| 43 |
1024,
|
| 44 |
188
|
| 45 |
],
|
| 46 |
-
"artifact": "encoder_T1500.
|
| 47 |
-
"
|
| 48 |
}
|
| 49 |
],
|
| 50 |
"multisig": false
|
|
@@ -67,73 +65,15 @@
|
|
| 67 |
"n_mels": 128,
|
| 68 |
"preemph": 0.97,
|
| 69 |
"log": true,
|
| 70 |
-
"frame_rate_hz_post_subsample": 12.5
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
"
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
1
|
| 80 |
-
],
|
| 81 |
-
"h": [
|
| 82 |
-
2,
|
| 83 |
-
1,
|
| 84 |
-
640
|
| 85 |
-
],
|
| 86 |
-
"c": [
|
| 87 |
-
2,
|
| 88 |
-
1,
|
| 89 |
-
640
|
| 90 |
-
]
|
| 91 |
-
},
|
| 92 |
-
"input_dtypes": {
|
| 93 |
-
"token": "int64",
|
| 94 |
-
"h": "float32",
|
| 95 |
-
"c": "float32"
|
| 96 |
-
},
|
| 97 |
-
"output_shapes": {
|
| 98 |
-
"g": [
|
| 99 |
-
1,
|
| 100 |
-
1,
|
| 101 |
-
640
|
| 102 |
-
],
|
| 103 |
-
"h": [
|
| 104 |
-
2,
|
| 105 |
-
1,
|
| 106 |
-
640
|
| 107 |
-
],
|
| 108 |
-
"c": [
|
| 109 |
-
2,
|
| 110 |
-
1,
|
| 111 |
-
640
|
| 112 |
-
]
|
| 113 |
-
}
|
| 114 |
-
},
|
| 115 |
-
"joint_step": {
|
| 116 |
-
"filename": "joint_step.pt2",
|
| 117 |
-
"size_mb": 24.14,
|
| 118 |
-
"input_shapes": {
|
| 119 |
-
"enc_frame": [
|
| 120 |
-
1,
|
| 121 |
-
1024,
|
| 122 |
-
1
|
| 123 |
-
],
|
| 124 |
-
"pred_frame": [
|
| 125 |
-
1,
|
| 126 |
-
640,
|
| 127 |
-
1
|
| 128 |
-
]
|
| 129 |
-
},
|
| 130 |
-
"output_shape": [
|
| 131 |
-
1,
|
| 132 |
-
1,
|
| 133 |
-
1,
|
| 134 |
-
8198
|
| 135 |
-
]
|
| 136 |
-
}
|
| 137 |
},
|
| 138 |
"tokenizer": {
|
| 139 |
"saved": true,
|
|
@@ -141,205 +81,93 @@
|
|
| 141 |
"vocab_size": 8192
|
| 142 |
},
|
| 143 |
"litert": {
|
| 144 |
-
"quant": "
|
| 145 |
"results": [
|
| 146 |
{
|
| 147 |
"graph": "encoder",
|
| 148 |
-
"
|
| 149 |
-
"output_artifact": "encoder_T1500.tflite",
|
| 150 |
-
"size_mb": 1150.88,
|
| 151 |
-
"convert_seconds": 158.59,
|
| 152 |
-
"quant": "fp16",
|
| 153 |
-
"torch_output_shapes": [
|
| 154 |
-
[
|
| 155 |
-
1,
|
| 156 |
-
1024,
|
| 157 |
-
188
|
| 158 |
-
],
|
| 159 |
-
[
|
| 160 |
-
1
|
| 161 |
-
]
|
| 162 |
-
],
|
| 163 |
-
"parity": {
|
| 164 |
-
"ok": true,
|
| 165 |
-
"max_abs_diff": 0.0,
|
| 166 |
-
"per_output_diffs": [
|
| 167 |
-
[
|
| 168 |
-
"shape mismatch",
|
| 169 |
-
[
|
| 170 |
-
1
|
| 171 |
-
],
|
| 172 |
-
[
|
| 173 |
-
1,
|
| 174 |
-
1024,
|
| 175 |
-
188
|
| 176 |
-
]
|
| 177 |
-
],
|
| 178 |
-
[
|
| 179 |
-
"shape mismatch",
|
| 180 |
-
[
|
| 181 |
-
1,
|
| 182 |
-
1024,
|
| 183 |
-
188
|
| 184 |
-
],
|
| 185 |
-
[
|
| 186 |
-
1
|
| 187 |
-
]
|
| 188 |
-
]
|
| 189 |
-
],
|
| 190 |
-
"tflite_output_shapes": [
|
| 191 |
-
[
|
| 192 |
-
1
|
| 193 |
-
],
|
| 194 |
-
[
|
| 195 |
-
1,
|
| 196 |
-
1024,
|
| 197 |
-
188
|
| 198 |
-
]
|
| 199 |
-
],
|
| 200 |
-
"torch_output_shapes": [
|
| 201 |
-
[
|
| 202 |
-
1,
|
| 203 |
-
1024,
|
| 204 |
-
188
|
| 205 |
-
],
|
| 206 |
-
[
|
| 207 |
-
1
|
| 208 |
-
]
|
| 209 |
-
]
|
| 210 |
-
}
|
| 211 |
},
|
| 212 |
{
|
| 213 |
"graph": "decoder_step",
|
| 214 |
-
"
|
| 215 |
-
"output_artifact": "decoder_step.tflite",
|
| 216 |
-
"size_mb": 22.55,
|
| 217 |
-
"convert_seconds": 1.92,
|
| 218 |
-
"quant": "fp16",
|
| 219 |
-
"torch_output_shapes": [
|
| 220 |
-
[
|
| 221 |
-
1,
|
| 222 |
-
1,
|
| 223 |
-
640
|
| 224 |
-
],
|
| 225 |
-
[
|
| 226 |
-
2,
|
| 227 |
-
1,
|
| 228 |
-
640
|
| 229 |
-
],
|
| 230 |
-
[
|
| 231 |
-
2,
|
| 232 |
-
1,
|
| 233 |
-
640
|
| 234 |
-
]
|
| 235 |
-
],
|
| 236 |
-
"parity": {
|
| 237 |
-
"ok": true,
|
| 238 |
-
"max_abs_diff": 0.0044100284576416016,
|
| 239 |
-
"per_output_diffs": [
|
| 240 |
-
[
|
| 241 |
-
"shape mismatch",
|
| 242 |
-
[
|
| 243 |
-
2,
|
| 244 |
-
1,
|
| 245 |
-
640
|
| 246 |
-
],
|
| 247 |
-
[
|
| 248 |
-
1,
|
| 249 |
-
1,
|
| 250 |
-
640
|
| 251 |
-
]
|
| 252 |
-
],
|
| 253 |
-
[
|
| 254 |
-
"shape mismatch",
|
| 255 |
-
[
|
| 256 |
-
1,
|
| 257 |
-
1,
|
| 258 |
-
640
|
| 259 |
-
],
|
| 260 |
-
[
|
| 261 |
-
2,
|
| 262 |
-
1,
|
| 263 |
-
640
|
| 264 |
-
]
|
| 265 |
-
],
|
| 266 |
-
0.0044100284576416016
|
| 267 |
-
],
|
| 268 |
-
"tflite_output_shapes": [
|
| 269 |
-
[
|
| 270 |
-
2,
|
| 271 |
-
1,
|
| 272 |
-
640
|
| 273 |
-
],
|
| 274 |
-
[
|
| 275 |
-
1,
|
| 276 |
-
1,
|
| 277 |
-
640
|
| 278 |
-
],
|
| 279 |
-
[
|
| 280 |
-
2,
|
| 281 |
-
1,
|
| 282 |
-
640
|
| 283 |
-
]
|
| 284 |
-
],
|
| 285 |
-
"torch_output_shapes": [
|
| 286 |
-
[
|
| 287 |
-
1,
|
| 288 |
-
1,
|
| 289 |
-
640
|
| 290 |
-
],
|
| 291 |
-
[
|
| 292 |
-
2,
|
| 293 |
-
1,
|
| 294 |
-
640
|
| 295 |
-
],
|
| 296 |
-
[
|
| 297 |
-
2,
|
| 298 |
-
1,
|
| 299 |
-
640
|
| 300 |
-
]
|
| 301 |
-
]
|
| 302 |
-
}
|
| 303 |
},
|
| 304 |
{
|
| 305 |
"graph": "joint_step",
|
| 306 |
-
"
|
| 307 |
-
|
| 308 |
-
|
| 309 |
-
|
| 310 |
-
|
| 311 |
-
|
| 312 |
-
|
| 313 |
-
|
| 314 |
-
|
| 315 |
-
|
| 316 |
-
|
| 317 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 318 |
],
|
| 319 |
-
"
|
| 320 |
-
|
| 321 |
-
|
| 322 |
-
|
| 323 |
-
|
| 324 |
-
|
| 325 |
-
|
| 326 |
-
|
| 327 |
-
|
| 328 |
-
|
| 329 |
-
1,
|
| 330 |
-
8198
|
| 331 |
-
]
|
| 332 |
-
],
|
| 333 |
-
"torch_output_shapes": [
|
| 334 |
-
[
|
| 335 |
-
1,
|
| 336 |
-
1,
|
| 337 |
-
1,
|
| 338 |
-
8198
|
| 339 |
-
]
|
| 340 |
-
]
|
| 341 |
-
}
|
| 342 |
}
|
| 343 |
]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 344 |
}
|
| 345 |
}
|
|
|
|
| 1 |
{
|
| 2 |
"model": "nvidia/parakeet-tdt-0.6b-v3",
|
|
|
|
|
|
|
| 3 |
"vocab_size": 8192,
|
| 4 |
"blank_id": 8192,
|
| 5 |
"durations": [
|
|
|
|
| 30 |
"buckets": [
|
| 31 |
{
|
| 32 |
"n_mel_frames": 1500,
|
| 33 |
+
"n_encoder_frames": 188,
|
| 34 |
"input_shape": [
|
| 35 |
1,
|
| 36 |
128,
|
|
|
|
| 41 |
1024,
|
| 42 |
188
|
| 43 |
],
|
| 44 |
+
"artifact": "encoder_T1500.tflite",
|
| 45 |
+
"signature": "serving_default"
|
| 46 |
}
|
| 47 |
],
|
| 48 |
"multisig": false
|
|
|
|
| 65 |
"n_mels": 128,
|
| 66 |
"preemph": 0.97,
|
| 67 |
"log": true,
|
| 68 |
+
"frame_rate_hz_post_subsample": 12.5,
|
| 69 |
+
"normalize": "per_feature",
|
| 70 |
+
"window": "hann",
|
| 71 |
+
"window_periodic": false,
|
| 72 |
+
"log_zero_guard_type": "add",
|
| 73 |
+
"log_zero_guard_value": 5.960464477539063e-08,
|
| 74 |
+
"mag_power": 2.0,
|
| 75 |
+
"mel_norm": "slaney",
|
| 76 |
+
"inference_dither": 0.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
},
|
| 78 |
"tokenizer": {
|
| 79 |
"saved": true,
|
|
|
|
| 81 |
"vocab_size": 8192
|
| 82 |
},
|
| 83 |
"litert": {
|
| 84 |
+
"quant": "dynamic_INT8",
|
| 85 |
"results": [
|
| 86 |
{
|
| 87 |
"graph": "encoder",
|
| 88 |
+
"output_artifact": "encoder_T1500.tflite"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 89 |
},
|
| 90 |
{
|
| 91 |
"graph": "decoder_step",
|
| 92 |
+
"output_artifact": "decoder_step.tflite"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 93 |
},
|
| 94 |
{
|
| 95 |
"graph": "joint_step",
|
| 96 |
+
"output_artifact": "joint_step.tflite"
|
| 97 |
+
}
|
| 98 |
+
]
|
| 99 |
+
},
|
| 100 |
+
"experimental": true,
|
| 101 |
+
"android_validated": false,
|
| 102 |
+
"inference_profile": {
|
| 103 |
+
"commit_seconds": 4.96,
|
| 104 |
+
"left_context_seconds": 4.96,
|
| 105 |
+
"right_context_seconds": 2.0,
|
| 106 |
+
"carry_decoder_state": false,
|
| 107 |
+
"tail_padding_only": true
|
| 108 |
+
},
|
| 109 |
+
"actual_encoder_tensors": {
|
| 110 |
+
"inputs": [
|
| 111 |
+
{
|
| 112 |
+
"name": "serving_default_args_0",
|
| 113 |
+
"shape": [
|
| 114 |
+
1,
|
| 115 |
+
128,
|
| 116 |
+
1500
|
| 117 |
+
],
|
| 118 |
+
"type": 0,
|
| 119 |
+
"dtype": "float32"
|
| 120 |
+
},
|
| 121 |
+
{
|
| 122 |
+
"name": "serving_default_args_1",
|
| 123 |
+
"shape": [
|
| 124 |
+
1
|
| 125 |
+
],
|
| 126 |
+
"type": 2,
|
| 127 |
+
"dtype": "int32"
|
| 128 |
+
}
|
| 129 |
+
],
|
| 130 |
+
"outputs": [
|
| 131 |
+
{
|
| 132 |
+
"name": "serving_default_output_0_output",
|
| 133 |
+
"shape": [
|
| 134 |
+
1,
|
| 135 |
+
1024,
|
| 136 |
+
188
|
| 137 |
],
|
| 138 |
+
"type": 0,
|
| 139 |
+
"dtype": "float32"
|
| 140 |
+
},
|
| 141 |
+
{
|
| 142 |
+
"name": "serving_default_output_1_output",
|
| 143 |
+
"shape": [
|
| 144 |
+
1
|
| 145 |
+
],
|
| 146 |
+
"type": 2,
|
| 147 |
+
"dtype": "int32"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 148 |
}
|
| 149 |
]
|
| 150 |
+
},
|
| 151 |
+
"tested_runtime": {
|
| 152 |
+
"ai_edge_litert": "2.1.2",
|
| 153 |
+
"numpy": "2.4.4",
|
| 154 |
+
"sentencepiece": "0.2.1",
|
| 155 |
+
"encoder_threads": 4,
|
| 156 |
+
"decoder_joint_threads": 1
|
| 157 |
+
},
|
| 158 |
+
"source_program": {
|
| 159 |
+
"artifact": "encoder_dynamicT.pt2",
|
| 160 |
+
"sha256": "d16113f726f854f57a616387d1cbb88d4f2f8f80e84732a72f9bbcb788e38009"
|
| 161 |
+
},
|
| 162 |
+
"sha256": {
|
| 163 |
+
"encoder_T1500.tflite": "b26804686f98d68f7394baf3b13bdcea8e846ebf4ecf2903e3fc30fe3a3882a8",
|
| 164 |
+
"decoder_step.tflite": "eb0bf3559a0b4cbdc3ca05b7e8ff948ee5ef158ce424667b62a85f6c769a9ce1",
|
| 165 |
+
"joint_step.tflite": "0e28c22fc426df9900ef4a1bd15760ec757e44f0fd1818e0afb51c4fe79031be",
|
| 166 |
+
"tokenizer.model": "eacec2b0a77f336d4a2ca4a25a7047575d3c2b74de47e997f4c205126ed3135e"
|
| 167 |
+
},
|
| 168 |
+
"release_id": "20260914-int8-t1500-v1",
|
| 169 |
+
"legacy_release": {
|
| 170 |
+
"directory": "legacy/",
|
| 171 |
+
"revision": "c528ceb57bbc4e366993e50c79b22e0417a68484"
|
| 172 |
}
|
| 173 |
}
|
mapped_model_file.h
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#pragma once
|
| 2 |
+
// Linux/Android native integration example. Keep this owner alive longer than
|
| 3 |
+
// the model/interpreter. Only pass data() to an API documented to accept an
|
| 4 |
+
// external, immutable model buffer. This is NOT a Java CompiledModel option.
|
| 5 |
+
#include <algorithm>
|
| 6 |
+
#include <array>
|
| 7 |
+
#include <cerrno>
|
| 8 |
+
#include <cstring>
|
| 9 |
+
#include <stdexcept>
|
| 10 |
+
#include <string>
|
| 11 |
+
#include <fcntl.h>
|
| 12 |
+
#include <sys/mman.h>
|
| 13 |
+
#include <sys/stat.h>
|
| 14 |
+
#include <unistd.h>
|
| 15 |
+
|
| 16 |
+
class MappedModelFile {
|
| 17 |
+
public:
|
| 18 |
+
explicit MappedModelFile(const char* path) {
|
| 19 |
+
fd_ = open(path, O_RDONLY | O_CLOEXEC);
|
| 20 |
+
if (fd_ < 0) throw std::runtime_error("Cannot open model");
|
| 21 |
+
struct stat st {};
|
| 22 |
+
if (fstat(fd_, &st) != 0 || st.st_size <= 0) {
|
| 23 |
+
close(fd_);
|
| 24 |
+
throw std::runtime_error("Invalid model file");
|
| 25 |
+
}
|
| 26 |
+
size_ = static_cast<size_t>(st.st_size);
|
| 27 |
+
address_ = mmap(nullptr, size_, PROT_READ, MAP_PRIVATE, fd_, 0);
|
| 28 |
+
if (address_ == MAP_FAILED) {
|
| 29 |
+
close(fd_);
|
| 30 |
+
throw std::runtime_error("Cannot map model");
|
| 31 |
+
}
|
| 32 |
+
}
|
| 33 |
+
MappedModelFile(const MappedModelFile&) = delete;
|
| 34 |
+
MappedModelFile& operator=(const MappedModelFile&) = delete;
|
| 35 |
+
~MappedModelFile() { munmap(address_, size_); close(fd_); }
|
| 36 |
+
const void* data() const { return address_; }
|
| 37 |
+
size_t size() const { return size_; }
|
| 38 |
+
|
| 39 |
+
// Call once after encoder delegate preparation and representative warmup.
|
| 40 |
+
// The map remains valid; kernels can fault required source pages back in.
|
| 41 |
+
// Never use MADV_DONTNEED on an anonymous ByteBuffer or writable model buffer.
|
| 42 |
+
void DropVerifiedSourcePages() {
|
| 43 |
+
struct stat st {};
|
| 44 |
+
if (fstat(fd_, &st) != 0 || static_cast<size_t>(st.st_size) != size_)
|
| 45 |
+
throw std::runtime_error("Model file changed size");
|
| 46 |
+
std::array<unsigned char, 65536> bytes {};
|
| 47 |
+
for (size_t offset = 0; offset < size_;) {
|
| 48 |
+
size_t count = std::min(bytes.size(), size_ - offset);
|
| 49 |
+
ssize_t got;
|
| 50 |
+
do { got = pread(fd_, bytes.data(), count, static_cast<off_t>(offset)); }
|
| 51 |
+
while (got < 0 && errno == EINTR);
|
| 52 |
+
if (got <= 0 || std::memcmp(bytes.data(),
|
| 53 |
+
static_cast<const unsigned char*>(address_) + offset, got) != 0)
|
| 54 |
+
throw std::runtime_error("Model mapping differs from file");
|
| 55 |
+
offset += static_cast<size_t>(got);
|
| 56 |
+
}
|
| 57 |
+
if (madvise(address_, size_, MADV_DONTNEED) != 0)
|
| 58 |
+
throw std::runtime_error(std::string("madvise failed: ") + std::strerror(errno));
|
| 59 |
+
}
|
| 60 |
+
private:
|
| 61 |
+
int fd_;
|
| 62 |
+
void* address_;
|
| 63 |
+
size_t size_;
|
| 64 |
+
};
|
parakeet-tdt-0.6b-v3-litert.zip
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cd49232e6756b62ba14049bb31bab799838e30170b235a6993888e57662fdc95
|
| 3 |
+
size 665277590
|
publication.json
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"repo_id": "spybyscript/parakeet-tdt-litert",
|
| 3 |
+
"release_id": "20260914-int8-t1500-v1",
|
| 4 |
+
"source_model": "nvidia/parakeet-tdt-0.6b-v3",
|
| 5 |
+
"prior_revision": "c528ceb57bbc4e366993e50c79b22e0417a68484",
|
| 6 |
+
"legacy_directory": "legacy/",
|
| 7 |
+
"encoder_precision": "dynamic_INT8",
|
| 8 |
+
"decoder_joint_precision": "FP16 weights, FP32 interfaces",
|
| 9 |
+
"android_validated": false,
|
| 10 |
+
"private_evaluation_material_included": false
|
| 11 |
+
}
|
validation.json
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"candidate": {
|
| 3 |
+
"bucket": 1500,
|
| 4 |
+
"variant": "all8",
|
| 5 |
+
"peak_rss_gb": 1.600212992,
|
| 6 |
+
"english_wer": 0.042953020134228186,
|
| 7 |
+
"english_seconds": 11.591583443339914,
|
| 8 |
+
"longform_seconds": 12.17721762927249,
|
| 9 |
+
"comparisons": {
|
| 10 |
+
"english": {
|
| 11 |
+
"clips": 33,
|
| 12 |
+
"exact_token_sequences": 29,
|
| 13 |
+
"exact_normalized_words": 31,
|
| 14 |
+
"token_edits": 10,
|
| 15 |
+
"word_edits_vs_fp16": 2
|
| 16 |
+
},
|
| 17 |
+
"longform": {
|
| 18 |
+
"clips": 4,
|
| 19 |
+
"exact_token_sequences": 1,
|
| 20 |
+
"exact_normalized_words": 2,
|
| 21 |
+
"token_edits": 73,
|
| 22 |
+
"word_edits_vs_fp16": 31
|
| 23 |
+
}
|
| 24 |
+
}
|
| 25 |
+
},
|
| 26 |
+
"reference": "Original T1500 FP16 LiteRT",
|
| 27 |
+
"english_reference_words": 745,
|
| 28 |
+
"english_reference_type": "Synthetic source text, not independently human-verified transcripts",
|
| 29 |
+
"longform_reference_type": "No full human transcripts; no WER claimed",
|
| 30 |
+
"android_validated": false,
|
| 31 |
+
"linux_page_advice": [
|
| 32 |
+
{
|
| 33 |
+
"corpus": "english",
|
| 34 |
+
"clips": 33,
|
| 35 |
+
"exact_tokens_vs_unhinted_int8": true,
|
| 36 |
+
"peak_rss_gb": 1.4610432,
|
| 37 |
+
"resident_rss_gb": 0.954400768,
|
| 38 |
+
"resident_pss_gb": 0.947043328,
|
| 39 |
+
"seconds": 10.858530042227358
|
| 40 |
+
},
|
| 41 |
+
{
|
| 42 |
+
"corpus": "longform",
|
| 43 |
+
"clips": 4,
|
| 44 |
+
"exact_tokens_vs_unhinted_int8": true,
|
| 45 |
+
"peak_rss_gb": 1.462030336,
|
| 46 |
+
"resident_rss_gb": 0.954167296,
|
| 47 |
+
"resident_pss_gb": 0.946869248,
|
| 48 |
+
"seconds": 11.424920583609492
|
| 49 |
+
}
|
| 50 |
+
]
|
| 51 |
+
}
|