Parakeet TDT 0.6B v3 for Core AI: 8-bit palettized encoder (0.65 GB), Mac and iPhone validated
56ae5ae verified | # Reproducing the conversion | |
| **You need:** | |
| - Python 3.11, with PyTorch 2.11.0, Transformers 5.17.0, coreai-torch 0.4.2, coreai-core 1.0.0b2, coreai-opt 0.2.1, numpy and huggingface_hub; | |
| - macOS 27 with Xcode 27. | |
| **The source:** `nvidia/parakeet-tdt-0.6b-v3` at revision `541d1f99c6b0c3cd0b11a95167540bb8edefd82b`. You need these files: | |
| - `README.md` | |
| - `config.json` | |
| - `generation_config.json` | |
| - `model.safetensors`, whose SHA-256 is `3a2026366188c8c68598edbbff92f8d11590a08e0ae2e6775544e7b07d6a5e11` | |
| - `tokenizer.json` | |
| - `tokenizer_config.json` | |
| Put them in a folder with a `source-provenance.json`: | |
| ```json | |
| {"repo_id": "nvidia/parakeet-tdt-0.6b-v3", "revision": "541d1f99c6b0c3cd0b11a95167540bb8edefd82b", | |
| "files": ["README.md", "config.json", "generation_config.json", "model.safetensors", "tokenizer.json", "tokenizer_config.json"]} | |
| ``` | |
| **Export**, from this folder: | |
| ```sh | |
| PYTHONPATH=. python parakeet_export.py --source /path/to/source --output /path/to/pal8 \ | |
| --encoder-weights pal8 --small-weights fp16 | |
| ``` | |
| **What the export does:** | |
| - **Encoder and projection:** palettized with coreai-opt's 8-bit per-tensor k-means preset (`KMeansPalettizerConfig.presets.w8()`), then cast to FP16 with a 16-bit output. | |
| - **Predictor and joint:** wrapped so their weights are FP16 while their inputs and outputs stay FP32. | |
| - **Frontend:** stays FP32. | |
| - **References:** each component's export-time reference tensors go in `references/`. | |
| - **Manifest:** lists every file's size and SHA-256. | |
| Defaults, with no flags, reproduce the plain FP16 conversion. | |
| **Files here:** | |
| - `parakeet_export.py`: the export. | |
| - `parakeet_adapters.py`: the frontend, encoder, predictor and joint graphs. | |
| - `nemotron/graphs.py`: the explicit LSTM predictor and joint graphs, shared with the Nemotron conversion. | |