| --- |
| license: openrail |
| base_model: |
| - Supertone/supertonic |
| tags: |
| - tts |
| - qualcomm |
| - qnn |
| - quantized |
| - qcs6490 |
| pipeline_tag: text-to-speech |
| language: |
| - en |
| --- |
| |
| # Supertonic TTS Quantization for Qualcomm chipsets |
|
|
| A step-by-step guide to quantize the [Supertonic TTS](https://huggingface.co/Supertone/supertonic) model for Qualcomm QCS6490 using QAIRT/QNN. |
| Note: To achieve the optimal performance and accuracy consider generating ctx binaries or serialised binaries specific to the architecture (v68 - mine) |
|
|
| ### I would be using QCS6490 chip architecture for walking through the steps. |
|
|
| ## Sample Output |
|
|
| Audio generated on QCS6490 board using quantized models (10 diffusion steps, raw and noisy): |
|
|
| <audio controls src="https://huggingface.co/dev-ansh-r/qualcomm-Supertonic-TTS-QCS6490/resolve/main/final_output.wav"></audio> |
|
|
| ## Requirements |
|
|
| - QAIRT/QNN SDK **v2.37** |
| - Python 3.8+ |
| - Target device: **QCS6490** |
|
|
| ## Pipeline Architecture |
|
|
| ```text |
| text + style |
| │ |
| ┌───────────┴───────────┐ |
| │ │ |
| duration_predictor text_encoder |
| │ │ |
| duration (scalar) text_emb (1,128,256) |
| │ │ |
| latent_mask (1,1,256) │ |
| └───────────┬───────────┘ |
| │ |
| vector_estimator (10 diffusion steps) |
| │ |
| denoised_latent |
| │ |
| vocoder |
| │ |
| audio (44.1kHz) |
| ``` |
|
|
| The `duration_predictor` outputs a single scalar representing the total speech duration. This is post-processed into a `latent_mask` that tells the `vector_estimator` how many of the 256 fixed-size latent frames are active speech vs padding. |
|
|
| ## Workflow |
|
|
| ### 1. Input Preparation |
|
|
| Prepare calibration inputs for model quantization. |
|
|
| `Input_Preparation.ipynb` |
|
|
| ### 2. Step-by-Step Quantization |
|
|
| Convert ONNX models to QNN format with quantization for HTP backend. |
|
|
| `Supertonic_TTS_StepbyStep.ipynb` |
|
|
| ### 3. Correlation Verification |
|
|
| Verify quantized model outputs against reference using cosine similarity. |
|
|
| `Correlation_Verification.ipynb` |
|
|
| ## Project Structure |
|
|
| ```text |
| ├── Input_Preparation.ipynb # Prepare calibration inputs |
| ├── Supertonic_TTS_StepbyStep.ipynb # ONNX → QNN quantization guide |
| ├── Correlation_Verification.ipynb # Output verification |
| ├── assets/ # ONNX models (git submodule) |
| │ └── onnx/ |
| │ ├── text_encoder.onnx |
| │ ├── duration_predictor.onnx |
| │ ├── vector_estimator.onnx |
| │ └── vocoder.onnx |
| ├── QNN_Models/ # Quantized QNN models (.bin, .cpp) |
| ├── QNN_Model_lib/ # QNN runtime libraries (aarch64) |
| ├── qnn_calibration/ # Calibration data for verification |
| ├── inputs/ # Prepared input data |
| └── board_output/ # Inference outputs from board |
| ``` |
|
|
| ## Models |
|
|
| | Model | Description | |
| |--------------------|---------------------------------------------| |
| | text_encoder | Encodes text tokens with style embedding | |
| | duration_predictor | Predicts phoneme durations | |
| | vector_estimator | Diffusion-based latent generator (10 steps) | |
| | vocoder | Converts latent to audio waveform | |
| |
| ### ONNX Models (Source) |
| |
| Located in `assets/onnx/` (git submodule from Hugging Face): |
| |
| - `text_encoder.onnx` |
| - `duration_predictor.onnx` |
| - `vector_estimator.onnx` |
| - `vocoder.onnx` |
|
|
| ### QNN Models (Quantized) |
|
|
| Located in `QNN_Models/`: |
|
|
| - `text_encoder_htp.bin` / `.cpp` |
| - `vector_estimator_htp.bin` / `.cpp` |
| - `vocoder_htp.bin` / `.cpp` |
|
|
| ### Compiled Libraries (Ready for Deployment) |
|
|
| Located in `QNN_Model_lib/aarch64-oe-linux-gcc11.2/`: |
|
|
| - `libtext_encoder_htp.so` |
| - `libvector_estimator_htp.so` |
| - `libvocoder_htp.so` |
| - `libduration_predictor_htp.so` |
|
|
| These `.so` files are compiled from the `.cpp` sources and are ready to be deployed (via SCP) to the board for inference. |
|
|
| > **Note:** The `duration_predictor` is quantized and compiled but not used in the current calibration-based workflow since `latent_mask` is precomputed. For an end-to-end pipeline with arbitrary text input, the duration predictor must run first to dynamically generate the `latent_mask`. |
| |
| ## Getting Started |
| |
| 1. Clone with submodules: |
| |
| ```bash |
| git clone --recurse-submodules https://github.com/dev-ansh-r/Supertonic-TTS-QCS6490 |
| ``` |
| |
| 2. Follow the notebooks in order: |
| - `Input_Preparation.ipynb` |
| - `Supertonic_TTS_StepbyStep.ipynb` |
| - `Correlation_Verification.ipynb` |
|
|
| ## Note |
|
|
| > Inference script and sample application are not provided. Optimization work is ongoing and will be released soon. |
|
|
| ## License |
|
|
| This model inherits the licensing from [Supertone/supertonic-2](https://huggingface.co/Supertone/supertonic-2): |
|
|
| - **Model:** OpenRAIL-M License |
| - **Code:** MIT License |
|
|
| Copyright (c) 2026 Supertone Inc. (original model) |
