| --- |
| license: apache-2.0 |
| language: |
| - de |
| - multilingual |
| base_model: openai/whisper-tiny |
| pipeline_tag: automatic-speech-recognition |
| tags: |
| - onnx |
| - whisper |
| - android |
| - mobile |
| --- |
| |
| # Whisper tiny for mobile (ONNX) |
|
|
| An int8 ONNX copy of OpenAI's multilingual Whisper tiny model, for on-device speech recognition with ONNX |
| Runtime. The Thì Thầm app uses it for German, with the decoder forced to `<|de|>`. |
|
|
| It is made from [Xenova/whisper-tiny](https://huggingface.co/Xenova/whisper-tiny) at commit `5332fcc35e32a33b86612b9a57a89be7906102b1`. |
| The only change is that the attention maps are no longer graph outputs. ONNX Runtime keeps declared graph |
| outputs in memory, so Xenova's files need much more memory for each transcription than the model needs. |
| The weights, the nodes and the results are unchanged. |
|
|
| ## Peak memory |
|
|
| Net peak memory of one transcription (both sessions loaded, the encoder, then 20 decoder steps), with ONNX |
| Runtime 1.31.1, 4 intra-op threads and graph optimization ENABLE_ALL, measured in fresh processes: |
| |
| | Model | Xenova file | This copy | |
| |---|---|---| |
| | tiny | 513 MB | 345 MB | |
| |
| ## Check against the original |
| |
| On 2 FLEURS German clips, 30 greedy decoder steps from the German prompt give identical tokens, and the |
| encoder output is identical, for Xenova's file and for this copy. |
| |
| ## Inputs and outputs |
| |
| - Encoder `onnx/encoder_model_quantized.onnx`: input `input_features`, float32 [1, 80, 3000] (log-mel of 30 s of audio); output `last_hidden_state`. |
| - Decoder `onnx/decoder_model_merged_quantized.onnx`: inputs `input_ids` (int64), `encoder_hidden_states`, `use_cache_branch` (bool) and `past_key_values.*`; outputs `logits` and `present.*`. |
| - German prompt: `[50258, 50261, 50359, 50363]` (`<|startoftranscript|>`, `<|de|>`, `<|transcribe|>`, `<|notimestamps|>`). Stop at `50257` (`<|endoftext|>`). Another language token works the same way. |
|
|
| ## Credits and license |
|
|
| - Whisper: OpenAI, [openai/whisper-tiny](https://huggingface.co/openai/whisper-tiny) (Apache 2.0 on Hugging Face; |
| the code and weights in [openai/whisper](https://github.com/openai/whisper) are under the MIT License). |
| - ONNX export: Xenova, [Xenova/whisper-tiny](https://huggingface.co/Xenova/whisper-tiny), made for transformers.js. |
| - This copy: made with `make_mobile.py` (included here for provenance; it runs from `tools/whisper_de` of the app's |
| repository). License: Apache 2.0, see `LICENSE`, as the files it is made from. |
| - Used by the Thì Thầm Android app for German speech recognition. |
|
|
