---
license: apache-2.0
language:
- de
- multilingual
base_model: openai/whisper-tiny
pipeline_tag: automatic-speech-recognition
tags:
- onnx
- whisper
- android
- mobile
---

# Whisper tiny for mobile (ONNX)

An int8 ONNX copy of OpenAI's multilingual Whisper tiny model, for on-device speech recognition with ONNX
Runtime. The Thì Thầm app uses it for German, with the decoder forced to `<|de|>`.

It is made from [Xenova/whisper-tiny](https://huggingface.co/Xenova/whisper-tiny) at commit `5332fcc35e32a33b86612b9a57a89be7906102b1`.
The only change is that the attention maps are no longer graph outputs. ONNX Runtime keeps declared graph
outputs in memory, so Xenova's files need much more memory for each transcription than the model needs.
The weights, the nodes and the results are unchanged.

## Peak memory

Net peak memory of one transcription (both sessions loaded, the encoder, then 20 decoder steps), with ONNX
Runtime 1.31.1, 4 intra-op threads and graph optimization ENABLE_ALL, measured in fresh processes:

| Model | Xenova file | This copy |
|---|---|---|
| tiny | 513 MB | 345 MB |

## Check against the original

On 2 FLEURS German clips, 30 greedy decoder steps from the German prompt give identical tokens, and the
encoder output is identical, for Xenova's file and for this copy.

## Inputs and outputs

- Encoder `onnx/encoder_model_quantized.onnx`: input `input_features`, float32 [1, 80, 3000] (log-mel of 30 s of audio); output `last_hidden_state`.
- Decoder `onnx/decoder_model_merged_quantized.onnx`: inputs `input_ids` (int64), `encoder_hidden_states`, `use_cache_branch` (bool) and `past_key_values.*`; outputs `logits` and `present.*`.
- German prompt: `[50258, 50261, 50359, 50363]` (`<|startoftranscript|>`, `<|de|>`, `<|transcribe|>`, `<|notimestamps|>`). Stop at `50257` (`<|endoftext|>`). Another language token works the same way.

## Credits and license

- Whisper: OpenAI, [openai/whisper-tiny](https://huggingface.co/openai/whisper-tiny) (Apache 2.0 on Hugging Face;
  the code and weights in [openai/whisper](https://github.com/openai/whisper) are under the MIT License).
- ONNX export: Xenova, [Xenova/whisper-tiny](https://huggingface.co/Xenova/whisper-tiny), made for transformers.js.
- This copy: made with `make_mobile.py` (included here for provenance; it runs from `tools/whisper_de` of the app's
  repository). License: Apache 2.0, see `LICENSE`, as the files it is made from.
- Used by the Thì Thầm Android app for German speech recognition.