--- license: apache-2.0 library_name: rkllm pipeline_tag: automatic-speech-recognition tags: - rk182x - rockchip - rknpu - llm - qwen - edge-ai - asr - speech-recognition - multilingual --- # Qwen3-ASR — [ASR] Streaming Speech Recognition [English](README.md) | [简体中文](README_CN.md) ## 1. Overview Streaming speech recognition: feed in an audio clip, get back the **detected language plus the transcript**, refreshed once per second (the RK182x streaming deployment of Qwen3-ASR-0.6B). The upstream Qwen3-ASR family supports **52 languages and dialects** (30 languages + 22 Chinese dialects) and handles speech, singing and songs with background music. This repository ships the **0.6B** variant running on the **streaming path**: audio is decoded in 1-second chunks and the committed text grows monotonically. This repository ships the RK182x deployment artifacts (RKNN graphs + weights) **together with the wdnwebui worker manifest** (`qwen3-asr.json`). | Item | Value | | --- | --- | | Model ID | `wardner/Qwen3-ASR` | | Display name | Qwen3-ASR | | Model type | ASR (speech → language + text) | | Compute target | RK182x | | Host platform | RK3588 | | Runtime | RKLLM + RKNN | | Capabilities | `swcap_audio_asr` | | Worker factory | `qwen3_asr` | | Memory estimate | ~1600 MB | | Init time | ~25 s | | Cards required | 1 | | Total weight size | 1.05 GB | ## 2. Files | Path | Description | Size | | --- | --- | --- | | `encoder_online.1828.rknn` | RKNN graph — **streaming** audio encoder (RK182x compute card, 1.82 TOPS generation) | 3.0 MB | | `encoder_online.weight` | Weights paired with the RKNN graph above | 357.5 MB | | `llm.1828.rknn` | RKNN graph — Qwen3-ASR-0.6B LLM | 45.3 MB | | `llm.weight` | Weights paired with the RKNN graph above | 369.3 MB | | `llm.embed.bin` | Token embedding table, fp16 (vocab 151936 × dim 1024) | 296.8 MB | | `llm.tokenizer.gguf` | GPT-2 BPE vocabulary + ByteLevel | 5.7 MB | | `mel_128_filters.txt` | Mel filterbank (128 mel × 202 fp32) — **must stay next to the weights** | 653.3 KB | | `qwen3-asr.json` | **worker manifest** — deliberately plain text | 1847 B | > `qwen3-asr.json` is **deliberately not** in Git LFS. It is the > manual of this distribution unit and must stay plain text so it can be > diffed, reviewed and previewed on the Hub. ## 3. Download ```bash hf download wardner/Qwen3-ASR --local-dir ./Qwen3-ASR ``` Or with git (requires Git LFS): ```bash git clone https://huggingface.co/wardner/Qwen3-ASR ``` If `huggingface.co` is unreachable from your network (DNS interception / blocked access), point the client at a mirror instead: ```bash HF_ENDPOINT=https://hf-mirror.com hf download wardner/Qwen3-ASR \ --local-dir ./Qwen3-ASR ``` Note: with `git clone` the weights are Git LFS **pointers** until `git lfs pull` completes. `hf download` fetches the real files directly, and does not create a `.git` directory — neither is required for the model to work; see §4. ## 4. Installation (wdnwebui) Clone the repository into the model root so the folder holds **both** the weights and the worker manifest: ```bash git clone https://huggingface.co/wardner/Qwen3-ASR /userdata/models/Qwen3-ASR ``` Then open the web UI → **Compute** → the card appears under "local compute" (the backend rescans the model root every ~2 s, **no restart needed**) → click "enable". The directory name does not have to match the model name: the SDK follows "the folder the manifest was found in", located through the `WDNAI_QWEN3_ASR_DIR` environment variable declared in the manifest. The card itself is **Streaming Speech Recognition** (`qwen3-asr`) in the app grid. It accepts an uploaded audio file, a video file with an audio track, or live audio from the board microphone; the recognized text is pushed to the browser over WebSocket as each 1-second chunk completes. ## 5. SHA-256 verification ```bash cd ./Qwen3-ASR sha256sum -c SHA256SUMS ``` All entries must report `OK` before deployment. ## 6. Caveats and limits See the `notes` field of `qwen3-asr.json` — it records the per-model specifics (streaming strategy, capability layout, side effects). The details that bite most often: - **This build is the streaming one.** The model folder ships `encoder_online.*`, not `encoder.*`, so the spec runs the streaming path (`use_online_encoder=True`, the SDK default). The encoder choice is a **load-time** decision — which RKNN file goes on the NPU — so it lives in the SDK spec, not in the manifest. Switching to the one-shot path needs `encoder.1828.rknn` + `encoder.weight` and `use_online_encoder=False`. - **Streaming strategy** (aligned with the C++ demo `examples/Qwen3_ASR/cpp/online_main.cc`): 1 s per chunk (`ONLINE_CHUNK_MS=1000`), at most 50 new tokens per chunk (`ONLINE_MAX_NEW_TOKENS=50`), the first 4 rounds commit nothing (`ONLINE_ROLLBACK_ROUNDS=4`), later rounds roll back the last 5 tokens (`ONLINE_ROLLBACK_TOKENS=5`). The last chunk commits everything. - **Only the committed prefix is monotonic.** The trailing "unconfirmed tail" is rewritten by the next round — that is by design, not a bug. - **The language head comes from the model.** It emits `language ` first (token `151704` is the separator); the SDK returns the language separately to the front end. With little context the first round can report the wrong language (e.g. `Chinese` for English speech) — the final language is taken from the committed token stream. - **Audio length.** The streaming demo caps at 60 s (`ONLINE_MAX_AUDIO_SECONDS`); the exported LLM context is 24k tokens (`MAX_CONTEXT_LEN = 1024*24`), which covers roughly 20 minutes of audio — the offline demo caps at 20 minutes (`OFFLINE_MAX_AUDIO_SECONDS`). - **`infer_timeout_s=120`.** One inference is N serial LLM rounds (15 s of audio = 16 rounds) — the 60 s default budget is not enough. - **`memory_estimate_mb=1600`**: encoder 375 MB + LLM 435 MB + embedding table 297 MB. - **Input format.** Any sample rate is resampled to 16 kHz mono. The mel frame count must be truncated to `n_frame * hop` exactly like the C demo does — without that truncation the model input width mismatches and the **first round already diverges**. - **Do not "improve" the resampler.** Linear interpolation is what matches the C demo byte-for-byte; swapping in a Kaiser-sinc filter drops words. - **Decoding must skip special tokens**, otherwise `<|im_end|>` leaks into the tail of the transcript. - **`mel_128_filters.txt` must be in the same directory** as the weights (128 mel × 202 fp32; the SDK truncates to the first 128 × 201 = `N_FFT/2+1` values). - **The tokenizer comes from another repo**: `llm.tokenizer.gguf` is exported from `Qwen/Qwen3-0.6B`, not from the Qwen3-ASR repository. - Model not deployed → load raises `FileNotFoundError`, which the worker's load flow turns into an `errored` state. ## 7. License The upstream model is released under the Apache License 2.0. The files here are RKNN-converted derivatives, not unmodified upstream weights. See [LICENSE](LICENSE) and [NOTICE](NOTICE).