Qwen3-ASR / README.md
leokwu's picture
docs(asr): label the Chinese README as 简体中文 in the language switcher
7d6f9fc verified
|
Raw History Blame Contribute Delete
7.14 kB
---
license: apache-2.0
library_name: rkllm
pipeline_tag: automatic-speech-recognition
tags:
- rk182x
- rockchip
- rknpu
- llm
- qwen
- edge-ai
- asr
- speech-recognition
- multilingual
---
# Qwen3-ASR — [ASR] Streaming Speech Recognition
[English](README.md) | [简体中文](README_CN.md)
## 1. Overview
Streaming speech recognition: feed in an audio clip, get back the
**detected language plus the transcript**, refreshed once per second
(the RK182x streaming deployment of Qwen3-ASR-0.6B).
The upstream Qwen3-ASR family supports **52 languages and dialects**
(30 languages + 22 Chinese dialects) and handles speech, singing and
songs with background music. This repository ships the **0.6B** variant
running on the **streaming path**: audio is decoded in 1-second chunks
and the committed text grows monotonically.
This repository ships the RK182x deployment artifacts (RKNN graphs +
weights) **together with the wdnwebui worker manifest**
(`qwen3-asr.json`).
| Item | Value |
| --- | --- |
| Model ID | `wardner/Qwen3-ASR` |
| Display name | Qwen3-ASR |
| Model type | ASR (speech → language + text) |
| Compute target | RK182x |
| Host platform | RK3588 |
| Runtime | RKLLM + RKNN |
| Capabilities | `swcap_audio_asr` |
| Worker factory | `qwen3_asr` |
| Memory estimate | ~1600 MB |
| Init time | ~25 s |
| Cards required | 1 |
| Total weight size | 1.05 GB |
## 2. Files
| Path | Description | Size |
| --- | --- | --- |
| `encoder_online.1828.rknn` | RKNN graph — **streaming** audio encoder (RK182x compute card, 1.82 TOPS generation) | 3.0 MB |
| `encoder_online.weight` | Weights paired with the RKNN graph above | 357.5 MB |
| `llm.1828.rknn` | RKNN graph — Qwen3-ASR-0.6B LLM | 45.3 MB |
| `llm.weight` | Weights paired with the RKNN graph above | 369.3 MB |
| `llm.embed.bin` | Token embedding table, fp16 (vocab 151936 × dim 1024) | 296.8 MB |
| `llm.tokenizer.gguf` | GPT-2 BPE vocabulary + ByteLevel | 5.7 MB |
| `mel_128_filters.txt` | Mel filterbank (128 mel × 202 fp32) — **must stay next to the weights** | 653.3 KB |
| `qwen3-asr.json` | **worker manifest** — deliberately plain text | 1847 B |
> `qwen3-asr.json` is **deliberately not** in Git LFS. It is the
> manual of this distribution unit and must stay plain text so it can be
> diffed, reviewed and previewed on the Hub.
## 3. Download
```bash
hf download wardner/Qwen3-ASR --local-dir ./Qwen3-ASR
```
Or with git (requires Git LFS):
```bash
git clone https://huggingface.co/wardner/Qwen3-ASR
```
If `huggingface.co` is unreachable from your network (DNS interception /
blocked access), point the client at a mirror instead:
```bash
HF_ENDPOINT=https://hf-mirror.com hf download wardner/Qwen3-ASR \
--local-dir ./Qwen3-ASR
```
Note: with `git clone` the weights are Git LFS **pointers** until
`git lfs pull` completes. `hf download` fetches the real files directly, and
does not create a `.git` directory — neither is required for the model to
work; see §4.
## 4. Installation (wdnwebui)
Clone the repository into the model root so the folder holds **both** the
weights and the worker manifest:
```bash
git clone https://huggingface.co/wardner/Qwen3-ASR /userdata/models/Qwen3-ASR
```
Then open the web UI → **Compute** → the card appears under "local compute"
(the backend rescans the model root every ~2 s, **no restart needed**) →
click "enable".
The directory name does not have to match the model name: the SDK follows
"the folder the manifest was found in", located through the
`WDNAI_QWEN3_ASR_DIR` environment variable declared in the manifest.
The card itself is **Streaming Speech Recognition** (`qwen3-asr`) in the app
grid. It accepts an uploaded audio file, a video file with an audio track,
or live audio from the board microphone; the recognized text is pushed to
the browser over WebSocket as each 1-second chunk completes.
## 5. SHA-256 verification
```bash
cd ./Qwen3-ASR
sha256sum -c SHA256SUMS
```
All entries must report `OK` before deployment.
## 6. Caveats and limits
See the `notes` field of `qwen3-asr.json` — it records the
per-model specifics (streaming strategy, capability layout, side effects).
The details that bite most often:
- **This build is the streaming one.** The model folder ships
`encoder_online.*`, not `encoder.*`, so the spec runs the streaming path
(`use_online_encoder=True`, the SDK default). The encoder choice is a
**load-time** decision — which RKNN file goes on the NPU — so it lives in
the SDK spec, not in the manifest. Switching to the one-shot path needs
`encoder.1828.rknn` + `encoder.weight` and `use_online_encoder=False`.
- **Streaming strategy** (aligned with the C++ demo
`examples/Qwen3_ASR/cpp/online_main.cc`): 1 s per chunk
(`ONLINE_CHUNK_MS=1000`), at most 50 new tokens per chunk
(`ONLINE_MAX_NEW_TOKENS=50`), the first 4 rounds commit nothing
(`ONLINE_ROLLBACK_ROUNDS=4`), later rounds roll back the last 5 tokens
(`ONLINE_ROLLBACK_TOKENS=5`). The last chunk commits everything.
- **Only the committed prefix is monotonic.** The trailing "unconfirmed
tail" is rewritten by the next round — that is by design, not a bug.
- **The language head comes from the model.** It emits `language <lang>`
first (token `151704` is the separator); the SDK returns the language
separately to the front end. With little context the first round can
report the wrong language (e.g. `Chinese` for English speech) — the
final language is taken from the committed token stream.
- **Audio length.** The streaming demo caps at 60 s
(`ONLINE_MAX_AUDIO_SECONDS`); the exported LLM context is 24k tokens
(`MAX_CONTEXT_LEN = 1024*24`), which covers roughly 20 minutes of
audio — the offline demo caps at 20 minutes (`OFFLINE_MAX_AUDIO_SECONDS`).
- **`infer_timeout_s=120`.** One inference is N serial LLM rounds
(15 s of audio = 16 rounds) — the 60 s default budget is not enough.
- **`memory_estimate_mb=1600`**: encoder 375 MB + LLM 435 MB + embedding
table 297 MB.
- **Input format.** Any sample rate is resampled to 16 kHz mono. The mel
frame count must be truncated to `n_frame * hop` exactly like the C demo
does — without that truncation the model input width mismatches and the
**first round already diverges**.
- **Do not "improve" the resampler.** Linear interpolation is what matches
the C demo byte-for-byte; swapping in a Kaiser-sinc filter drops words.
- **Decoding must skip special tokens**, otherwise `<|im_end|>` leaks into
the tail of the transcript.
- **`mel_128_filters.txt` must be in the same directory** as the weights
(128 mel × 202 fp32; the SDK truncates to the first 128 × 201 =
`N_FFT/2+1` values).
- **The tokenizer comes from another repo**: `llm.tokenizer.gguf` is
exported from `Qwen/Qwen3-0.6B`, not from the Qwen3-ASR repository.
- Model not deployed → load raises `FileNotFoundError`, which the worker's
load flow turns into an `errored` state.
## 7. License
The upstream model is released under the Apache License 2.0. The files here
are RKNN-converted derivatives, not unmodified upstream weights. See
[LICENSE](LICENSE) and [NOTICE](NOTICE).