Qwen3-ASR / README.md
leokwu's picture
docs(asr): label the Chinese README as 简体中文 in the language switcher
7d6f9fc verified
|
Raw History Blame Contribute Delete
7.14 kB
metadata
license: apache-2.0
library_name: rkllm
pipeline_tag: automatic-speech-recognition
tags:
  - rk182x
  - rockchip
  - rknpu
  - llm
  - qwen
  - edge-ai
  - asr
  - speech-recognition
  - multilingual

Qwen3-ASR — [ASR] Streaming Speech Recognition

English | 简体中文

1. Overview

Streaming speech recognition: feed in an audio clip, get back the detected language plus the transcript, refreshed once per second (the RK182x streaming deployment of Qwen3-ASR-0.6B).

The upstream Qwen3-ASR family supports 52 languages and dialects (30 languages + 22 Chinese dialects) and handles speech, singing and songs with background music. This repository ships the 0.6B variant running on the streaming path: audio is decoded in 1-second chunks and the committed text grows monotonically.

This repository ships the RK182x deployment artifacts (RKNN graphs + weights) together with the wdnwebui worker manifest (qwen3-asr.json).

Item Value
Model ID wardner/Qwen3-ASR
Display name Qwen3-ASR
Model type ASR (speech → language + text)
Compute target RK182x
Host platform RK3588
Runtime RKLLM + RKNN
Capabilities swcap_audio_asr
Worker factory qwen3_asr
Memory estimate ~1600 MB
Init time ~25 s
Cards required 1
Total weight size 1.05 GB

2. Files

Path Description Size
encoder_online.1828.rknn RKNN graph — streaming audio encoder (RK182x compute card, 1.82 TOPS generation) 3.0 MB
encoder_online.weight Weights paired with the RKNN graph above 357.5 MB
llm.1828.rknn RKNN graph — Qwen3-ASR-0.6B LLM 45.3 MB
llm.weight Weights paired with the RKNN graph above 369.3 MB
llm.embed.bin Token embedding table, fp16 (vocab 151936 × dim 1024) 296.8 MB
llm.tokenizer.gguf GPT-2 BPE vocabulary + ByteLevel 5.7 MB
mel_128_filters.txt Mel filterbank (128 mel × 202 fp32) — must stay next to the weights 653.3 KB
qwen3-asr.json worker manifest — deliberately plain text 1847 B

qwen3-asr.json is deliberately not in Git LFS. It is the manual of this distribution unit and must stay plain text so it can be diffed, reviewed and previewed on the Hub.

3. Download

hf download wardner/Qwen3-ASR --local-dir ./Qwen3-ASR

Or with git (requires Git LFS):

git clone https://huggingface.co/wardner/Qwen3-ASR

If huggingface.co is unreachable from your network (DNS interception / blocked access), point the client at a mirror instead:

HF_ENDPOINT=https://hf-mirror.com hf download wardner/Qwen3-ASR \
  --local-dir ./Qwen3-ASR

Note: with git clone the weights are Git LFS pointers until git lfs pull completes. hf download fetches the real files directly, and does not create a .git directory — neither is required for the model to work; see §4.

4. Installation (wdnwebui)

Clone the repository into the model root so the folder holds both the weights and the worker manifest:

git clone https://huggingface.co/wardner/Qwen3-ASR /userdata/models/Qwen3-ASR

Then open the web UI → Compute → the card appears under "local compute" (the backend rescans the model root every ~2 s, no restart needed) → click "enable".

The directory name does not have to match the model name: the SDK follows "the folder the manifest was found in", located through the WDNAI_QWEN3_ASR_DIR environment variable declared in the manifest.

The card itself is Streaming Speech Recognition (qwen3-asr) in the app grid. It accepts an uploaded audio file, a video file with an audio track, or live audio from the board microphone; the recognized text is pushed to the browser over WebSocket as each 1-second chunk completes.

5. SHA-256 verification

cd ./Qwen3-ASR
sha256sum -c SHA256SUMS

All entries must report OK before deployment.

6. Caveats and limits

See the notes field of qwen3-asr.json — it records the per-model specifics (streaming strategy, capability layout, side effects). The details that bite most often:

  • This build is the streaming one. The model folder ships encoder_online.*, not encoder.*, so the spec runs the streaming path (use_online_encoder=True, the SDK default). The encoder choice is a load-time decision — which RKNN file goes on the NPU — so it lives in the SDK spec, not in the manifest. Switching to the one-shot path needs encoder.1828.rknn + encoder.weight and use_online_encoder=False.
  • Streaming strategy (aligned with the C++ demo examples/Qwen3_ASR/cpp/online_main.cc): 1 s per chunk (ONLINE_CHUNK_MS=1000), at most 50 new tokens per chunk (ONLINE_MAX_NEW_TOKENS=50), the first 4 rounds commit nothing (ONLINE_ROLLBACK_ROUNDS=4), later rounds roll back the last 5 tokens (ONLINE_ROLLBACK_TOKENS=5). The last chunk commits everything.
  • Only the committed prefix is monotonic. The trailing "unconfirmed tail" is rewritten by the next round — that is by design, not a bug.
  • The language head comes from the model. It emits language <lang> first (token 151704 is the separator); the SDK returns the language separately to the front end. With little context the first round can report the wrong language (e.g. Chinese for English speech) — the final language is taken from the committed token stream.
  • Audio length. The streaming demo caps at 60 s (ONLINE_MAX_AUDIO_SECONDS); the exported LLM context is 24k tokens (MAX_CONTEXT_LEN = 1024*24), which covers roughly 20 minutes of audio — the offline demo caps at 20 minutes (OFFLINE_MAX_AUDIO_SECONDS).
  • infer_timeout_s=120. One inference is N serial LLM rounds (15 s of audio = 16 rounds) — the 60 s default budget is not enough.
  • memory_estimate_mb=1600: encoder 375 MB + LLM 435 MB + embedding table 297 MB.
  • Input format. Any sample rate is resampled to 16 kHz mono. The mel frame count must be truncated to n_frame * hop exactly like the C demo does — without that truncation the model input width mismatches and the first round already diverges.
  • Do not "improve" the resampler. Linear interpolation is what matches the C demo byte-for-byte; swapping in a Kaiser-sinc filter drops words.
  • Decoding must skip special tokens, otherwise <|im_end|> leaks into the tail of the transcript.
  • mel_128_filters.txt must be in the same directory as the weights (128 mel × 202 fp32; the SDK truncates to the first 128 × 201 = N_FFT/2+1 values).
  • The tokenizer comes from another repo: llm.tokenizer.gguf is exported from Qwen/Qwen3-0.6B, not from the Qwen3-ASR repository.
  • Model not deployed → load raises FileNotFoundError, which the worker's load flow turns into an errored state.

7. License

The upstream model is released under the Apache License 2.0. The files here are RKNN-converted derivatives, not unmodified upstream weights. See LICENSE and NOTICE.