Automatic Speech Recognition
RKLLM
GGUF
ASR
rk182x
rockchip
rknpu
llm
qwen
edge-ai
asr
speech-recognition
multilingual
conversational
Instructions to use wardner/Qwen3-ASR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- RKLLM
How to use wardner/Qwen3-ASR with RKLLM:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use wardner/Qwen3-ASR with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf wardner/Qwen3-ASR # Run inference directly in the terminal: llama cli -hf wardner/Qwen3-ASR
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf wardner/Qwen3-ASR # Run inference directly in the terminal: llama cli -hf wardner/Qwen3-ASR
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf wardner/Qwen3-ASR # Run inference directly in the terminal: ./build/bin/llama cli -hf wardner/Qwen3-ASR
Use Docker
docker model run hf.co/wardner/Qwen3-ASR
- Llama new
- LM Studio
- Jan
- Ollama
How to use wardner/Qwen3-ASR with Ollama:
ollama run hf.co/wardner/Qwen3-ASR
- Unsloth Desktop
- Pi
How to use wardner/Qwen3-ASR with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf wardner/Qwen3-ASR
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "wardner/Qwen3-ASR" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use wardner/Qwen3-ASR with Docker Model Runner:
docker model run hf.co/wardner/Qwen3-ASR
- Lemonade
How to use wardner/Qwen3-ASR with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull wardner/Qwen3-ASR
Run and chat with the model
lemonade run user.Qwen3-ASR-{{QUANT_TAG}}List all available models
lemonade list
- Hermes Agent
How to use wardner/Qwen3-ASR with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf wardner/Qwen3-ASR
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default wardner/Qwen3-ASR
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use wardner/Qwen3-ASR with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf wardner/Qwen3-ASR
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "wardner/Qwen3-ASR" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
|
Download README.md from wardner/Qwen3-ASR: direct link, hf CLI and curl.
- Browser
- Download file 7.14 kB
-
https://huggingface.co/wardner/Qwen3-ASR/resolve/main/README.md
- Command line
-
hf download hf://wardner/Qwen3-ASR/README.md
-
curl -L -o README.md https://huggingface.co/wardner/Qwen3-ASR/resolve/main/README.md
7.14 kB
| license: apache-2.0 | |
| library_name: rkllm | |
| pipeline_tag: automatic-speech-recognition | |
| tags: | |
| - rk182x | |
| - rockchip | |
| - rknpu | |
| - llm | |
| - qwen | |
| - edge-ai | |
| - asr | |
| - speech-recognition | |
| - multilingual | |
| # Qwen3-ASR — [ASR] Streaming Speech Recognition | |
| [English](README.md) | [ç®€ä½“ä¸æ–‡](README_CN.md) | |
| ## 1. Overview | |
| Streaming speech recognition: feed in an audio clip, get back the | |
| **detected language plus the transcript**, refreshed once per second | |
| (the RK182x streaming deployment of Qwen3-ASR-0.6B). | |
| The upstream Qwen3-ASR family supports **52 languages and dialects** | |
| (30 languages + 22 Chinese dialects) and handles speech, singing and | |
| songs with background music. This repository ships the **0.6B** variant | |
| running on the **streaming path**: audio is decoded in 1-second chunks | |
| and the committed text grows monotonically. | |
| This repository ships the RK182x deployment artifacts (RKNN graphs + | |
| weights) **together with the wdnwebui worker manifest** | |
| (`qwen3-asr.json`). | |
| | Item | Value | | |
| | --- | --- | | |
| | Model ID | `wardner/Qwen3-ASR` | | |
| | Display name | Qwen3-ASR | | |
| | Model type | ASR (speech → language + text) | | |
| | Compute target | RK182x | | |
| | Host platform | RK3588 | | |
| | Runtime | RKLLM + RKNN | | |
| | Capabilities | `swcap_audio_asr` | | |
| | Worker factory | `qwen3_asr` | | |
| | Memory estimate | ~1600 MB | | |
| | Init time | ~25 s | | |
| | Cards required | 1 | | |
| | Total weight size | 1.05 GB | | |
| ## 2. Files | |
| | Path | Description | Size | | |
| | --- | --- | --- | | |
| | `encoder_online.1828.rknn` | RKNN graph — **streaming** audio encoder (RK182x compute card, 1.82 TOPS generation) | 3.0 MB | | |
| | `encoder_online.weight` | Weights paired with the RKNN graph above | 357.5 MB | | |
| | `llm.1828.rknn` | RKNN graph — Qwen3-ASR-0.6B LLM | 45.3 MB | | |
| | `llm.weight` | Weights paired with the RKNN graph above | 369.3 MB | | |
| | `llm.embed.bin` | Token embedding table, fp16 (vocab 151936 × dim 1024) | 296.8 MB | | |
| | `llm.tokenizer.gguf` | GPT-2 BPE vocabulary + ByteLevel | 5.7 MB | | |
| | `mel_128_filters.txt` | Mel filterbank (128 mel × 202 fp32) — **must stay next to the weights** | 653.3 KB | | |
| | `qwen3-asr.json` | **worker manifest** — deliberately plain text | 1847 B | | |
| > `qwen3-asr.json` is **deliberately not** in Git LFS. It is the | |
| > manual of this distribution unit and must stay plain text so it can be | |
| > diffed, reviewed and previewed on the Hub. | |
| ## 3. Download | |
| ```bash | |
| hf download wardner/Qwen3-ASR --local-dir ./Qwen3-ASR | |
| ``` | |
| Or with git (requires Git LFS): | |
| ```bash | |
| git clone https://huggingface.co/wardner/Qwen3-ASR | |
| ``` | |
| If `huggingface.co` is unreachable from your network (DNS interception / | |
| blocked access), point the client at a mirror instead: | |
| ```bash | |
| HF_ENDPOINT=https://hf-mirror.com hf download wardner/Qwen3-ASR \ | |
| --local-dir ./Qwen3-ASR | |
| ``` | |
| Note: with `git clone` the weights are Git LFS **pointers** until | |
| `git lfs pull` completes. `hf download` fetches the real files directly, and | |
| does not create a `.git` directory — neither is required for the model to | |
| work; see §4. | |
| ## 4. Installation (wdnwebui) | |
| Clone the repository into the model root so the folder holds **both** the | |
| weights and the worker manifest: | |
| ```bash | |
| git clone https://huggingface.co/wardner/Qwen3-ASR /userdata/models/Qwen3-ASR | |
| ``` | |
| Then open the web UI → **Compute** → the card appears under "local compute" | |
| (the backend rescans the model root every ~2 s, **no restart needed**) → | |
| click "enable". | |
| The directory name does not have to match the model name: the SDK follows | |
| "the folder the manifest was found in", located through the | |
| `WDNAI_QWEN3_ASR_DIR` environment variable declared in the manifest. | |
| The card itself is **Streaming Speech Recognition** (`qwen3-asr`) in the app | |
| grid. It accepts an uploaded audio file, a video file with an audio track, | |
| or live audio from the board microphone; the recognized text is pushed to | |
| the browser over WebSocket as each 1-second chunk completes. | |
| ## 5. SHA-256 verification | |
| ```bash | |
| cd ./Qwen3-ASR | |
| sha256sum -c SHA256SUMS | |
| ``` | |
| All entries must report `OK` before deployment. | |
| ## 6. Caveats and limits | |
| See the `notes` field of `qwen3-asr.json` — it records the | |
| per-model specifics (streaming strategy, capability layout, side effects). | |
| The details that bite most often: | |
| - **This build is the streaming one.** The model folder ships | |
| `encoder_online.*`, not `encoder.*`, so the spec runs the streaming path | |
| (`use_online_encoder=True`, the SDK default). The encoder choice is a | |
| **load-time** decision — which RKNN file goes on the NPU — so it lives in | |
| the SDK spec, not in the manifest. Switching to the one-shot path needs | |
| `encoder.1828.rknn` + `encoder.weight` and `use_online_encoder=False`. | |
| - **Streaming strategy** (aligned with the C++ demo | |
| `examples/Qwen3_ASR/cpp/online_main.cc`): 1 s per chunk | |
| (`ONLINE_CHUNK_MS=1000`), at most 50 new tokens per chunk | |
| (`ONLINE_MAX_NEW_TOKENS=50`), the first 4 rounds commit nothing | |
| (`ONLINE_ROLLBACK_ROUNDS=4`), later rounds roll back the last 5 tokens | |
| (`ONLINE_ROLLBACK_TOKENS=5`). The last chunk commits everything. | |
| - **Only the committed prefix is monotonic.** The trailing "unconfirmed | |
| tail" is rewritten by the next round — that is by design, not a bug. | |
| - **The language head comes from the model.** It emits `language <lang>` | |
| first (token `151704` is the separator); the SDK returns the language | |
| separately to the front end. With little context the first round can | |
| report the wrong language (e.g. `Chinese` for English speech) — the | |
| final language is taken from the committed token stream. | |
| - **Audio length.** The streaming demo caps at 60 s | |
| (`ONLINE_MAX_AUDIO_SECONDS`); the exported LLM context is 24k tokens | |
| (`MAX_CONTEXT_LEN = 1024*24`), which covers roughly 20 minutes of | |
| audio — the offline demo caps at 20 minutes (`OFFLINE_MAX_AUDIO_SECONDS`). | |
| - **`infer_timeout_s=120`.** One inference is N serial LLM rounds | |
| (15 s of audio = 16 rounds) — the 60 s default budget is not enough. | |
| - **`memory_estimate_mb=1600`**: encoder 375 MB + LLM 435 MB + embedding | |
| table 297 MB. | |
| - **Input format.** Any sample rate is resampled to 16 kHz mono. The mel | |
| frame count must be truncated to `n_frame * hop` exactly like the C demo | |
| does — without that truncation the model input width mismatches and the | |
| **first round already diverges**. | |
| - **Do not "improve" the resampler.** Linear interpolation is what matches | |
| the C demo byte-for-byte; swapping in a Kaiser-sinc filter drops words. | |
| - **Decoding must skip special tokens**, otherwise `<|im_end|>` leaks into | |
| the tail of the transcript. | |
| - **`mel_128_filters.txt` must be in the same directory** as the weights | |
| (128 mel × 202 fp32; the SDK truncates to the first 128 × 201 = | |
| `N_FFT/2+1` values). | |
| - **The tokenizer comes from another repo**: `llm.tokenizer.gguf` is | |
| exported from `Qwen/Qwen3-0.6B`, not from the Qwen3-ASR repository. | |
| - Model not deployed → load raises `FileNotFoundError`, which the worker's | |
| load flow turns into an `errored` state. | |
| ## 7. License | |
| The upstream model is released under the Apache License 2.0. The files here | |
| are RKNN-converted derivatives, not unmodified upstream weights. See | |
| [LICENSE](LICENSE) and [NOTICE](NOTICE). | |