File size: 4,108 Bytes
afb7114 cdc5ab8 afb7114 cdc5ab8 afb7114 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 | ---
license: apache-2.0
base_model: Qwen/Qwen3-ASR-0.6B-hf
library_name: onnxruntime
pipeline_tag: automatic-speech-recognition
tags:
- qwen3-asr
- tensorrt
- onnx
- bf16
- fp16
---
<p align="center">
<a href="https://github.com/lost0427/Qwen3-ASR-TensorRT"><img src="https://img.shields.io/badge/GitHub-source-181717?logo=github" alt="GitHub source"></a>
<a href="https://huggingface.co/Retrieve0/Qwen3-ASR-0.6B-TensorRT-Encoder"><img src="https://img.shields.io/badge/Hugging_Face-model-yellow?logo=huggingface" alt="Hugging Face model"></a>
</p>
# Qwen3-ASR 0.6B TensorRT Encoder
## English
ONNX exports of the audio encoder from [Qwen/Qwen3-ASR-0.6B-hf](https://huggingface.co/Qwen/Qwen3-ASR-0.6B-hf), prepared for TensorRT inference. Use an encoder export with a compatible Qwen3-ASR text decoder, such as the [EXL3 decoder](https://huggingface.co/Retrieve0/Qwen3-ASR-0.6B-EXL3-Decoder). The [Qwen3-ASR-TensorRT repository](https://github.com/lost0427/Qwen3-ASR-TensorRT) integrates the encoder and decoder for transcription.
### Files
| Precision | Files | Use |
| --- | --- | --- |
| BF16 | `encoder-window.bf16.onnx`, `encoder-window.bf16.onnx.data` | Reference export for GPUs with BF16 support |
| FP16 | `encoder-window.fp16.onnx`, `encoder-window.fp16.onnx.data` | Compatibility export for older NVIDIA GPUs |
TensorRT `.engine` files are built locally for the target GPU and TensorRT environment.
### Precision and calibration
BF16 is the reference precision. FP16 supports NVIDIA GPUs without BF16. This repository publishes the BF16 and FP16 encoder exports.
### Requirements
- NVIDIA GPU and CUDA
- Reference environment: CUDA 12.8, TensorRT 11.4.0.106, Python 3.10–3.13
- The [Qwen3-ASR-TensorRT repository](https://github.com/lost0427/Qwen3-ASR-TensorRT), which includes the encoder/decoder integration
Windows is tested and supported. Linux should work with compatible CUDA, TensorRT, PyTorch, and ExLlamaV3 packages.
### Build
```powershell
uv sync --extra quantization
uv run qwen3-asr --model Qwen/Qwen3-ASR-0.6B-hf build --encoder-precision bf16
uv run qwen3-asr --model Qwen/Qwen3-ASR-0.6B-hf build --encoder-precision fp16
```
### License and citation
This export is derived from `Qwen/Qwen3-ASR-0.6B-hf` and is published under Apache-2.0. Retain the upstream attribution and license. Please also cite the Qwen3-ASR work and the runtime project.
## 中文
本仓库提供 [Qwen/Qwen3-ASR-0.6B-hf](https://huggingface.co/Qwen/Qwen3-ASR-0.6B-hf) 音频编码器的 ONNX 导出，可用于 TensorRT 推理。使用时需配套兼容的 Qwen3-ASR 文本解码器，例如 [EXL3 解码器](https://huggingface.co/Retrieve0/Qwen3-ASR-0.6B-EXL3-Decoder)。[Qwen3-ASR-TensorRT 仓库](https://github.com/lost0427/Qwen3-ASR-TensorRT)已集成 encoder 和 decoder，可用于完整转写。
### 文件
| 精度 | 文件 | 用途 |
| --- | --- | --- |
| BF16 | `encoder-window.bf16.onnx`、`encoder-window.bf16.onnx.data` | 支持 BF16 的 GPU 基准导出 |
| FP16 | `encoder-window.fp16.onnx`、`encoder-window.fp16.onnx.data` | 兼容不支持 BF16 的较旧 NVIDIA GPU |
TensorRT `.engine` 文件需根据目标 GPU 和 TensorRT 环境在本地构建。
### 精度与校准
BF16 是基准精度，FP16 用于不支持 BF16 的 NVIDIA GPU。本仓库发布 BF16 和 FP16 encoder 导出文件。
### 运行要求
- NVIDIA GPU 与 CUDA
- 参考环境：CUDA 12.8、TensorRT 11.4.0.106、Python 3.10–3.13
- [Qwen3-ASR-TensorRT 仓库](https://github.com/lost0427/Qwen3-ASR-TensorRT)，其中已集成 encoder 和 decoder
项目已在 Windows 上测试并可用。Linux 环境在 CUDA、TensorRT、PyTorch 和 ExLlamaV3 软件包兼容时也可运行。
### 构建
```powershell
uv sync --extra quantization
uv run qwen3-asr --model Qwen/Qwen3-ASR-0.6B-hf build --encoder-precision bf16
uv run qwen3-asr --model Qwen/Qwen3-ASR-0.6B-hf build --encoder-precision fp16
```
### 许可证与引用
本导出基于 `Qwen/Qwen3-ASR-0.6B-hf`，以 Apache-2.0 发布。请保留上游归属说明和许可证，并引用 Qwen3-ASR 工作及本运行时项目。
|