Publish Qwen3-ASR model files

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+encoder-window.bf16.onnx.data filter=lfs diff=lfs merge=lfs -text
+encoder-window.fp16.onnx.data filter=lfs diff=lfs merge=lfs -text
+encoder-window.int8.onnx_data filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,104 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen3-ASR-1.7B-hf
+library_name: onnxruntime
+pipeline_tag: automatic-speech-recognition
+tags:
+- qwen3-asr
+- tensorrt
+- onnx
+- bf16
+- fp16
+- int8
+---
+
+<p align="center">
+  <a href="https://github.com/lost0427/Qwen3-ASR-TensorRT"><img src="https://img.shields.io/badge/GitHub-source-181717?logo=github" alt="GitHub source"></a>
+  <a href="https://huggingface.co/Retrieve0/Qwen3-ASR-1.7B-TensorRT-Encoder"><img src="https://img.shields.io/badge/Hugging_Face-model-yellow?logo=huggingface" alt="Hugging Face model"></a>
+</p>
+
+# Qwen3-ASR 1.7B TensorRT Encoder
+
+## English
+
+ONNX exports of the audio encoder from [Qwen/Qwen3-ASR-1.7B-hf](https://huggingface.co/Qwen/Qwen3-ASR-1.7B-hf), prepared for TensorRT inference. Use an encoder export with a compatible Qwen3-ASR text decoder, such as the [EXL3 decoder](https://huggingface.co/Retrieve0/Qwen3-ASR-1.7B-EXL3-Decoder). The [Qwen3-ASR-TensorRT repository](https://github.com/lost0427/Qwen3-ASR-TensorRT) integrates the encoder and decoder for transcription.
+
+### Files
+
+| Precision | Files | Use |
+| --- | --- | --- |
+| BF16 | `encoder-window.bf16.onnx`, `encoder-window.bf16.onnx.data` | Reference export for GPUs with BF16 support |
+| FP16 | `encoder-window.fp16.onnx`, `encoder-window.fp16.onnx.data` | Compatibility export for older NVIDIA GPUs |
+| INT8 Q/DQ | `encoder-window.int8.onnx`, `encoder-window.int8.onnx_data` | Calibrated quantized export |
+
+TensorRT `.engine` files are not included because they depend on the target GPU architecture and TensorRT build environment. Build an engine locally with this project.
+
+### Precision and calibration
+
+BF16 is the project reference precision. FP16 supports NVIDIA GPUs without BF16. INT8 applies symmetric Q/DQ quantization to Conv and MatMul operations; remaining operations use floating-point precision. Calibration used 32 audio files and produced 52 encoder windows.
+
+### Requirements
+
+- NVIDIA GPU and CUDA
+- Reference environment: CUDA 12.8, TensorRT 11.4.0.106, Python 3.10–3.13
+- The [Qwen3-ASR-TensorRT repository](https://github.com/lost0427/Qwen3-ASR-TensorRT), which includes the encoder/decoder integration
+
+Windows is tested and supported. Linux should work with compatible CUDA, TensorRT, PyTorch, and ExLlamaV3 packages.
+
+### Build
+
+```powershell
+uv sync --extra quantization
+
+uv run qwen3-asr --model Qwen/Qwen3-ASR-1.7B-hf build --encoder-precision bf16
+uv run qwen3-asr --model Qwen/Qwen3-ASR-1.7B-hf build --encoder-precision fp16
+
+uv run qwen3-asr --model Qwen/Qwen3-ASR-1.7B-hf quantize-encoder .\calibration-audio --max-audio-files 32
+uv run qwen3-asr --model Qwen/Qwen3-ASR-1.7B-hf build --encoder-precision int8
+```
+
+### License and citation
+
+This export is derived from `Qwen/Qwen3-ASR-1.7B-hf` and is published under Apache-2.0. Retain the upstream attribution and license. Please also cite the Qwen3-ASR work and the runtime project.
+
+## 中文
+
+本仓库提供 [Qwen/Qwen3-ASR-1.7B-hf](https://huggingface.co/Qwen/Qwen3-ASR-1.7B-hf) 音频编码器的 ONNX 导出，可用于 TensorRT 推理。使用时需配套兼容的 Qwen3-ASR 文本解码器，例如 [EXL3 解码器](https://huggingface.co/Retrieve0/Qwen3-ASR-1.7B-EXL3-Decoder)。[Qwen3-ASR-TensorRT 仓库](https://github.com/lost0427/Qwen3-ASR-TensorRT)已集成 encoder 和 decoder，可用于完整转写。
+
+### 文件
+
+| 精度 | 文件 | 用途 |
+| --- | --- | --- |
+| BF16 | `encoder-window.bf16.onnx`、`encoder-window.bf16.onnx.data` | 支持 BF16 的 GPU 上的基准导出 |
+| FP16 | `encoder-window.fp16.onnx`、`encoder-window.fp16.onnx.data` | 兼容不支持 BF16 的较旧 NVIDIA GPU |
+| INT8 Q/DQ | `encoder-window.int8.onnx`、`encoder-window.int8.onnx_data` | 经校准的量化导出 |
+
+本仓库不提供 TensorRT `.engine` 文件，因为它们依赖目标 GPU 架构和 TensorRT 构建环境。请在目标设备上使用本项目构建 engine。
+
+### 精度与校准
+
+BF16 是本项目的基准精度。FP16 用于不支持 BF16 的 NVIDIA GPU。INT8 对 Conv 和 MatMul 运算应用对称 Q/DQ 量化，其余运算保留浮点精度。校准使用了 32 个音频文件，生成 52 个 encoder window。
+
+### 运行要求
+
+- NVIDIA GPU 与 CUDA
+- 参考环境：CUDA 12.8、TensorRT 11.4.0.106、Python 3.10–3.13
+- [Qwen3-ASR-TensorRT 仓库](https://github.com/lost0427/Qwen3-ASR-TensorRT)，其中已包含 encoder/decoder 集成
+
+项目已在 Windows 上测试并可用。Linux 环境在 CUDA、TensorRT、PyTorch 和 ExLlamaV3 软件包版本兼容时也可运行。
+
+### 构建
+
+```powershell
+uv sync --extra quantization
+
+uv run qwen3-asr --model Qwen/Qwen3-ASR-1.7B-hf build --encoder-precision bf16
+uv run qwen3-asr --model Qwen/Qwen3-ASR-1.7B-hf build --encoder-precision fp16
+
+uv run qwen3-asr --model Qwen/Qwen3-ASR-1.7B-hf quantize-encoder .\calibration-audio --max-audio-files 32
+uv run qwen3-asr --model Qwen/Qwen3-ASR-1.7B-hf build --encoder-precision int8
+```
+
+### 许可证与引用
+
+本导出基于 `Qwen/Qwen3-ASR-1.7B-hf`，以 Apache-2.0 发布。请保留上游归属说明和许可证，并引用 Qwen3-ASR 工作及本运行时项目。

chat_template.jinja

@@ -0,0 +1,8 @@
+{%- set ns = namespace(system_text='') -%}{%- for m in messages -%}{%- if m.role == 'system' -%}{%- if m.content is string -%}{%- set ns.system_text = ns.system_text + m.content -%}{%- else -%}{%- for c in m.content -%}{%- if c.type == 'text' and (c.text is defined) -%}{%- set ns.system_text = ns.system_text + c.text -%}{%- endif -%}{%- endfor -%}{%- endif -%}{%- endif -%}{%- endfor -%}{%- set ns2 = namespace(audio_tokens='') -%}{%- for m in messages -%}{%- if m.content is not string -%}{%- for c in m.content -%}{%- if c.type == 'audio' or ('audio' in c) or ('audio_url' in c) -%}{%- set ns2.audio_tokens = ns2.audio_tokens + '<|audio_start|><|audio_pad|><|audio_end|>' -%}{%- endif -%}{%- endfor -%}{%- endif -%}{%- endfor -%}{{- '<|im_start|>system
+' + ns.system_text + '<|im_end|>
+' -}}{{- '<|im_start|>user
+' + ns2.audio_tokens + '<|im_end|>
+' -}}{%- for m in messages -%}{%- if m.role == 'assistant' -%}{%- set ns3 = namespace(assistant_text='') -%}{%- if m.content is string -%}{%- set ns3.assistant_text = m.content -%}{%- else -%}{%- for c in m.content -%}{%- if c.type == 'text' and (c.text is defined) -%}{%- set ns3.assistant_text = ns3.assistant_text + c.text -%}{%- endif -%}{%- endfor -%}{%- endif -%}{{- '<|im_start|>assistant
+' -}}{% generation %}{{- ns3.assistant_text + '<|im_end|>
+' -}}{% endgeneration %}{%- endif -%}{%- endfor -%}{%- if add_generation_prompt -%}{{- '<|im_start|>assistant
+' -}}{%- endif -%}

config.json

@@ -0,0 +1,97 @@
+{
+  "architectures": [
+    "Qwen3ASRForConditionalGeneration"
+  ],
+  "audio_config": {
+    "activation_dropout": 0,
+    "activation_function": "gelu",
+    "attention_dropout": 0,
+    "conv_chunksize": 500,
+    "d_model": 1024,
+    "downsample_hidden_size": 480,
+    "dropout": 0,
+    "encoder_attention_heads": 16,
+    "encoder_ffn_dim": 4096,
+    "encoder_layers": 24,
+    "initializer_range": 0.02,
+    "max_position_embeddings": 13,
+    "model_type": "qwen3_asr_encoder",
+    "n_window": 50,
+    "n_window_infer": 800,
+    "num_key_value_heads": 16,
+    "num_mel_bins": 128,
+    "output_dim": 2048,
+    "scale_embedding": false
+  },
+  "audio_token_id": 151676,
+  "dtype": "bfloat16",
+  "eos_token_id": [
+    151643,
+    151645
+  ],
+  "initializer_range": 0.02,
+  "model_type": "qwen3_asr",
+  "pad_token_id": 151645,
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "bos_token_id": null,
+    "eos_token_id": null,
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 2048,
+    "initializer_range": 0.02,
+    "intermediate_size": 6144,
+    "layer_types": [
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention",
+      "full_attention"
+    ],
+    "max_position_embeddings": 65536,
+    "max_window_layers": 28,
+    "model_type": "qwen3",
+    "num_attention_heads": 16,
+    "num_hidden_layers": 28,
+    "num_key_value_heads": 8,
+    "pad_token_id": null,
+    "rms_norm_eps": 1e-06,
+    "rope_parameters": {
+      "rope_theta": 1000000,
+      "rope_type": "default"
+    },
+    "sliding_window": null,
+    "tie_word_embeddings": true,
+    "use_cache": true,
+    "use_sliding_window": false,
+    "vocab_size": 151936
+  },
+  "tie_word_embeddings": true,
+  "timestamp_token_id": 151705,
+  "token_classification_bias": false,
+  "transformers_version": "5.13.0.dev0"
+}

encoder-window.bf16.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:231e77593bd96b3335207f4bbd3927e94c89270d54589581ca05357180cc2bac
+size 924195

encoder-window.bf16.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37698aa0cb2060137868768ae376e2af0828c768ac4c726bbdcb8d386b26f514
+size 635009088

encoder-window.fp16.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c2357719ed004677b7c79f2e47afa163d7c4491002628cbee831ca75d2d4667
+size 924193

encoder-window.fp16.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fe5c39bb66ba91e328623e84dd090ec7b8eccda83a8670e80d7a0e0ee30dfdd
+size 635009088

encoder-window.int8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6cc2b80f21dd5036919e8a67abc4d90452500f17eaa24f74e530c248b225f585
+size 1893394

encoder-window.int8.onnx_data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c92889d879e992d617a5c1070bf206fb08acf96c056c62657a6a094ed95aaef
+size 634978752

generation_config.json

@@ -0,0 +1,10 @@
+{
+  "do_sample": false,
+  "eos_token_id": [
+    151643,
+    151645
+  ],
+  "max_new_tokens": 512,
+  "pad_token_id": 151645,
+  "transformers_version": "5.13.0.dev0"
+}

processor_config.json

@@ -0,0 +1,20 @@
+{
+  "feature_extractor": {
+    "chunk_length": 30,
+    "dither": 0.0,
+    "feature_extractor_type": "Qwen3ASRFeatureExtractor",
+    "feature_size": 128,
+    "hop_length": 160,
+    "min_length": 8000,
+    "n_fft": 400,
+    "n_samples": 480000,
+    "n_window": 50,
+    "nb_max_frames": 3000,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "return_attention_mask": true,
+    "sampling_rate": 16000
+  },
+  "processor_class": "Qwen3ASRProcessor",
+  "timestamp_segment_time": 80
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe1fad59be22a41ee293363fcf95fdedbc7c93f3b49270b1d2e18bd1399a7a05
+size 11429653

tokenizer_config.json

@@ -0,0 +1,32 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "processor_class": "Qwen3ASRProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}