Retrieve0/Qwen3-ASR-1.7B-EXL3-Decoder

Qwen3-ASR 1.7B EXL3 Decoder

English

EXL3 quantized text-decoder weights derived from Qwen/Qwen3-ASR-1.7B-hf. Use with a compatible TensorRT encoder. The Qwen3-ASR-TensorRT repository integrates the encoder and decoder for transcription. These weights use the ExLlamaV3 format and are loaded by the project runtime.

Variants

Variant	Approximate size	Use
decoder-exl3-3bpw	1.30 GiB	Lower memory use and faster decoding
decoder-exl3-4bpw	1.46 GiB	Higher-precision alternative

Each variant includes EXL3 model.safetensors, tokenizer files, model and generation configuration, and quantization configuration. EXL3 uses quantized weight tensors alongside floating-point scale, normalization, and auxiliary tensors; the variants are identified by their EXL3 bitrate.

Requirements

• NVIDIA GPU and CUDA
• ExLlamaV3 1.6.6 with compatible PyTorch and CUDA versions
• Reference environment: PyTorch 2.10.0, CUDA 12.8, Python 3.10–3.13
• The Qwen3-ASR-TensorRT repository, which includes the encoder/decoder integration

Windows is tested and supported. Linux should work with compatible CUDA, TensorRT, PyTorch, and ExLlamaV3 packages.

Usage

uv sync

uv run qwen3-asr --model Qwen/Qwen3-ASR-1.7B-hf transcribe .\audio.wav `
  --backend tensorrt-exllama `
  --decoder-dir .\decoder-exl3-3bpw `
  --language Chinese

Start the HTTP API with the same backend:

uv run qwen3-asr --model Qwen/Qwen3-ASR-1.7B-hf serve `
  --backend tensorrt-exllama `
  --decoder-dir .\decoder-exl3-3bpw

License and citation

These weights are derived from Qwen/Qwen3-ASR-1.7B-hf and are published under Apache-2.0. Retain the upstream attribution and license. ExLlamaV3 and runtime dependencies have their own licenses. Please cite Qwen3-ASR, ExLlamaV3, and the runtime project.

中文

本仓库提供基于 Qwen/Qwen3-ASR-1.7B-hf 转换的 EXL3 量化文本解码器权重。使用时需配套兼容的 TensorRT encoder。Qwen3-ASR-TensorRT 仓库已集成 encoder 和 decoder，可用于完整转写。本权重采用 ExLlamaV3 格式，由项目运行时加载。

量化版本

版本	大小约	用途
decoder-exl3-3bpw	1.30 GiB	显存占用较低、解码速度较快
decoder-exl3-4bpw	1.46 GiB	精度优先的替代版本

每个版本包含 EXL3 model.safetensors、tokenizer、模型与生成配置、量化配置。EXL3 权重由量化张量以及浮点缩放、归一化和辅助张量组成，版本以 EXL3 bpw 标识。

运行要求

• NVIDIA GPU 与 CUDA
• ExLlamaV3 1.6.6，以及相互兼容的 PyTorch 和 CUDA 版本
• 参考环境：PyTorch 2.10.0、CUDA 12.8、Python 3.10–3.13
• Qwen3-ASR-TensorRT 仓库，其中已包含 encoder/decoder 集成

项目已在 Windows 上测试并可用。Linux 环境在 CUDA、TensorRT、PyTorch 和 ExLlamaV3 软件包版本兼容时也可运行。

使用

uv sync

uv run qwen3-asr --model Qwen/Qwen3-ASR-1.7B-hf transcribe .\audio.wav `
  --backend tensorrt-exllama `
  --decoder-dir .\decoder-exl3-3bpw `
  --language Chinese

使用相同 backend 启动 HTTP API：

uv run qwen3-asr --model Qwen/Qwen3-ASR-1.7B-hf serve `
  --backend tensorrt-exllama `
  --decoder-dir .\decoder-exl3-3bpw

许可证与引用

这些权重基于 Qwen/Qwen3-ASR-1.7B-hf，以 Apache-2.0 发布。请保留上游归属说明和许可证。ExLlamaV3 与运行时依赖各自遵循其许可证。请引用 Qwen3-ASR、ExLlamaV3 及运行时项目。