Update benchmark numbers
Browse files
README.md
CHANGED
|
@@ -40,7 +40,7 @@ The existing `decoder-exl3-3bpw` and `decoder-exl3-4bpw` files are legacy export
|
|
| 40 |
|
| 41 |
The separately published `decoder-exl3-3bpw-train-clean-100` and `decoder-exl3-4bpw-train-clean-100` directories contain the calibrated exports. Their calibration tensors contain 200 packed rows of 2,048 tokens, built from 1,603 paired audio requests and transcripts in LibriSpeech `train-clean-100`. No `test-clean` samples were used. Calibration changes quantization statistics; it is not fine-tuning.
|
| 42 |
|
| 43 |
-
On all 2,620 LibriSpeech `test-clean` utterances, the calibrated 3 bpw variant measured 2.
|
| 44 |
|
| 45 |
The `train-clean-100` variants are tuned for LibriSpeech-style English read speech. For general-purpose audio and Chinese ASR, the legacy decoders remain the default recommendation.
|
| 46 |
|
|
@@ -50,7 +50,7 @@ The `train-clean-100` variants are tuned for LibriSpeech-style English read spee
|
|
| 50 |
|
| 51 |

|
| 52 |
|
| 53 |
-
All configurations use all 2,620 test-clean utterances. The EXL3 decoders are about 2.
|
| 54 |
|
| 55 |
### Requirements
|
| 56 |
|
|
@@ -97,7 +97,7 @@ These weights are derived from `Qwen/Qwen3-ASR-0.6B-hf` and are published under
|
|
| 97 |
|
| 98 |
独立目录 `decoder-exl3-3bpw-train-clean-100` 和 `decoder-exl3-4bpw-train-clean-100` 中保存新版权重。校准张量由 LibriSpeech `train-clean-100` 的 1,603 组音频请求和转写文本打包为 200 行,每行 2,048 个 token。校准未使用 `test-clean` 样本。校准会影响量化统计,不会微调模型。
|
| 99 |
|
| 100 |
-
在全部 2,620 条 LibriSpeech `test-clean` 音频上,新版 3 bpw 的 WER 为 2.
|
| 101 |
|
| 102 |
`train-clean-100` 版本针对 LibriSpeech 风格的英语朗读调优。通用音频和中文 ASR 仍默认推荐 legacy 版本。
|
| 103 |
|
|
|
|
| 40 |
|
| 41 |
The separately published `decoder-exl3-3bpw-train-clean-100` and `decoder-exl3-4bpw-train-clean-100` directories contain the calibrated exports. Their calibration tensors contain 200 packed rows of 2,048 tokens, built from 1,603 paired audio requests and transcripts in LibriSpeech `train-clean-100`. No `test-clean` samples were used. Calibration changes quantization statistics; it is not fine-tuning.
|
| 42 |
|
| 43 |
+
On all 2,620 LibriSpeech `test-clean` utterances, the calibrated 3 bpw variant measured 2.5527% WER and 98.32 ms median latency, compared with 2.8784% and 103.43 ms for the legacy 3 bpw file. The calibrated 4 bpw variant measured 2.3419% WER and 96.83 ms median, compared with 2.4228% and 96.65 ms for the legacy 4 bpw file. These are two-pass 2026-11-10 rerun medians; timings are not repeated-run averages.
|
| 44 |
|
| 45 |
The `train-clean-100` variants are tuned for LibriSpeech-style English read speech. For general-purpose audio and Chinese ASR, the legacy decoders remain the default recommendation.
|
| 46 |
|
|
|
|
| 50 |
|
| 51 |

|
| 52 |
|
| 53 |
+
All configurations use all 2,620 test-clean utterances. The EXL3 decoders are about 2.9–3.1× faster than GGUF Q4_K. The official `qwen-asr` Transformers backend reaches 2.3212% WER at 787.12 ms, about 8.1× slower than the best EXL3 variant. The chart labels give WER, median latency, and the speed ratio.
|
| 54 |
|
| 55 |
### Requirements
|
| 56 |
|
|
|
|
| 97 |
|
| 98 |
独立目录 `decoder-exl3-3bpw-train-clean-100` 和 `decoder-exl3-4bpw-train-clean-100` 中保存新版权重。校准张量由 LibriSpeech `train-clean-100` 的 1,603 组音频请求和转写文本打包为 200 行,每行 2,048 个 token。校准未使用 `test-clean` 样本。校准会影响量化统计,不会微调模型。
|
| 99 |
|
| 100 |
+
在全部 2,620 条 LibriSpeech `test-clean` 音频上,新版 3 bpw 的 WER 为 2.5527%,中位延迟为 98.32 ms;旧版 3 bpw 分别为 2.8784% 和 103.43 ms。新版 4 bpw 的 WER 为 2.3419%,中位延迟为 96.83 ms;旧版 4 bpw 分别为 2.4228% 和 96.65 ms。以上为 2026-11-10 两轮复测的中位数,延迟不是多轮平均值。
|
| 101 |
|
| 102 |
`train-clean-100` 版本针对 LibriSpeech 风格的英语朗读调优。通用音频和中文 ASR 仍默认推荐 legacy 版本。
|
| 103 |
|