Update benchmark numbers
Browse files
README.md
CHANGED
|
@@ -40,7 +40,7 @@ The existing 3 bpw and 4 bpw files are legacy exports. They were quantized with
|
|
| 40 |
|
| 41 |
The `decoder-exl3-3bpw-train-clean-100` and `decoder-exl3-4bpw-train-clean-100` directories contain the calibrated exports. Their calibration tensors contain 200 packed rows of 2,048 tokens, built from 1,603 paired audio requests and transcripts in LibriSpeech `train-clean-100`. No `test-clean` samples were used. Calibration changes quantization statistics; it is not fine-tuning.
|
| 42 |
|
| 43 |
-
On all 2,620 LibriSpeech `test-clean` utterances, the calibrated 4 bpw variant measured 1.
|
| 44 |
|
| 45 |
The `train-clean-100` variants are tuned for LibriSpeech-style English read speech. For general-purpose audio and Chinese ASR, the legacy decoders remain the default recommendation.
|
| 46 |
|
|
@@ -105,7 +105,7 @@ These weights are derived from `Qwen/Qwen3-ASR-1.7B-hf` and are published under
|
|
| 105 |
|
| 106 |
`decoder-exl3-3bpw-train-clean-100` 和 `decoder-exl3-4bpw-train-clean-100` 目录保存新版权重。校准张量由 LibriSpeech `train-clean-100` 的 1,603 组音频请求和转写文本打包为 200 行,每行 2,048 个 token。校准未使用 `test-clean` 样本。校准会影响量化统计,不会微调模型。
|
| 107 |
|
| 108 |
-
在全部 2,620 条 LibriSpeech `test-clean` 音频上,新版 4 bpw 的 WER 为 1.
|
| 109 |
|
| 110 |
`train-clean-100` 版本针对 LibriSpeech 风格的英语朗读调优。通用音频和中文 ASR 仍默认推荐 legacy 版本。
|
| 111 |
|
|
|
|
| 40 |
|
| 41 |
The `decoder-exl3-3bpw-train-clean-100` and `decoder-exl3-4bpw-train-clean-100` directories contain the calibrated exports. Their calibration tensors contain 200 packed rows of 2,048 tokens, built from 1,603 paired audio requests and transcripts in LibriSpeech `train-clean-100`. No `test-clean` samples were used. Calibration changes quantization statistics; it is not fine-tuning.
|
| 42 |
|
| 43 |
+
On all 2,620 LibriSpeech `test-clean` utterances, the calibrated 4 bpw variant measured 1.9108% WER and 205.11 ms median latency, compared with 1.9390% and 204.36 ms for the legacy 4 bpw file. The calibrated 3 bpw variant measured 2.1178% WER and 192.18 ms median, compared with 2.0256% and 189.06 ms for the legacy 3 bpw file; calibration did not improve the 3 bpw tradeoff. Each variant was run once; timings are not repeated-run averages.
|
| 44 |
|
| 45 |
The `train-clean-100` variants are tuned for LibriSpeech-style English read speech. For general-purpose audio and Chinese ASR, the legacy decoders remain the default recommendation.
|
| 46 |
|
|
|
|
| 105 |
|
| 106 |
`decoder-exl3-3bpw-train-clean-100` 和 `decoder-exl3-4bpw-train-clean-100` 目录保存新版权重。校准张量由 LibriSpeech `train-clean-100` 的 1,603 组音频请求和转写文本打包为 200 行,每行 2,048 个 token。校准未使用 `test-clean` 样本。校准会影响量化统计,不会微调模型。
|
| 107 |
|
| 108 |
+
在全部 2,620 条 LibriSpeech `test-clean` 音频上,新版 4 bpw 的 WER 为 1.9108%,中位延迟为 205.11 ms;旧版 4 bpw 分别为 1.9390% 和 204.36 ms。新版 3 bpw 的 WER 为 2.1178%,中位延迟为 192.18 ms;旧版 3 bpw 分别为 2.0256% 和 189.06 ms,校准没有改善 3 bpw 的取舍。每个版本只运行了一次,延迟不是多轮平均值。
|
| 109 |
|
| 110 |
`train-clean-100` 版本针对 LibriSpeech 风格的英语朗读调优。通用音频和中文 ASR 仍默认推荐 legacy 版本。
|
| 111 |
|