bezzam HF Staff commited on
Commit
2bb067e
·
verified ·
1 Parent(s): a21c97c

Add Open ASR Leaderboard results (English and multilingual)

Browse files

Results from the Open ASR Leaderboard, English and multilingual tracks, with RTFx.

Files changed (1) hide show
  1. README.md +44 -0
README.md CHANGED
@@ -130,3 +130,47 @@ Pass these options through vLLM's `chat_template_kwargs`, or directly to
130
  ## License
131
 
132
  Licensed under the [Apache License 2.0](LICENSE).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
130
  ## License
131
 
132
  Licensed under the [Apache License 2.0](LICENSE).
133
+
134
+ ## Open ASR Leaderboard results
135
+
136
+ Evaluated on the [Open ASR Leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard?version=1.16.1) (release 18-11-2026, v1.16.1), run on an H200 with vLLM. Compare against other open models on the multilingual tracks with [this view](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard?languages=Reson8%2FRumble-2-Flash&proprietary=false) (API models hidden).
137
+
138
+ ### English track
139
+
140
+ Average WER over the 7 default splits: **6.08** (RTFx 6338).
141
+
142
+ | Split | WER (%) | RTFx |
143
+ |---|---|---|
144
+ | AMI | 9.24 | 4245 |
145
+ | Earnings22 | 6.45 | 4811 |
146
+ | GigaSpeech | 7.56 | 5978 |
147
+ | SPGISpeech | 3.08 | 7308 |
148
+ | URGENT24 | 10.25 | 3314 |
149
+ | Voice Arena Monsoon | 3.56 | 3760 |
150
+ | VoxPopuli | 2.43 | 4222 |
151
+
152
+ ### Multilingual track
153
+
154
+ | Language | Dataset | WER (%) | RTFx |
155
+ |---|---|---|---|
156
+ | German | FLEURS | 4.77 | 5967 |
157
+ | German | Common Voice | 6.28 | 5967 |
158
+ | Spanish | FLEURS | 2.82 | 6079 |
159
+ | Spanish | Common Voice | 4.33 | 6079 |
160
+ | Spanish | MLS | 4.04 | 6079 |
161
+ | French | FLEURS | 5.95 | 6001 |
162
+ | French | Common Voice | 10.01 | 6001 |
163
+ | French | MLS | 5.66 | 6001 |
164
+ | Italian | FLEURS | 2.53 | 5772 |
165
+ | Italian | Common Voice | 4.84 | 5772 |
166
+ | Italian | MLS | 7.90 | 5772 |
167
+ | Dutch | FLEURS | 6.05 | 3494 |
168
+ | Dutch | Common Voice | 4.92 | 3494 |
169
+ | Dutch | MLS | 9.38 | 3494 |
170
+ | Portuguese | FLEURS | 5.31 | 4723 |
171
+ | Portuguese | MLS | 7.61 | 4723 |
172
+ | Hindi | Voice Arena Monsoon | 14.11 | 2101 |
173
+ | Chinese (CER) | FLEURS | 5.82 | 5379 |
174
+ | Chinese (CER) | AISHELL-1 | 7.31 | 5379 |
175
+
176
+ RTFx is the inverse real-time factor (seconds of audio transcribed per second of compute).