Card: CER lead, WER level (both harness runs)
Browse files
README.md
CHANGED
|
@@ -121,9 +121,11 @@ Freya-TR-Eval ([freyavoice/freya-tr-eval](https://huggingface.co/datasets/freyav
|
|
| 121 |
| EMA Lightning 1.0.1 (our run) | 8.6M | 1.00 (0.89–1.17) | 0.20 | 3.30 | 2.88 |
|
| 122 |
| VoxCPM2 speaking the same voice (the teacher; 1 seed) | much larger | 0.92 | 0.20 | 3.42 | 3.25 |
|
| 123 |
|
| 124 |
-
With its default settings Antalia-2 Mini
|
| 125 |
-
|
| 126 |
-
|
|
|
|
|
|
|
| 127 |
in the 8 kHz band, the band a phone call carries, and is smaller (7.62M vs 8.6M parameters). EMA
|
| 128 |
Lightning's own card reports 0.92 / 0.18 / ~3.30 on the same protocol.
|
| 129 |
|
|
|
|
| 121 |
| EMA Lightning 1.0.1 (our run) | 8.6M | 1.00 (0.89–1.17) | 0.20 | 3.30 | 2.88 |
|
| 122 |
| VoxCPM2 speaking the same voice (the teacher; 1 seed) | much larger | 0.92 | 0.20 | 3.42 | 3.25 |
|
| 123 |
|
| 124 |
+
With its default settings Antalia-2 Mini makes the fewest character errors and is level with EMA
|
| 125 |
+
Lightning on word errors. On this harness: CER 0.19% vs 0.20%, WER 0.97% vs 1.00%. In a second,
|
| 126 |
+
independent run on the public benchmark page's harness (other seeds, one NVIDIA L4): CER 0.20% vs
|
| 127 |
+
0.22%, the lowest of the 15 Turkish systems measured there, and WER 1.05% vs 1.04%. Both models vary
|
| 128 |
+
by about ±0.1 WER from seed to seed, so the WER difference is within noise either way. EMA Lightning is ahead on full-band UTMOS (3.30 vs 3.25); Antalia-2 Mini scores higher on UTMOS
|
| 129 |
in the 8 kHz band, the band a phone call carries, and is smaller (7.62M vs 8.6M parameters). EMA
|
| 130 |
Lightning's own card reports 0.92 / 0.18 / ~3.30 on the same protocol.
|
| 131 |
|