Stage verified Inflect v2 release candidate and evaluation evidence
Browse files- docs/EVALUATION.md +4 -4
- release_manifest.json +2 -2
docs/EVALUATION.md
CHANGED
|
@@ -2,20 +2,20 @@
|
|
| 2 |
|
| 3 |
## Matched corpus
|
| 4 |
|
| 5 |
-
|
| 6 |
|
| 7 |
## Systems
|
| 8 |
|
| 9 |
- Inflect-Micro-v2 and Inflect-Nano-v2, fixed release checkpoints.
|
| 10 |
- KittenTTS Nano, Bruno and Hugo reported separately.
|
| 11 |
- Piper low, Ryan and Danny reported separately.
|
| 12 |
-
- Supertonic 3
|
| 13 |
|
| 14 |
-
|
| 15 |
|
| 16 |
## Intelligibility
|
| 17 |
|
| 18 |
-
|
| 19 |
|
| 20 |
## Predicted quality
|
| 21 |
|
|
|
|
| 2 |
|
| 3 |
## Matched corpus
|
| 4 |
|
| 5 |
+
Modern400 intelligibility results use 400 identical unseen English prompts per system. The set combines 200 fixed modern/stress prompts with 200 deterministic Google FLEURS `en_us` test prompts. Exact normalized text was excluded against 87,362 training transcripts. The frozen prompt manifest, corpus hash, exclusion report, compact summary, and compressed row-level reports are included under `evaluation/final/modern400/`. UTMOS22 remains a separate matched quality evaluation and is not merged with Modern400.
|
| 6 |
|
| 7 |
## Systems
|
| 8 |
|
| 9 |
- Inflect-Micro-v2 and Inflect-Nano-v2, fixed release checkpoints.
|
| 10 |
- KittenTTS Nano, Bruno and Hugo reported separately.
|
| 11 |
- Piper low, Ryan and Danny reported separately.
|
| 12 |
+
- Supertonic 3 v1.3.1, official voice M2 at 3 and 8 steps, reported separately.
|
| 13 |
|
| 14 |
+
Kitten and Piper voices are reported separately. The two Supertonic step counts are never pooled.
|
| 15 |
|
| 16 |
## Intelligibility
|
| 17 |
|
| 18 |
+
Semantic WER is run independently with Qwen3-ASR-1.7B and Nemotron 3.5 ASR 0.6B. References and hypotheses are normalized into equivalent spoken English forms so formatting differences such as `12.6` versus `twelve point six` are not automatically counted as speech errors. Both reports include every hypothesis, per-clip errors, prompt-category breakdowns, and 10,000-sample bootstrap intervals. ASR-family results are never averaged into a single headline value.
|
| 19 |
|
| 20 |
## Predicted quality
|
| 21 |
|
release_manifest.json
CHANGED
|
@@ -89,8 +89,8 @@
|
|
| 89 |
},
|
| 90 |
{
|
| 91 |
"path": "docs/EVALUATION.md",
|
| 92 |
-
"bytes":
|
| 93 |
-
"sha256": "
|
| 94 |
},
|
| 95 |
{
|
| 96 |
"path": "docs/FINETUNING.md",
|
|
|
|
| 89 |
},
|
| 90 |
{
|
| 91 |
"path": "docs/EVALUATION.md",
|
| 92 |
+
"bytes": 2487,
|
| 93 |
+
"sha256": "05d2dcf13660693890ed329ce3dab3653a5ee6f96b1fc75a15dd0a2c9b031f93"
|
| 94 |
},
|
| 95 |
{
|
| 96 |
"path": "docs/FINETUNING.md",
|