File size: 1,871 Bytes
588788b 09db84f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | # Final Aligner Selection
Decision: `SELECTED_MFA` (2026-09-06).
This decision supersedes the 2026-09-04 `SELECTED_WHISPERX` operational choice after the
previously missing strict MFA full-36 evaluation was completed. Historical Stage 5 evidence
remains unchanged.
MFA, WhisperX, and NeMo now all have successful 36/36 candidates. On the five rows with
manual first-word gold, MFA and WhisperX remain effectively tied: MFA first-word-end MAE is
0.0162 s (median 0.0110, max 0.0300), while WhisperX is 0.0154 s (median 0.0140,
max 0.0310). NeMo is 0.2210 s with a 0.9200 s maximum error on EN0949.
Across all 36 rows, MFA and WhisperX differ by at least 0.250 s on 7 rows, versus 28 for
MFA/NeMo and 32 for WhisperX/NeMo. MFA and WhisperX are especially close at first-word
boundaries (mean absolute differences 0.059306 s start and 0.018694 s end). Their material
disagreements are concentrated at last-word boundaries. On the largest disputed cases,
retained MFA forced-alignment evidence and independent NeMo last-boundary timings support
the later MFA placements. This is diagnostic evidence rather than manual ground truth.
The canonical final output is `results/final/final-annotated-36.csv` with SHA-256
`37bcb85fc22fb142dae40453b90dd3b427bbfd657861bee7529cd433915535a8` and is byte-identical to
`results/postfinal/mfa-full-36/annotated-36-mfa.csv`.
## Limitations
- Manual gold covers only `first_word_text` and `first_word_end_s` on five rows.
- First-word start and both last-word boundaries have no manual ground truth.
- Pairwise agreement is not accuracy.
- The 0.0008 s MFA/WhisperX five-row MAE difference is too small to rank the systems by itself.
- Runtime results are from different hosts/cache states and are secondary evidence.
- This selection is an operational choice from the available evidence, not a claim of universal timing superiority.
|