Testing and Quality
This page defines the deterministic release gate and records the current per-file coverage map. Ordinary tests do not download models, call remote services, require a GPU, or invoke heavyweight aligners.
Environment
Use the core development environment or the local equivalent:
conda activate speech-text-data-aligners
python -m pip install -e . --no-deps
Pinned development tools are in requirements.txt and
environments/core-dev.yml.
Deterministic Gate
Run from the repository root:
python -m pytest
ruff check .
ruff format --check .
python -m mypy src
lint-imports
coverage erase
coverage run -m pytest
coverage report --show-missing
coverage json -o coverage.json
python -m pytest tests/quality/test_per_file_coverage.py -q
coverage.json is generated evidence and is ignored by Git. Regenerate it before
running the per-file gate; a stale file is not accepted as current proof.
Current Observed Gate
Captured in this checkout on 2026-10-30 before documentation-only changes to source:
| Gate | Result |
|---|---|
| Pytest | 635 passed, 3 skipped |
| Aggregate implementation line coverage | 98.74% |
| Lowest implementation file | 93.33% |
| Lowest shared-layer file | 95.65% |
| Ruff lint/format | Pass |
| Strict mypy | Pass in 53 source files |
| Import Linter | 7 kept, 0 broken |
| Architecture tests | 10 passed |
| Header/manifest/per-file tests | Pass |
The repository gate is at least 90% for every non-empty implementation file and at
least 90% aggregate. This repository additionally enforces at least 95% for shared
core, data, application, benchmarking, and runtime files.
Test Boundaries
| Test area | Purpose |
|---|---|
tests/unit/core |
Normalization, mapping, models, validation, errors, extraction, readiness |
tests/unit/data |
CSV preservation, WAV resolution/probing, readiness cohort verification |
tests/unit/application |
Public use cases, publication, benchmarking, readiness, materialization |
tests/unit/benchmarking |
Metrics, reports, readiness distributions |
tests/unit/runtime |
Process, artifacts, receipts, evidence integrity |
tests/unit/backends |
Configs, native builders/parsers, adapters, registry, isolated providers |
tests/unit/cli |
Argument validation and delegation to application services |
tests/architecture |
Dependency direction and readiness isolation |
tests/quality |
Headers, manifest consistency, per-file coverage |
tests/integration/* |
Opt-in real subprocess/model smoke tests |
Integration tests are marked integration plus mfa, nemo, or whisperx. They are
excluded from ordinary confidence unless their runtime and explicit inputs are
available.
Opt-In Integration
MFA:
RUN_MFA_INTEGRATION=1 \
MFA_CONDA_PREFIX=/opt/conda/envs/speech-aligners-mfa \
MFA_SMOKE_WAV=/absolute/path/to/file.wav \
MFA_SMOKE_TRANSCRIPT='Exact authoritative transcript.' \
python -m pytest tests/integration/mfa/test_mfa_real.py -m 'integration and mfa' -v
WhisperX:
RUN_WHISPERX_INTEGRATION=1 \
WHISPERX_PYTHON=/opt/conda/envs/speech-aligners-whisperx/bin/python \
WHISPERX_SMOKE_WAV=/absolute/path/to/file.wav \
WHISPERX_SMOKE_TRANSCRIPT='Exact authoritative transcript.' \
WHISPERX_MODEL_DIR=/opt/models/whisperx-align \
python -m pytest tests/integration/whisperx/test_whisperx_real.py -m 'integration and whisperx' -v
NeMo:
NEMO_PYTHON=/opt/runtimes/nemo-speech-v3.0.0/.venv/bin/python \
NEMO_ALIGN_SCRIPT=/opt/runtimes/nemo-speech-v3.0.0/tools/nemo_forced_aligner/align.py \
NEMO_MODEL_PATH=/opt/models/stt_en_fastconformer_hybrid_large_pc.nemo \
NEMO_WAV_PATH=/absolute/path/to/file.wav \
NEMO_TRANSCRIPT='Exact authoritative transcript.' \
NEMO_DEVICE=cuda \
python -m pytest tests/integration/nemo/test_nemo_real.py -m 'integration and nemo' -v
NeMo may download the registry model only after NEMO_ALLOW_MODEL_DOWNLOAD=1 is set
explicitly. A local checkpoint is preferred. Integration commands require the listed
hardware/data and are not part of ordinary unit CI.
Implementation-to-Test and Coverage Map
Coverage values are current line coverage for each implementation file. Empty package markers are intentionally zero bytes and excluded from implementation coverage; they provide no executable behavior.
| Implementation | Focused test | Coverage |
|---|---|---|
speech_text_data_aligners/__init__.py |
tests/unit/test_public_api.py |
100.00% |
application/annotate.py |
tests/unit/application/test_annotate.py |
98.00% |
application/benchmark.py |
tests/unit/application/test_benchmark.py |
100.00% |
application/materialize.py |
tests/unit/application/test_materialize.py |
95.68% |
application/readiness.py |
tests/unit/application/test_readiness.py |
100.00% |
backends/registry.py |
tests/unit/backends/test_registry.py |
100.00% |
backends/mfa/backend.py |
tests/unit/backends/mfa/test_backend.py |
100.00% |
backends/mfa/config.py |
tests/unit/backends/mfa/test_config.py |
99.03% |
backends/mfa/corpus.py |
tests/unit/backends/mfa/test_corpus.py |
99.19% |
backends/mfa/runner.py |
tests/unit/backends/mfa/test_runner.py |
98.62% |
backends/mfa/textgrid.py |
tests/unit/backends/mfa/test_textgrid.py |
97.91% |
backends/nemo/backend.py |
tests/unit/backends/nemo/test_backend.py |
100.00% |
backends/nemo/config.py |
tests/unit/backends/nemo/test_config.py |
98.11% |
backends/nemo/ctm.py |
tests/unit/backends/nemo/test_ctm.py |
100.00% |
backends/nemo/manifest.py |
tests/unit/backends/nemo/test_manifest.py |
100.00% |
backends/nemo/runner.py |
tests/unit/backends/nemo/test_runner.py |
100.00% |
backends/whisperx/backend.py |
tests/unit/backends/whisperx/test_backend.py |
100.00% |
backends/whisperx/config.py |
tests/unit/backends/whisperx/test_config.py |
100.00% |
backends/whisperx/parser.py |
tests/unit/backends/whisperx/test_parser.py |
100.00% |
backends/whisperx/provider.py |
tests/unit/backends/whisperx/test_provider.py |
98.74% |
backends/whisperx/segments.py |
tests/unit/backends/whisperx/test_segments.py |
100.00% |
backends/whisperx/worker.py |
tests/unit/backends/whisperx/test_worker.py |
93.33% |
benchmarking/metrics.py |
tests/unit/benchmarking/test_metrics.py |
100.00% |
benchmarking/readiness.py |
tests/unit/benchmarking/test_readiness.py |
100.00% |
benchmarking/reports.py |
tests/unit/benchmarking/test_reports.py |
98.84% |
cli/annotate.py |
tests/unit/cli/test_annotate.py |
100.00% |
cli/benchmark.py |
tests/unit/cli/test_benchmark.py |
100.00% |
cli/main.py |
tests/unit/cli/test_main.py |
95.83% |
cli/readiness.py |
tests/unit/cli/test_readiness.py |
100.00% |
core/errors.py |
tests/unit/core/test_errors.py |
100.00% |
core/extraction.py |
tests/unit/core/test_extraction.py |
95.65% |
core/models.py |
tests/unit/core/test_models.py |
100.00% |
core/normalization.py |
tests/unit/core/test_normalization.py |
100.00% |
core/protocols.py |
tests/unit/core/test_protocols.py |
100.00% |
core/readiness.py |
tests/unit/core/test_readiness.py |
100.00% |
core/validation.py |
tests/unit/core/test_validation.py |
98.33% |
data/audio.py |
tests/unit/data/test_audio.py |
97.37% |
data/metadata.py |
tests/unit/data/test_metadata.py |
98.08% |
data/readiness.py |
tests/unit/data/test_readiness.py |
100.00% |
runtime/alignment_evidence.py |
tests/unit/runtime/test_alignment_evidence.py |
100.00% |
runtime/artifacts.py |
tests/unit/runtime/test_artifacts.py |
100.00% |
runtime/process.py |
tests/unit/runtime/test_process.py |
100.00% |
runtime/receipts.py |
tests/unit/runtime/test_receipts.py |
100.00% |
Coverage Interpretation
- Coverage is line coverage from the deterministic full suite with real integration tests skipped.
- High coverage does not replace behavioral review or empirical alignment evidence.
- A skipped integration test is not counted as a successful real backend run.
- Aggregate coverage cannot compensate for a file below its individual threshold.
- When an implementation file is added, add its focused test and update this map in the same change.