# Architecture

Speech-Text Data Aligners is a library-first forced-alignment framework. The core
contract accepts WAV audio plus a known, authoritative transcript and returns ordered
backend-neutral timed words. The CLI is a composition and presentation layer.

The mandatory design rules are in
[`docs/MODULAR_ARCHITECTURE_CONTRACT.md`](docs/MODULAR_ARCHITECTURE_CONTRACT.md).
Runtime alignment semantics are in
[`docs/ALIGNER_ENGINEERING_CONTRACT.md`](docs/ALIGNER_ENGINEERING_CONTRACT.md).

## Design Goals

- Keep authoritative reference text immutable.
- Keep core logic deterministic and independent of model/runtime packages.
- Make MFA, NeMo, and WhisperX interchangeable through `AlignmentBackend`.
- Contain native formats inside their adapters.
- Provide reusable Python services rather than CLI-only workflows.
- Keep optional runtimes independently unavailable.
- Preserve source metadata and deterministic output order.
- Represent failure explicitly and never fabricate timing.

## Layer and Dependency Direction

```text
CLI --------------------> Application --------------------> Core
                              |                              ^
                              +----------> Data -------------+
                              +----------> Benchmarking -----+
                              +----------> Runtime ----------+

CLI -- lazy composition --> Concrete Backends
                            |              |
                            +------------> Core
                            +------------> Runtime
```

Enforced consequences:

- Core does not import application, data, benchmarking, runtime, CLI, or backends.
- Data and benchmarking do not import concrete backends or CLI.
- Application services receive `AlignmentBackend` implementations by injection and do
  not branch on concrete backend names.
- Concrete backend packages may depend on core/runtime and their own native modules,
  but never on a sibling backend.
- CLI composition may select a registry factory; it does not own alignment logic.
- Backend factories are lazy dotted references, so importing the public library does
  not load heavyweight aligner dependencies.

## Current Package Tree

```text
src/speech_text_data_aligners/
|-- __init__.py                    stable public facade
|-- core/
|   |-- models.py                  immutable neutral request/result/provenance values
|   |-- protocols.py               AlignmentBackend and structural contracts
|   |-- errors.py                  project exception/failure taxonomy
|   |-- normalization.py           lexical plans and monotonic reconciliation
|   |-- validation.py              timing, lexical, and publication-path invariants
|   |-- extraction.py              first/last word output fields
|   `-- readiness.py               semantic-boundary mapping and consensus
|-- data/
|   |-- metadata.py                preserving, atomic CSV input/output
|   |-- audio.py                   audio resolution and lightweight WAV probing
|   `-- readiness.py               frozen readiness cohort/hash verification
|-- application/
|   |-- annotate.py                reusable annotation orchestration
|   |-- benchmark.py               identical-request benchmark orchestration
|   |-- readiness.py               evidence reuse and offline readiness execution
|   `-- materialize.py             accepted-evidence full-word timing materialization
|-- benchmarking/
|   |-- metrics.py                 sparse-gold, runtime, and pairwise metrics
|   |-- reports.py                 deterministic JSON/CSV/Markdown reporting
|   `-- readiness.py               readiness distributions and sensitivity summaries
|-- runtime/
|   |-- process.py                 shell-free injectable subprocess boundary
|   |-- artifacts.py               temporary/retained artifact lifecycle
|   |-- receipts.py                hashes, host facts, and run receipts
|   `-- alignment_evidence.py      full-word evidence integrity and verification
|-- backends/
|   |-- registry.py                immutable lazy backend factories
|   |-- nemo/
|   |   |-- config.py              typed NFA runtime/model configuration
|   |   |-- manifest.py            reference/output manifest parsing
|   |   |-- runner.py              isolated NFA execution
|   |   |-- ctm.py                 CTM parsing/mapping
|   |   `-- backend.py             neutral NFA adapter
|   |-- mfa/
|   |   |-- config.py              typed MFA runtime/model configuration
|   |   |-- corpus.py              temporary corpus/label construction
|   |   |-- runner.py              isolated MFA execution
|   |   |-- textgrid.py            TextGrid parsing/mapping
|   |   `-- backend.py             neutral MFA adapter
|   `-- whisperx/
|       |-- config.py              typed worker/runtime/model configuration
|       |-- segments.py            full-duration reference segments
|       |-- provider.py            isolated-worker transport
|       |-- worker.py              lazy heavyweight native runtime
|       |-- parser.py              native word parsing/mapping
|       `-- backend.py             neutral WhisperX adapter
`-- cli/
    |-- main.py                    command dispatch and typed-error exit status
    |-- annotate.py                annotation argument adapter
    |-- benchmark.py               benchmark argument adapter
    `-- readiness.py               readiness catalog/composition adapter
```

Empty package-marker `__init__.py` files are deliberately zero bytes. Each non-empty
implementation module has a focused test module and meets its individual line-coverage
gate.

## Module Responsibilities

| Area | Owns | Does not own |
|---|---|---|
| `core` | Neutral values, normalization/reconciliation, validation, extraction, readiness rules | CSV, subprocesses, native formats, model loading |
| `data` | Metadata/audio/cohort input and source preservation | Backend selection or metrics |
| `application` | Reuse-case orchestration and publication policy | Native parsing or benchmark math |
| `benchmarking` | Deterministic metrics and report rendering | Backend execution or native result dictionaries |
| `runtime` | Process, artifact, receipt, and evidence integrity primitives | Application row loops or model semantics |
| `backends.*` | Config, native inputs/outputs, execution, parsing, mapping | Metadata orchestration or sibling backend imports |
| `cli` | Argument parsing, backend composition, presentation | Alignment, CSV loops, metric computation |

## Shared Alignment Contract

```python
result = backend.align(request)
outcomes = backend.align_batch(requests)
```

`AlignmentRequest` contains:

- caller/request identity;
- resolved WAV path;
- exact authoritative transcript;
- probed WAV duration;
- immutable JSON-safe context.

A successful `AlignmentResult` contains one `WordAlignment` per original lexical word,
plus provenance, diagnostics, and measured runtime. A failure is represented by
`AlignmentFailure`; it contains no timing fields. `AlignmentOutcome` enforces exactly
one result/failure variant.

The root package exports `AlignmentBackend`, neutral request/result/failure/provenance
models, annotation services, and benchmark services. Custom backends can implement the
protocol without importing a concrete adapter.

## Transcript and Alignment Flow

```text
source metadata
    |
    +-- immutable en_transcript
    |
    +-- resolve <audio_dir>/<audio_id>.wav and probe duration
    v
lexical preflight and original-character mapping
    v
backend-specific reversible reference representation
    v
native model/process execution
    v
backend-local native output parsing
    v
shared monotonic reconciliation and validation
    v
WordAlignment / AlignmentFailure
    v
first/last extraction or benchmark observation
```

Backends may normalize punctuation, case, numbers, abbreviations, or native token
units internally. Every successful native token must map deterministically to the
original transcript's lexical words. Shared validation rejects incomplete coverage,
non-finite/negative/reversed/overlapping timing, and out-of-audio values. It never
clamps, interpolates, extrapolates, or invents a timestamp.

## Native Boundary Containment

| Backend | Native structures contained locally | External boundary |
|---|---|---|
| NeMo | Reference/output manifests and word CTM rows | Isolated NFA process |
| MFA | Temporary WAV/LAB corpus and TextGrid tiers | Isolated Conda MFA process |
| WhisperX | Reference segments and word-segment dictionaries | Isolated heavyweight worker |

Only neutral models, diagnostics, and provenance cross these boundaries. Parent-process
imports do not require NeMo, MFA, WhisperX, Torch, or Torchaudio.

## Annotation Publication

`application.annotate` prepares all source rows, invokes one injected backend, validates
and merges outcomes in source order, extracts six boundary columns, and atomically
publishes the enriched CSV. Continue-on-error preserves failed source rows with blank
alignment cells and structured diagnostics. Strict mode publishes no annotated output
after a row failure.

## Benchmark Composition

`application.benchmark` constructs one immutable ordered request tuple and supplies it
to every injected backend. It joins sparse manual gold by `(line_id, audio_id)`, scores
available first-word text/end fields, and compares successful first/last boundaries
pairwise. Reports are backend-neutral; native artifact paths appear only as opaque
references.

## Offline Readiness and Evidence

The readiness extension retains the same backend isolation:

```text
metadata + frozen boundaries + WAVs
    |
    v
cohort/hash verification
    |
    +-- verified full-word evidence capsules --> reuse audit
    |
    `-- missing/rejected case-backend pairs --> injected AlignmentBackend
                                                  |
                                                  v
                                      full-word evidence verification
                                                  |
                                                  v
                                  boundary mapping + consensus/reporting
```

`core.readiness` owns mapping/extraction/consensus, `data.readiness` owns cohort
verification, `runtime.alignment_evidence` owns capsule integrity, and
`application.readiness` orchestrates reuse/execution. `application.materialize`
materializes deterministic full-word timing from accepted immutable evidence without
importing or executing an aligner.

## Optional Dependency and Registry Behavior

`BackendRegistry` stores immutable dotted factories. Construction imports only the
selected backend module and validates the result against `AlignmentBackend`. An absent
optional runtime raises an actionable `BackendUnavailableError` only when selected.
One backend's absence does not affect backend-neutral imports or sibling construction.

## Architecture Verification

Release gates enforce:

- AST/import-boundary tests for prohibited directions and heavyweight imports;
- seven Import Linter contracts;
- backend sibling isolation;
- public-library tests with fake/injected backends;
- optional dependency absence;
- focused implementation-to-test mapping;
- per-file and aggregate coverage.

The current observed gate is documented in [`docs/TESTING.md`](docs/TESTING.md).

## Evolution Rules

Adding a backend should require a config, native input/execution/parser adapter, and
focused tests. It should not require changes to metadata I/O, common normalization,
first/last extraction, benchmark metrics, or the CLI beyond registry composition.
Shared helpers are introduced only when semantics are genuinely identical.