docs/MODULAR_ARCHITECTURE_CONTRACT.md

Modular and Reusable Architecture Contract

Purpose

This document is a mandatory architecture contract for the speech-text-data-aligners implementation. The project must be usable both as a Python library and through a CLI. NeMo Forced Aligner (NFA), Montreal Forced Aligner (MFA), and WhisperX are interchangeable alignment providers behind a shared application contract.

Modularity is an acceptance requirement, not a style preference.

Architectural principles

1. Dependency inversion: application/core code depends on alignment protocols and shared data models, never on concrete aligners.
2. Backend isolation: NeMo, MFA, and WhisperX implementations do not import one another.
3. Single responsibility: model types, normalization, metadata I/O, validation, backend-native artifact construction/parsing, orchestration, benchmarking, and CLI composition live in separate modules/packages.
4. Library first: reusable Python APIs implement behavior; the CLI is a thin composition/presentation layer.
5. Deterministic core: shared parsing, normalization, mapping, validation, extraction, and metrics must be unit-testable without GPU, model weights, network, or external aligner executables.
6. Native boundary containment: backend-specific structures such as NeMo manifests/CTMs, MFA corpus/TextGrids, and WhisperX result dictionaries are converted at the backend boundary and do not leak into the core API.
7. Explicit configuration: backend configuration is represented by typed configuration objects instead of scattered environment-variable/global reads.
8. Structured failures/provenance: common result, diagnostic, and provenance types cross package boundaries; raw backend exceptions should be translated into project exceptions/diagnostics at the adapter boundary.
9. Replaceability: adding a fourth aligner later must not require changes to transcript normalization, metadata I/O, benchmark metrics, or first/last-word extraction except for explicit protocol evolution.
10. No hidden singleton/global model state: heavyweight clients/models/process runners must be injectable or owned by a backend instance with a documented lifecycle.

Required package layering

Use a src/ layout. The exact filenames may be refined during Stage 0, but the accepted design must preserve these layers and dependency directions.

src/speech_text_data_aligners/
├── __init__.py
├── core/
│   ├── models.py              # backend-neutral value/result/provenance types
│   ├── protocols.py           # AlignmentBackend and supporting protocols
│   ├── errors.py              # project exception taxonomy
│   ├── normalization.py       # lexical extraction + reversible mappings
│   ├── validation.py          # alignment/timestamp invariants
│   └── extraction.py          # first/last lexical boundary selection
├── data/
│   ├── metadata.py            # CSV read/write/preservation contract
│   └── audio.py               # audio_id/path resolution + lightweight metadata
├── application/
│   ├── annotate.py            # reusable annotation use case/service
│   └── benchmark.py           # reusable backend evaluation/comparison use case
├── benchmarking/
│   ├── metrics.py             # deterministic gold/pairwise metrics
│   └── reports.py             # machine/human-readable reporting
├── runtime/
│   ├── process.py             # injectable subprocess abstraction
│   └── receipts.py            # environment/model/run provenance capture
├── backends/
│   ├── nemo/
│   │   ├── backend.py         # AlignmentBackend adapter/orchestration
│   │   ├── config.py          # typed NFA config
│   │   ├── manifest.py        # native input artifact construction
│   │   └── ctm.py             # native CTM parser/mapping
│   ├── mfa/
│   │   ├── backend.py
│   │   ├── config.py
│   │   ├── corpus.py          # temporary corpus/native inputs
│   │   └── textgrid.py        # word-tier parser/mapping
│   └── whisperx/
│       ├── backend.py
│       ├── config.py
│       ├── provider.py        # injectable model/API call boundary
│       └── parser.py          # native result conversion/mapping
└── cli/
    ├── main.py                # composition root only
    ├── annotate.py            # argument/presentation adapter
    └── benchmark.py           # argument/presentation adapter

Do not create files solely to imitate this tree. A smaller module set is acceptable when a file would otherwise be trivial, but every retained module must have one clear responsibility and the layer boundaries below remain mandatory. Conversely, do not collapse unrelated concerns into monolithic backend or CLI modules merely to reduce file count.

Mandatory dependency direction

Allowed high-level direction:

cli ───────────────► application ───────────────► core
 │                       │                         ▲
 │                       ├────────► data ──────────┤
 │                       ├────────► benchmarking ──┤
 │                       └────────► runtime ───────┤
 │
 └── composition only ─► concrete backends ──────► core/runtime

Rules:

• core MUST NOT import application, cli, or any concrete backend.
• data MUST NOT import a concrete backend or cli.
• benchmarking MUST operate on backend-neutral records/results and MUST NOT parse native CTM/TextGrid/WhisperX structures.
• application MUST accept AlignmentBackend instances/protocols through dependency injection; it MUST NOT hard-code NeMo/MFA/WhisperX behavior.
• concrete backends MAY depend on core and runtime; they MUST NOT import another concrete backend.
• cli is the composition root and MAY select/register concrete backends, but business logic must stay out of it.
• backend registry/factory behavior, if used, must use lazy imports or explicit registration so importing the core library does not require every heavyweight optional dependency.
• optional backend dependencies must not be imported at module import time from backend-neutral packages.

Public reusable API

The implementation must provide documented Python-callable use cases equivalent in capability to:

alignments = backend.align(audio_path=audio_path, transcript=transcript)

result = annotate_metadata(
    metadata_csv=metadata_csv,
    audio_dir=audio_dir,
    output_csv=output_csv,
    backend=backend,
    strict=False,
)

report = benchmark_backend(
    backend=backend,
    metadata_csv=metadata_csv,
    reference_csv=reference_csv,
    audio_dir=audio_dir,
)

Names may differ, but users must not be forced to invoke a CLI subprocess to use the framework programmatically.

Backend contract

The common protocol must accept a known transcript plus audio and return ordered backend-neutral word alignments with structured provenance/diagnostics. It must not expose native CTM/TextGrid/WhisperX dictionaries to callers.

Backends must separate at least these responsibilities when non-trivial:

• typed backend configuration;
• native input preparation;
• execution/model invocation;
• native output parsing;
• mapping/validation into backend-neutral records.

Native parsers/builders should be deterministic functions/classes and independently unit-testable.

Transcript normalization ownership

Common lexical tokenization and original-text identity live in core. Backend-specific normalization policies may extend/adapt the common mapping but must not duplicate the entire metadata pipeline.

Every backend result must map back to original-facing lexical word text deterministically. The original en_transcript remains immutable.

CLI contract

The CLI must be a thin adapter:

1. parse/validate arguments;
2. construct backend/configuration objects;
3. call reusable application services;
4. render exit status/results.

The CLI must not contain CTM/TextGrid parsing, transcript normalization algorithms, benchmark math, CSV enrichment loops, model-loading algorithms, or backend-specific alignment logic beyond composition.

Optional dependency contract

• Core library import and deterministic unit tests must work without NeMo, MFA, WhisperX, model weights, or MFA executables installed.
• Backend packages must fail with actionable dependency errors only when that backend is selected/constructed/executed.
• MFA runtime should remain environment-isolated by default.
• NeMo and WhisperX may share an environment only after demonstrated compatibility.

Architecture verification

Implement a deterministic architecture gate using import-linter, AST/import inspection tests, or an equivalent mechanism. It must detect at least:

• core importing concrete backends;
• one backend importing another backend;
• data/benchmarking importing concrete backends;
• backend-neutral modules importing optional heavy backend packages;
• business logic placed only in CLI without an application/library entry point.

Architecture tests are part of ordinary deterministic CI and must not require external aligners.

Test mirroring and coverage

Every project-owned non-test Python implementation file must have a corresponding focused test module unless it is a deliberately empty package marker. This includes CLI, application services, runtime wrappers, native parsers, and backend adapters.

Each in-scope implementation file must achieve at least 90% line coverage individually. Prefer >=95% for deterministic shared/core modules.

Tests should mirror package boundaries, for example:

tests/unit/core/test_normalization.py
tests/unit/data/test_metadata.py
tests/unit/application/test_annotate.py
tests/unit/backends/nemo/test_ctm.py
tests/unit/backends/mfa/test_textgrid.py
tests/unit/backends/whisperx/test_parser.py
tests/unit/cli/test_annotate.py
tests/architecture/test_import_boundaries.py
tests/integration/nemo/test_nemo_real.py
tests/integration/mfa/test_mfa_real.py
tests/integration/whisperx/test_whisperx_real.py

Do not count skipped heavyweight integration paths as a substitute for deterministic unit coverage.

Maintainability constraints

During review, flag and refactor:

• duplicated CSV/normalization/validation logic across backends;
• native backend formats escaping adapter packages;
• circular imports;
• imports of heavyweight optional dependencies from core modules;
• global mutable registries/models without lifecycle control;
• subprocess invocation duplicated across backend modules when a common runtime abstraction is sufficient;
• large mixed-responsibility modules that combine configuration, execution, parsing, CSV orchestration, and CLI concerns;
• backend conditionals such as if backend == "nemo" scattered through application/core logic.

Architecture acceptance evidence

The final report must contain:

1. final package tree;
2. module responsibility table;
3. dependency-direction/architecture-test results;
4. public library API examples;
5. mapping of every implementation Python file to its focused test file and coverage percentage;
6. confirmation that importing the backend-neutral library succeeds without heavyweight optional backends installed;
7. confirmation that each backend can be disabled/absent without breaking the others;
8. any deviations from this contract with explicit rationale.