# Architecture

## One engine, many hosts

```
   crates/microquack-core        crates/microquack-llm
   (canonical: all sound          (the droid mind: tiny
    behavior)                      transformer inference)
        \                          /            |
         crates/microquack-wasm            trained in llm/
        /      (browser binding)           (PyTorch -> binary weights)
  crates/microquack-cli
  (render / validate / fit / schema)
           |
      apps/web (TypeScript: UI, canvas, WebAudio, 3D stage only)
           |
  microduck robot (native crates, 48 kHz PCM)   <- planned consumer
```

`microquack-core` contains everything that affects the sound: score types and
parsing, pitch curves, envelopes, oscillators and partials, ring modulation,
noise, the seeded PRNG, phrase grammar, the canonical semantic emotion
registry and voice-mode shaping, contour output for motion, and rendering —
both full-phrase (`render_score`) and incremental
blocks (`Engine::render_into`, which `render_score` itself uses, so offline
and streaming output are identical). It contains no WebAudio, WASM APIs,
device access, filesystem loading, browser code, or Python-specific code.

## The score contract

A score is a JSON list of syllables with snake_case fields (`f0_start`,
`amp_points`, ...), defined once as Serde types in `microquack-core`.
`schema/score.schema.json` is generated from those types (`pnpm run schema`);
nothing else defines the format. `schema/emotions.json` and
`apps/web/src/emotions.generated.ts` are likewise generated from the Rust
registry with `pnpm run emotions`. Python uses the JSON file for teacher-data
generation. Scores use seconds and hertz, so they are sample-rate
independent; the renderer owns all rate-specific behavior. The TypeScript app
treats scores as opaque JSON strings — it never reimplements or redeclares the
format.

## The chat and the droid mind

The visitor reads and writes English; the robot answers in droid speak with
an English gloss. `crates/microquack-llm` runs a ~1.3M-parameter
character-level transformer (trained from scratch in `llm/` on a synthetic
space-adventure corpus) with a KV cache and greedy decoding, compiled into
the same WASM module as the synth. Its output is `(emotion, reply)`; the
emotion drives phrase generation and the body-language gestures. Rust/
PyTorch parity is enforced by `cargo test -p microquack-llm` against the
committed model (`apps/web/public/assets/llm/droidmind.bin`) and test
vectors. The current model artifact supports its original eight emotions.
Rust validates that list against the canonical 73-state registry, then maps
each semantic state to one of 12 broad voice modes and four shared affect
values. The expanded 1,000-row teacher set is review data only; it does not
replace the production weights. Between conversations the robot utters an
idle thought every 10–15 seconds.

## The web app

`apps/web` owns UI, canvas, and WebAudio only:

- `src/wasm/` — generated wasm-bindgen output (`pnpm run build:wasm`).
- `robot.ts` — assembles the real microduck from its mesh library
  (`microduck.glb`) and kinematic tree (`kinematics.json`, MuJoCo
  convention), exposes head joints and a synthetic jaw hinge.
- `scene.ts` — the procedural night-plain world (three moons, stars, rocks)
  and the speech-driven articulation.
- `mind.ts` / `droidtext.ts` / `gestures.ts` — the chat brain wrapper,
  droid-speak captions, and generic body language driven by the Rust voice
  mode and affect values. TypeScript does not map the 73 semantic states.
- Short phrases render whole (`render_score` → `AudioBuffer`). Continuous
  synthesis, when it arrives, must run the `Engine` in an AudioWorklet —
  WASM alone does not prevent main-thread stalls.

## Research

Fitting is a CLI subcommand (`microquack fit`): it analyzes a real WAV
(pitch track, envelope, harmonics, noisiness) and emits a breakpoint score,
rendering every candidate through the same engine that ships. `research/`
holds the fitted corpus (`scores/fits.json`), pregenerated phrase scores,
and the frozen conformance fixtures.

## Conformance gate

`crates/microquack-core/tests/conformance.rs` renders all 90 frozen scores
(79 fitted + 6 phrases + 5 presets) against `research/fixtures/`: exact
sample counts, RMS/peak within 1e-6, and 21 full waveforms within 1e-4 per
sample. Streaming must equal offline rendering exactly, and phrase generation
must be seed-deterministic. Any DSP change that breaks these is a behavior
change and requires refreezing fixtures deliberately.

## Device integration (planned)

The Microduck repository consumes `microquack-core` at an exact version
(`microquack-core = "=x.y.z"`, or a pinned git revision until published) —
never vendored source. The device adapter feeds `Engine::render_into` into
its existing 48 kHz PCM path and reads `ControlFrame` values to animate the
physical beak and body. The 79 fitted scores must be revalidated at 48 kHz
before the robot ships them.

## Release model

One git tag should produce, all reporting the same version and commit:

- the `microquack-core` crate (device),
- the WASM/npm package (website),
- the CLI binary (render, validate, fit, schema, emotions),
- the score corpus and generated schemas.

## Roadmap

See [VISION.md](VISION.md): preset conversations, the droid-speak language
model, procedural full-body motion, and browser-side locomotion policies from
`microduck_rl` via ONNX Runtime Web.