docs/VISION.md

Vision

What microquack is for, where it is going, and the constraints that hold regardless of implementation detail.

The goal

microquack is the voice of the microduck robot. One engine — microquack-core (Rust) — is the single source of truth for every sound: the same crate renders on the website (WASM), in the fitting CLI, and eventually on the robot itself (native, 48 kHz PCM). TypeScript exists only for the web presentation. Nothing that affects the sound may exist twice.

The repository is also meant to be exemplary: conformance-tested ports, one score contract, strict quality gates, and a research trail (research/) that shows how the sounds were derived.

The experience

microduck talks by itself. Nobody selects presets and nobody has to type for it to speak. The site shows the robot in a rendered world; it wakes up, looks around, and babbles droid-speak continuously — mood-shaped phrases with pauses, like overhearing a small robot thinking out loud. Captions transcribe each utterance as droid text ("bweep brrrt boo?") with its mood.

The conversation layer is a chat: the visitor reads and writes English (preset chips or free text), and the robot answers in droid speak with an English gloss underneath. A very small embedded language model (in the spirit of minionese), trained in this repository and run in the browser, chooses a short English reply and an emotion; the engine turns that emotion into sound. Speech stays fully procedural — the model only chooses what to say and how it feels, never how it sounds. When nobody is talking to it, the robot says something on its own every 10–15 seconds.

Emotions are first-class. Rust defines 73 child-safe semantic states, from curiosity and joy to jealousy, shame, grief, and awe. It maps them to 12 broad voice modes and shared affect values. The English reply carries the exact meaning; the voice and real robot joints carry its broader tone and energy. The current eight-emotion model remains in production until the expanded teacher data is reviewed and a later training run is approved.

The stage

The site renders the real robot — assembled from microduck's mesh library and kinematic tree — on the bridge of a spaceship orbiting a planet: a window band with structural mullions looking out on space and moons, as realistic as browser rendering sensibly allows. Other microducks (the real roller variant, in the other real colourways) glide around the bridge doing chores at the crew consoles. The camera does not swivel: a fixed, chat-like framing on the robot's upper body with only the slowest natural drift. Assets are permissively licensed only (CC0 / public domain / CC-BY with credit); no copyrighted names or designs, ever. That also applies to audio: the corpus is fitted only from open-source droid sample packs — never from film audio.

Motion evolves in the same direction as speech:

1. Speech-driven articulation (current): jaw, head pitch/yaw driven by the engine's control frames; idle look-around.
2. Procedural full-body motion: weight shifts, steps, gestures synced to phrase moods.
3. Policy-driven motion: the trained microduck locomotion policies (from microduck_rl, as in the Microduck Sandbox) running in the browser via ONNX Runtime Web, blended with speech-driven gestures — the duck walks around its world while it talks.

Device parity

Whatever ships here must be directly usable on the robot:

• The robot consumes microquack-core as a versioned crate (exact version, or an immutable git revision until published) — never vendored source.
• 48 kHz is the canonical device output rate; scores are written in seconds and hertz so they are sample-rate independent, and the renderer owns all rate-specific behavior.
• Engine::render_into feeds the robot's PCM path in blocks; Engine::control_frame drives the physical beak and body the same way it drives the 3D model.

Non-negotiables

• One engine. Ports and hosts are bindings, not implementations.
• Behavior changes to the engine require updating the conformance fixtures deliberately, never silently.
• Everything audible is synthesized from a JSON score at runtime.
• No copyrighted audio or 3D assets; the corpus sources are the open-source packs credited in research/README.md.