Project Overview — read this first, every session
This document exists to formally introduce this workspace to any model operating in it — chat or autonomous builder — so the mission, the architecture, and the operating rules are established before any task begins.
The mission
NOBILITY POSH FRAMEWORK — building exceptional 2D-to-4D humanized companions & colleagues.
Miranda-Engine is the harness — the lab, not a pipeline. This is the single most important architectural fact about this project and it must never get flattened into "Miranda is a fixed ASR→LLM→TTS→render chain." She isn't. Miranda is workflow-agnostic: the substrate (shared-memory IPC bus, node contracts, telemetry, ephemeral session isolation) that any number of candidate pipeline configurations run inside, each independently measured against the same 2D control image ("EVE"). A quad-test — four parallel pipeline variants (different ASR models, different TTS engines, different rendering approaches), each in its own isolated harness session, each producing its own EVE render, scored head-to-head against the same Instant Presence Standard — is a first-class capability this harness must support, not a stretch goal. Design every piece of Miranda-Engine with "this needs to run N times in parallel, independently, comparably" in mind.
Miranda's larger claim: she can take the description of any pipeline — from an AWS reference architecture to a theoretical research paper — and engineer it into a running, measured, scored implementation without rebuilding the harness. Pipelines come and go. The harness is permanent.
Pipeline 1: AWS-native (batteries-included — first live test)
Full spec: .kiro/steering/pipeline-1-aws-native.md — read that file for the node mapping, SDK names, Polly viseme→BlendshapeFrame adapter spec, and credential access instructions.
The node labels in THE VANITY's left pane (Riva ASR / Nemotron Agent / Hive TTS / Audio2Face-3D / AnimGraph / Omniverse Stream) are role labels — they describe the class of work at each node position, not literal product deployments. For Pipeline 1, each role maps to an AWS-managed service Kiro already has direct access to via the project's AWS Bedrock credits:
| Node role label | Pipeline 1 AWS service | What it does |
|---|---|---|
| Riva ASR | Amazon Transcribe Streaming | Real-time WebSocket STT, managed, no GPU |
| Nemotron Agent | Amazon Bedrock (Nova Pro or Claude Haiku) | Converse API — model-agnostic routing and response generation |
| Hive TTS | Amazon Polly Neural TTS | 60+ neural voices, billed per character, zero standing cost |
| Audio2Face-3D | Amazon Polly Speech Marks (visemes) | 22 viseme types time-aligned to audio — populates BlendshapeFrame on the IPC bus |
| AnimGraph | Amazon Sumerian Hosts SDK (aws-samples/amazon-sumerian-hosts) |
Pre-built character rig + Polly lip-sync, Three.js, works in browser today with zero GPU |
| Omniverse Stream | Amazon Kinesis Video Streams WebRTC | Managed WebRTC signaling, no signaling server to build |
Why Pipeline 1 before anything else: it proves the IPC bus, node contracts, and THE VANITY GUI work end-to-end with real signals from real AWS services — before we introduce research-grade components (parakeet.cpp, SIMD kinematics, Gaussian-splat rendering) in later pipelines. Pipeline 1's latency numbers are the baseline. Pipeline 2 must beat them in a head-to-head quad-test.
The node labels in THE VANITY do NOT need to be renamed — they describe the role permanently. The AWS service is just what fills that role in Pipeline 1. When Pipeline 2 uses parakeet.cpp instead of Transcribe, the "Riva ASR" node slot still exists; a different implementation is plugged into it.
What's already built in this workspace
- The Rust workspace skeleton — 6 crates (
miranda-core/ipc/audio/nodes/supervisor/transport),cargo buildverified clean. - The prior-attempt client (
client-apps/web,client-services/ace-controller) — real, working Vite+React+TS code carried forward from the first attempt (eve-ecc), not a from-scratch rebuild. This is THE VANITY's frontend — the React Flow node graph (left pane) and EVE's static 2D reference image (right pane). - The Instant Presence Standard (
eve-ecc-docs/INSTANT-PRESENCE-STANDARD.md) — the No Loop Video Protocol: instantly alive from frame one, waist-up mid-frame composition, waving as the opening gesture, choreographed micromovement, zero loop-video anywhere. - 5 Work Order specs (
.kiro/specs/wo1-*throughwo5-*) — each with EARS-notation requirements, a design doc, and a CAT-tagged task list. - The CAT-5 Model Routing Protocol (
.kiro/steering/model-routing-protocol.md) — see below. - Pipeline 1 steering doc (
.kiro/steering/pipeline-1-aws-native.md) — the AWS-native first pipeline: full node mapping, Polly viseme adapter, Sumerian Hosts setup, credential access. - 4 global Kiro skills —
nobility-posh-framework(this project's master science/architecture reference),live-avatar-expert(real-time avatar generation research),aws-pipeline-architect(AWS deployment + Podman container strategy),llamacpp-huggingface-expert(local GGUF/llama.cpp tooling).
The CAT-5 Model Routing Protocol — mandatory, global, no deviation
Every task in every Work Order is tagged [CAT 1] through [CAT 5]. This governs which model handles it — deliberately mixed-provider, Claude reserved for CAT 4-5 only:
| CAT | Model | Why |
|---|---|---|
| 1 | Qwen3 Coder Next | Coding-specialized, cheapest tier — mechanical, zero reasoning required |
| 2 | Amazon Nova Lite | Light reasoning, well-known patterns |
| 3 | Amazon Nova Pro | The workhorse — most tasks land here, strong reasoning-per-dollar |
| 4 | Claude Sonnet 5, escalate to Opus 5 after 2 real failed verification attempts | Real risk, bounded scope — first tier where Claude is required |
| 5 | Claude Opus 5 only, no exceptions | Silent-failure-risk engineering: lock-free concurrency, unsafe memory/ABI correctness, real-time SIMD math, novel shader algorithms |
Before starting any task, check its CAT tag and switch to the required model if the active session doesn't match. See the full protocol doc for the exact escalation-notice wording. Run node scripts/cat-router-check.mjs at the start of any session to see the current backlog by tier.
28 of 36 total tasks (CAT 1-3) never touch Claude at all. Claude is confined to the 5 CAT 4 tasks and 3 CAT 5 tasks — roughly 22% of the build, spent on the highest-value engineering. This is a deliberate cost/capability allocation, not a default — don't reach for Claude out of convenience on a CAT 1-3 task.
Non-negotiable operating rules (full detail in .kiro/steering/build-standards.md)
- No claimed progress without real command-output evidence.
- No simulated inference, ever — say plainly when something isn't wired up yet.
- Reuse before rebuild — check
client-apps/weband the existing crate stubs first. - GPU cost discipline — never leave a GPU instance idle.
- Podman hybrid placement — containerize WO-2/4/5, keep WO-1's IPC bus and GPU rendering bare-metal.
- Ephemeral session isolation — every workflow test runs in a fresh, disposable Podman container.
Where to start
Work Order 1 (.kiro/specs/wo1-workspace-ipc-backbone/) is the foundation everything else depends on. Start there. WO-1's IPC bus is pipeline-agnostic — it serves Pipeline 1 and all future pipelines without change. Build it once.
After WO-1: read .kiro/steering/pipeline-1-aws-native.md to understand which node implementations Pipeline 1 uses for WO-2 through WO-5 — they are all simpler than the full Work Order specs (AWS managed services instead of custom Rust/C++) and can run in parallel with the deeper WO-2/WO-3/WO-4/WO-5 builds.