ARIA / ARIA_ARCHITECTURE.md
Malina
ARIA: event-driven multimodal agent (brain + Cloudflare backend + Messenger UI)
4131875
|
Raw History Blame Contribute Delete
6.3 kB

A newer version of the Gradio SDK is available: 6.32.0

Upgrade

ARIA β€” Architecture

ARIA (Adaptive Response & Intelligence Agent) is a Messenger-style, event-driven, multimodal conversational agent deployed under the NNLABS brand.

  • Root showcase: https://nnlabs.pl (landing β€” ARIA listed alongside NINA / IRIS / LUNA)
  • App: https://app.nnlabs.pl/aria (the chat)
  • Brain: https://nextneural-aria.hf.space (Gradio Space, vision-capable)

Locked decisions

Layer Choice
Brain Gradio HF Space + vision model, emits structured JSON bubbles
Realtime Durable Object (AriaConversation) + WebSocket; server-side idle alarm
Placement 4th persona, login-gated (reuses existing auth); own budget: 20 messages / 5h / user
Media Cloudflare R2 (aria-media bucket), signed URLs passed to the brain
Voice Polish-first / auto-detect; witty young-adult friend, no emojis, rare "XD/lmao"

Topology

 Browser (aria.html + aria-chat.js)
   β”‚  HTTPS (REST: history, upload-url)   β”‚  WSS (live events/bubbles)
   β–Ό                                      β–Ό
 Cloudflare Pages  app.nnlabs.pl  (APP_WEBSITE/_worker.js)
   β€’ validates nnl_session cookie + usage window (REUSED)
   β€’ /api/aria/*  β†’  service binding  β†’  aria-worker
                                          β”‚
                                          β–Ό
                          aria-worker (standard Worker)
                            β€’ routes REST + WS upgrade
                            β€’ binding: ARIA_DO  (Durable Object)
                            β€’ binding: ARIA_MEDIA (R2)
                            β€’ binding: DB (D1, shared app-nnlabs-db)
                                          β”‚
                          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                          β–Ό                               β–Ό
              AriaConversation DO                  nextneural-aria.hf.space
              β€’ per-conversation state             (Gradio queue/join + SSE)
              β€’ event log + clustering             β€’ vision + reasoning
              β€’ idle alarm (10s) β†’ proactive       β€’ returns JSON bubbles
              β€’ fan-out bubbles over WS

Why a separate Worker: Cloudflare Pages Advanced Mode cannot define a Durable Object. The DO + R2 live in a standalone Worker; the existing Pages _worker.js forwards /api/aria/* to it via a service binding after enforcing auth. NINA/IRIS/LUNA code is never touched.

Event-driven model

Every input is an event ({ id, ts, type, payload }):

  • message β€” text. payload { text }
  • image β€” image upload. payload { key, url, mime, name }
  • file β€” generic file. payload { key, url, mime, name, size }
  • system_idle β€” emitted by the DO after 10s of user inactivity. payload { since }

Turn lifecycle (inside the DO)

  1. Client sends user event(s) over WS β†’ DO appends to the event log, persists to D1.
  2. DO clusters consecutive un-answered user events into one logical turn (debounced ~600ms so "i gruszki" sent right after "kup jabΕ‚ka" is one context).
  3. DO builds structured context (recent events + resolved topic) and calls the brain.
  4. Brain returns [{ bubble, target_event_id, type }].
  5. DO streams typing then each bubble over WS, persists bubbles to D1.
  6. DO (re)arms a 10s alarm. On fire it emits system_idle; the brain may return a proactive bubble (e.g. "co to?" for an image sent without text). Idle fires at most N times per turn to avoid nagging.

Bubble contract (brain β†’ DO β†’ client)

[
  { "bubble": "co to?", "target_event_id": "evt_123", "type": "proactive" },
  { "bubble": "jabΕ‚ka ogarniΔ™te", "target_event_id": "evt_456", "type": "response" },
  { "bubble": "gruszki teΕΌ dorzucΔ™", "target_event_id": "evt_457", "type": "follow_up" }
]

type: response (direct answer) Β· follow_up (continues a chained user msg like "i gruszki") Β· proactive (DO-initiated, usually after system_idle or a text-less image).

WebSocket protocol (DO ↔ client)

Client β†’ DO: { t: "event", event: {id,ts,type,payload} }, { t: "seen" }, { t: "ping" } DO β†’ client: { t: "typing", on: true|false }, { t: "bubble", bubble, target_event_id, kind }, { t: "error", v }, { t: "history", events, bubbles }, { t: "pong" }

Persistence (D1 β€” see cloudflare/db/migration_aria.sql)

  • aria_conversations β€” one row per chat (user-owned, themed).
  • aria_events β€” append-only user/system event log.
  • aria_bubbles β€” assistant bubbles with their target_event_id + type.

Usage budget

ARIA enforces 20 messages per rolling 5h window per user, tracked in its own aria_usage table (separate from the NINA/IRIS/LUNA model_usage pool β€” a messenger sends far more, shorter turns). The window starts at the first message and resets after 5h. Each user message (text, image, or file) costs one; system_idle and ARIA's own proactive bubbles never count. Enforcement lives in the Durable Object (aria-worker/src/usage.js) β€” the only layer that sees each message inside the socket. The DO pushes { t:"usage", remaining, resetAt } on connect and after each message, and { t:"limit", resetAt } when the budget is exhausted.

Media

Client uploads bytes via POST /api/aria/conversations/:id/upload (proxied to R2 by the worker), then sends an image/file event referencing the returned key. The DO mints a short-lived HMAC-signed GET URL to hand to the brain. An uploaded attachment is a message and so counts as one against the 20/5h budget.

Reuse map (from NNLABS.PL/WEBSITE)

  • Auth, sessions, usage window, rate-limit β†’ existing Pages _worker.js (unchanged).
  • Design tokens, .chat-bubble, .chat-typing, markdown renderer β†’ reused by aria.css / aria-chat.js.
  • Gradio queue/join + SSE proxy pattern β†’ reimplemented in aria-worker/orchestrator.js.
  • D1 database app-nnlabs-db β†’ shared; ARIA adds its own tables only.

Integration & safety

cloudflare/integrate.sh copies the new files additively into the live site (no overwrites; refuses if a target exists and differs). The Pages _worker.js change is a small, reviewed snippet (_worker.aria.js) added by hand, not an overwrite.