| # Multimodal architecture & growth |
|
|
| **Version 1.2 · July 2026** |
|
|
| ## Summary |
|
|
| Arriella supports multimodal work in **two different ways**: |
|
|
| 1. **Routed vision (core text tiers)** — image paths → Gemma3 description → Scout / Growth / Flagship / Ascension |
| 2. **Native Omni (Grapevine extension)** — Qwen2.5-Omni-3B student with vision/audio (and pre-export video) — [grapevine.md](../guides/grapevine.md) |
|
|
| Core text tiers also support: |
|
|
| - **Thinking** — Ollama `--think` ↔ Arriella `/think`…`/answer` (or `<scratchpad>` / `<answer>`); Ascension uses DeepSeek-native think |
| - **Post-deploy growth** — artifact ingest + organic LoRA grow |
|
|
| Certification notes and older wording live in [`../archive/2026-07/FLEET_MULTIMODAL_PAPER.md`](../archive/2026-07/FLEET_MULTIMODAL_PAPER.md). This page is the maintained version. |
|
|
| ## Architecture A — routed (core text) |
|
|
| ``` |
| User image ──► Gemma3 (Ollama) ──► categorized description |
| │ |
| User prompt ──► Arriella text tier |
| think=true → scratchpad / thinking channel |
| answer body → user |
| ``` |
|
|
| Text tiers **never** see raw pixels on this workstation’s VRAM budget. Vision is behavior absorbed via `vision_teacher_anchor` distill grows — not by stuffing a VLM into 0.5–1.8B weights. |
|
|
| Thinking is absorbed similarly from family-matched teachers (`qwen3:0.6b`, `lfm2.5-thinking:1.2b`, `deepseek-r1:1.5b`, etc.). |
|
|
| ## Architecture B — native Omni (Grapevine) |
|
|
| ``` |
| User text / image / audio ──► Qwen2.5-Omni-3B (+ Arriella LoRA + mmproj) |
| ↓ |
| Ollama arriella-grapevine |
| ``` |
|
|
| Grapevine is a **fleet extension**, not a fifth core text model. Inkling is a capability target only. Details and smoke vs Flagship: [grapevine.md](../guides/grapevine.md), root [`grapevine.md`](../../grapevine.md). |
|
|
| ## Growth paths |
|
|
| | Path | Script / entry | Use | |
| |------|----------------|-----| |
| | Artifact ingest | `fleet_artifact_ingest.py` | `ollama:` / `lora:` / `hf:` teachers | |
| | Organic grow | `fleet_organic.py`, `fleet_grow.py` | Datasets / multimodal anchors | |
| | Grapevine repair | `prepare_grapevine_*`, `train_arriella_inkling_local.py` | Omni identity / deployment repair | |
|
|
| Example (text tier): |
|
|
| ```powershell |
| .\.venv\Scripts\python.exe scripts\fleet_organic.py --id arriella-growth --multimodal |
| ``` |
|
|
| ## Formats |
|
|
| See [../guides/reasoning-format.md](../guides/reasoning-format.md). |
|
|
