# XROS Shell — a spatial OS for the Meta Quest 3

A working prototype of a **spatial operating system shell** combining three
elements, inspired by the DiOS paper (*An Extended Reality Operating System for
the Metaverse*, arXiv:2201.03256):

1. **Spatial computing** — panels anchored in your real room (passthrough),
   driven by a shared world model rather than per-app state.
2. **A terminal** — a command line you use *inside* VR, as the primary interface.
3. **Generative UI** — describe what you want in natural language and the
   system generates a panel (currently a rule-based stub; an LLM can be wired
   in later).

## Architecture

```
Quest 3 (Unity)                          Your computer (Python)
┌─────────────────────────┐              ┌──────────────────────────┐
│ WorldPanelManager       │◄──spawn/move─┤  xros_server.py         │
│  (spatial panels)       │   /remove     │   ├─ WorldState         │
│ SpatialShellTerminal    │──command─────►│   └─ SpatialShell       │
│  (terminal panel)       │◄──output──────│      (spawn/move/ls/gen)│
│ XrosClient (WebSocket)  │              │                          │
└─────────────────────────┘              └──────────────────────────┘
```

The backend is the "OS brain": a shared world-state store plus a spatial shell
(command interpreter). The Quest app is a thin client that renders the world
and hosts the terminal.

## Backend (Python)

| File | Role |
|---|---|
| `xros_world.py` | World-state store: scene graph + object registry + event bus |
| `xros_shell.py` | Spatial shell: command interpreter (`spawn`, `move`, `ls`, `gen`, `clear`) |
| `xros_server.py` | WebSocket server exposing the world-state + shell API |
| `xros_client.py` | Python client library + demo |
| `test_xros.py` | End-to-end tests (unit + integration) |

Run it:

```bash
pip install -r requirements.txt
python xros_server.py --host 0.0.0.0 --port 8765   # serve on your LAN
python test_xros.py                                 # verify it works
```

Shell commands: `help`, `ls`, `spawn <type> [x y z] [--title T] [--color C]`,
`move <id> x y z`, `rm <id>`, `clear`, `gen <description>`.

## Frontend (Unity, Quest 3)

| File | Role |
|---|---|
| `XrosClient.cs` | WebSocket client (Newtonsoft.Json, no other packages) |
| `SpatialShellTerminal.cs` | Terminal panel: command input + output log |
| `WorldPanelManager.cs` | Renders world events as spatial panels |
| `UNITY_SETUP.md` | Step-by-step Unity + Meta XR SDK setup and build guide |

See **UNITY_SETUP.md** for the full assembly instructions (project setup, Meta
XR SDK, passthrough, hand tracking, and building to the Quest 3).

## Roadmap

- **Phase 1 (done):** shared world-state service + spatial shell + terminal.
- **Phase 2:** CV as a system service (shared scene understanding).
- **Phase 3:** agentic interface layer (AIOS-style scheduling/context).
- **Phase 4:** spatial developer experience (Code Cubes / structured editing).
- **Phase 5:** privacy/security at the OS layer (capability-based access).