unity setup guide

UNITY_SETUP.md

@@ -0,0 +1,103 @@
+# XROS Shell — Unity (Quest 3) Setup Guide
+
+This guide assembles the Unity frontend for the spatial OS shell. The backend
+(Python) runs on your computer; the Quest 3 app is a thin client that renders
+spatial panels and hosts the terminal.
+
+## Architecture
+
+```
+Quest 3 (Unity)                          Your computer (Python)
+┌─────────────────────────┐              ┌──────────────────────────┐
+│ WorldPanelManager       │◄──spawn/move─┤  xros_server.py         │
+│  (spatial panels)       │   /remove     │   ├─ WorldState         │
+│ SpatialShellTerminal    │──command─────►│   └─ SpatialShell       │
+│  (terminal panel)       │◄──output──────┤      (spawn/move/ls/gen)│
+│ XrosClient (WebSocket)  │              │                          │
+└─────────────────────────┘              └──────────────────────────┘
+```
+
+## 1. Prerequisites
+
+- **Unity 2022.3 LTS** (or newer) with **Android Build Support** module.
+- **Meta XR All-in-One SDK** (or Core SDK) — via the Unity Package Manager.
+- **Quest 3** in **Developer Mode** (Meta Horizon app → Settings → Developer).
+- **Python 3.11+** on your computer, with `websockets` installed
+  (`pip install websockets`).
+
+## 2. Create the project
+
+1. New project → **3D (URP)** template.
+2. **File → Build Settings → Android → Switch Platform**.
+3. **Edit → Project Settings → Player → Other Settings**:
+   - Set **Minimum API Level** to 29+ (Quest 3 requires 32).
+   - Set **Scripting Backend** to IL2CPP, **Target Architectures** ARM64.
+4. **Package Manager** → add **Newtonsoft Json**
+   (`com.unity.nuget.newtonsoft-json`) — required by `XrosClient.cs`.
+5. **Package Manager** → add **TextMeshPro** (import TMP Essentials when prompted).
+
+## 3. Import Meta XR SDK
+
+1. **Package Manager → Add package by name**:
+   `com.meta.xr.sdk.all` (All-in-One) — or `com.meta.xr.sdk.core` for the minimum.
+2. **Edit → Project Settings → XR Plug-in Management** → install **OpenXR**,
+   enable it for Android, and add **Meta Quest Support**.
+3. **Edit → Project Settings → Oculus** → enable **Passthrough** and
+   **Hand Tracking** (or use the Meta XR setup wizard).
+
+## 4. Build the scene
+
+1. Delete the default camera. Add **OVRCameraRig** (Meta XR → OVRCameraRig).
+2. Add an **OVRPassthroughLayer** to the rig's CenterEyeAnchor for passthrough.
+3. Add **OVRHand** components (left + right) to the rig for hand tracking.
+4. Create an empty GameObject **"XROS"** and add the three scripts:
+   - `XrosClient.cs`
+   - `WorldPanelManager.cs`
+   - `SpatialShellTerminal.cs`
+
+## 5. Terminal panel UI
+
+1. Under the rig, create a **Canvas** (World Space), size ~0.8 × 0.5 m,
+   positioned ~1.5 m in front of the user.
+2. Add a **TMP_InputField** (the command line) and a **TMP_Text** (the output
+   log) as children.
+3. On `SpatialShellTerminal`, assign `client` (the XrosClient), `input`, and
+   `output`. Wire the input field's **OnSubmit** to `Submit()`.
+
+## 6. Panel prefab
+
+1. Create a **Quad** (or a small World-Space Canvas) with a **TMP_Text** child
+   named "Title".
+2. Save it as a prefab and assign it to `WorldPanelManager.panelPrefab`.
+
+## 7. Point the client at your backend
+
+1. Find your computer's **LAN IP** (`ipconfig` / `ifconfig`).
+2. On `XrosClient`, set `serverUrl` to `ws://<your-lan-ip>:8765`.
+3. Start the backend on your computer:
+   `python xros_server.py --host 0.0.0.0 --port 8765`
+4. Ensure the Quest and your computer are on the **same Wi-Fi network**.
+
+## 8. Build & run
+
+1. **File → Build Settings → Build And Run** with the Quest connected via USB
+   (or use **Quest Link** / **Air Link** for fast iteration).
+2. Put on the headset. You should see passthrough, the terminal panel, and
+   your hands.
+3. Type in the terminal:
+   - `help`
+   - `spawn panel --title "Hello world"`
+   - `gen a clock for the wall`
+   - `ls`
+
+A panel should appear in the room for each `spawn`/`gen`, driven by the shared
+world model on your computer.
+
+## Troubleshooting
+
+- **No connection**: check the LAN IP, that the backend is running with
+  `--host 0.0.0.0`, and that both devices are on the same network.
+- **No passthrough**: enable Passthrough in Project Settings → Oculus, and add
+  the OVRPassthroughLayer.
+- **No hands**: enable Hand Tracking and add OVRHand components.
+- **JSON errors**: confirm Newtonsoft Json is installed in the Package Manager.