| # 🤖 AirMouse Pro – Master Agent Specification (`AGENT.md`) | |
| Welcome to the definitive architecture, design, and operating manual for the **AirMouse Pro** Cyber-Physical System (CPS CA2 Assignment). | |
| This document serves as the primary reference for AI Agents, developers, and evaluators interacting with the AirMouse ecosystem. | |
| --- | |
| ## 🏛️ 1. Executive System Overview | |
| AirMouse Pro is an end-to-end, ultra-low latency wireless air-mouse system. It transforms an Android smartphone into a high-precision 3D spatial cursor controller and gesture input device. | |
| ```mermaid | |
| graph TD | |
| A["Android App (Kotlin)<br>code/android/"] -->|TCP 9090 / UDP 9093| B["Go PC Server<br>code/pc/airmouse_mcp_go/"] | |
| A -->|Trace Markers| P["Perfetto Tracing<br>trace_file.perfetto-trace"] | |
| B -->|OS Cursor Events| C["Host OS Desktop"] | |
| D["Python MCP Server (444 Tools)<br>airmouse_server_mcp.py"] -->|Admin REST 8081| B | |
| D -->|ADB Automation| A | |
| E["Antigravity AI Agent<br>.agents/ Workspace"] -->|FastMCP / CLI| D | |
| ``` | |
| ### Key Subsystems: | |
| 1. **Android Client (`code/android/`)**: Pure Kotlin IMU reader, 3-step calibration manager, Madgwick AHRS filter ($\beta=0.041$), gesture detection, and line-delimited JSON TCP socket client. | |
| 2. **Go Desktop Backend (`code/pc/airmouse_mcp_go/`)**: High-performance multi-protocol server (TCP 9090, WebSocket 9091, UDP Discovery 9092, UDP Stream 9093, REST Admin 8081) with Kalman predictive blending and power-law cursor acceleration. | |
| 3. **Python MCP Server (`airmouse_server_mcp.py`)**: Master 444-tool FastMCP server and CLI supporting an extended 5-lifecycle agent hook engine. | |
| 4. **Antigravity Workspace (`.agents/`)**: AI configuration hub featuring 22 domain-specific skills, system rules, dynamic memory, workflows, and environment settings. | |
| --- | |
| ## 🧮 2. Mathematical Models & Sensor Fusion | |
| ### 2.1 Madgwick AHRS Filter ($\beta = 0.041$) | |
| The orientation orientation quaternion $q = [q_1, q_2, q_3, q_4]^T$ is computed continuously using gradient descent optimization to fuse gyroscope ($\mathbf{\omega}$ in $\text{rad/s}$), accelerometer ($\mathbf{a}$ in $\text{m/s}^2$), and magnetometer ($\mathbf{m}$ in $\mu\text{T}$) inputs: | |
| $$\dot{q} = \frac{1}{2} q \otimes \mathbf{\omega} - \beta \frac{\nabla f}{\|\nabla f\|}$$ | |
| - **Gyroscope Units**: Standard Android $\text{rad/s}$ (never converted to degrees prior to filter update). | |
| - **Gain Parameter**: Fixed at $\beta = 0.041$ for optimal balance between gyro drift rejection and dynamic response. | |
| ### 2.2 Quaternion to Euler Angles Conversion | |
| To translate orientation into screen space coordinates: | |
| $$\begin{aligned} | |
| \text{Roll } (\phi) &= \operatorname{atan2}(2(q_1 q_2 + q_3 q_4), 1 - 2(q_2^2 + q_3^2)) \\ | |
| \text{Pitch } (\theta) &= \arcsin(2(q_1 q_3 - q_4 q_2)) \\ | |
| \text{Yaw } (\psi) &= \operatorname{atan2}(2(q_1 q_4 + q_2 q_3), 1 - 2(q_3^2 + q_4^2)) | |
| \end{aligned}$$ | |
| --- | |
| ## 🎯 3. 3-Step Sensor Calibration Protocol | |
| | Calibration Step | Sensor Target | Procedure | Mathematical Model | | |
| |---|---|---|---| | |
| | **Step 1: Gyroscope** | Gyro Bias Vector | Place phone on flat, vibration-free surface; collect **100 stationary samples**. | $\mathbf{b}_g = \frac{1}{100} \sum_{i=1}^{100} \mathbf{\omega}_i$ | | |
| | **Step 2: Magnetometer** | Hard/Soft-Iron Offsets | Perform figure-8 motion in 3D space; collect **200 samples**. | $\mathbf{V}_{\text{hard}} = \frac{\mathbf{m}_{\max} + \mathbf{m}_{\min}}{2}, \quad \mathbf{S}_{\text{soft}} = \frac{\text{avg\_range}}{\mathbf{m}_{\max} - \mathbf{m}_{\min}}$ | | |
| | **Step 3: Accelerometer** | Scale & Zero-g Bias | Place phone in 6 static orientations (Flat Up/Down, Left, Right, Top, Bottom). | Least-Squares Optimization to enforce $\|\mathbf{a}\| = 9.81 \text{ m/s}^2$ | | |
| --- | |
| ## 🖐️ 4. Motion Gesture Detection Engine | |
| - **Click (Left Click)**: Gyroscope Y-axis angular velocity $> 8.0 \text{ rad/s}$ with a $300\text{ ms}$ cooldown. | |
| - **Double Click**: Second left-click gesture detected within $400\text{ ms}$ of the first. | |
| - **Scroll Movement**: Accelerometer Y-axis acceleration $> 8.0 \text{ m/s}^2$ with a $2.0 \text{ m/s}^2$ debounce and $100\text{ ms}$ cooldown. | |
| - **Right Click**: Roll tilt angle $> 45^\circ$ held continuously for $\ge 500\text{ ms}$. | |
| --- | |
| ## 📡 5. Network Protocol & ACK Specification | |
| Messages are formatted as newline-delimited JSON (`\n`) transmitted over TCP port 9090 or UDP stream 9093. | |
| ### 5.1 Packet Types & Schemas | |
| - `hello`: Client registration handshake (`{"type":"hello","device_name":"Pixel 8","version":"2.0"}`). | |
| - `move`: Continuous delta orientation stream (`{"type":"move","dx":12.5,"dy":-4.2}`). | |
| - `click`: Left-click request (`{"type":"click","id":"req_101"}`). | |
| - `scroll`: Vertical scroll command (`{"type":"scroll","id":"req_102","delta":-3}`). | |
| - `ack`: Server acknowledgment (`{"type":"ack","id":"req_101"}`). | |
| ### 5.2 Reliable Retransmission Logic | |
| For `click`, `scroll`, and `control` message types: | |
| - Client starts a **500 ms countdown timer** upon sending. | |
| - If no matching `ack` is received within $500\text{ ms}$, the packet is retransmitted (maximum **3 retries** before declaring a dropped connection). | |
| --- | |
| ## 🪝 6. Extended 5-Lifecycle Agent Hook Engine | |
| The MCP server CLI & stdio interface includes an embedded hook engine managing 5 distinct lifecycle events: | |
| 1. **`pretool`** (PreToolUse): Executed immediately before a specific tool runs. | |
| 2. **`posttool`** (PostToolUse): Executed immediately after a specific tool returns. | |
| 3. **`preinvoke`** (PreInvocation): Executed once at the start of an agent invocation request. | |
| 4. **`postinvoke`** (PostInvocation): Executed once after the full invocation completes. | |
| 5. **`stop`** (Stop): Graceful shutdown hook executed via signal handlers (`SIGINT`/`SIGTERM`) or `atexit`. | |
| - **Configuration File**: `~/.gemini/config/hooks.json` | |
| - **Audit Log File**: `~/.gemini/logs/hooks.log` | |
| - **Environment Injections**: `HOOK_TYPE`, `TOOL_NAME`, `ARGS`, `RESULT`. | |
| --- | |
| ## 🧠 7. Antigravity Agent Workspace Layout (`.agents/`) | |
| The workspace `.agents/` directory provides workspace-level context to Google Antigravity: | |
| ``` | |
| .agents/ | |
| ├── .env # Environment variables (AIRMOUSE_PROJECT_ROOT, ports) | |
| ├── .env.template # Template for required system environment variables | |
| ├── config.json # Runtime configuration (gemini-3.1-pro, temp 0.15, max_tokens 16384) | |
| ├── instructions.md # Master system instructions & official documentation links | |
| ├── README.md # Directory guide and documentation sitemap | |
| ├── memory/ | |
| │ ├── known_issues.json # Knowledge graph of resolved bugs and tuning parameters | |
| │ └── project_state.json # Dynamic state tracking (builds, servers, devices) | |
| ├── rules/ | |
| │ └── airmouse-standards.md # Pure Kotlin/Go math rules & 500ms ACK policies | |
| ├── workflows/ | |
| │ └── deploy.yml # Multi-stage CI/CD pipeline script | |
| └── skills/ # 22 Domain Skill Folders with SKILL.md files | |
| ``` | |
| --- | |
| ## 📈 8. Perfetto Trace Analysis (Questions Q1–Q11) | |
| The system includes Android `Trace.beginSection()` / `endSection()` markers: | |
| - `AirMouseApp.Sensors.sensor_read` | |
| - `AirMouseApp.Filter.complementary` | |
| - `AirMouseApp.Filter.compute_delta` | |
| - `AirMouseApp.Communication.send` | |
| ### Metric Summary: | |
| - **Q1 (End-to-End Motion Latency)**: $14.2\text{ ms}$ average latency from sensor read to TCP send. | |
| - **Q2 (Filter Execution Time)**: $0.48\text{ ms}$ per Madgwick iteration. | |
| - **Q3 (Network Retransmission Overhead)**: $< 0.1\%$ packet loss under 5GHz Wi-Fi. | |
| - **Q4–Q11**: Full metric queries documented in [`Gozarish_Perfetto_CA2.md`](file:///Users/tahamajs/Documents/uni/CPS/Files/ComputerAssignments/CA2/Gozarish_Perfetto_CA2.md) and [`code/pc/perfetto_queries.sql`](file:///Users/tahamajs/Documents/uni/CPS/Files/ComputerAssignments/CA2/code/pc/perfetto_queries.sql). | |
| --- | |
| ## 🌐 9. Official Google Antigravity Documentation Sitemap | |
| - 🏠 **Main Documentation Home**: https://antigravity.google/docs | |
| - ⚡ **Skills System**: https://antigravity.google/docs/skills | |
| - 📜 **Rules System**: https://antigravity.google/docs/rules | |
| - 🪝 **Hooks System**: https://antigravity.google/docs/hooks | |
| - 🧩 **Plugins Registry**: https://antigravity.google/docs/plugins | |
| - 🏎️ **Sidecars**: https://antigravity.google/docs/sidecars | |
| - 🔌 **Model Context Protocol (MCP)**: https://antigravity.google/docs/mcp | |
| - 🌐 **Browser Automation & Testing**: https://antigravity.google/docs/browser | |
| - 🛡️ **Agent Permissions & Security**: https://antigravity.google/docs/agent-permissions | |
| - 📜 **Changelog & Release Notes**: https://antigravity.google/changelog | |
| - 🆘 **Troubleshooting & Support**: https://antigravity.google/support | |
Xet Storage Details
- Size:
- 8.77 kB
- Xet hash:
- 568e42a4a5a99502d1732948e2880281d98c5180468669f4a8ed1d667849edff
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.