Hugging Face's logo

Buckets:
tahamajs
/
CPS

Files
xet
tahamajs/CPS / Files /ComputerAssignments /CA2 /AGENT.md
tahamajs's picture
tahamajs
preview code
|
download
raw
8.77 kB

๐Ÿค– AirMouse Pro โ€“ Master Agent Specification (AGENT.md)

Welcome to the definitive architecture, design, and operating manual for the AirMouse Pro Cyber-Physical System (CPS CA2 Assignment).

This document serves as the primary reference for AI Agents, developers, and evaluators interacting with the AirMouse ecosystem.

๐Ÿ›๏ธ 1. Executive System Overview

AirMouse Pro is an end-to-end, ultra-low latency wireless air-mouse system. It transforms an Android smartphone into a high-precision 3D spatial cursor controller and gesture input device.

graph TD
    A["Android App (Kotlin)<br>code/android/"] -->|TCP 9090 / UDP 9093| B["Go PC Server<br>code/pc/airmouse_mcp_go/"]
    A -->|Trace Markers| P["Perfetto Tracing<br>trace_file.perfetto-trace"]
    B -->|OS Cursor Events| C["Host OS Desktop"]
    D["Python MCP Server (444 Tools)<br>airmouse_server_mcp.py"] -->|Admin REST 8081| B
    D -->|ADB Automation| A
    E["Antigravity AI Agent<br>.agents/ Workspace"] -->|FastMCP / CLI| D

Key Subsystems:

  1. Android Client (code/android/): Pure Kotlin IMU reader, 3-step calibration manager, Madgwick AHRS filter ($\beta=0.041$), gesture detection, and line-delimited JSON TCP socket client.
  2. Go Desktop Backend (code/pc/airmouse_mcp_go/): High-performance multi-protocol server (TCP 9090, WebSocket 9091, UDP Discovery 9092, UDP Stream 9093, REST Admin 8081) with Kalman predictive blending and power-law cursor acceleration.
  3. Python MCP Server (airmouse_server_mcp.py): Master 444-tool FastMCP server and CLI supporting an extended 5-lifecycle agent hook engine.
  4. Antigravity Workspace (.agents/): AI configuration hub featuring 22 domain-specific skills, system rules, dynamic memory, workflows, and environment settings.

๐Ÿงฎ 2. Mathematical Models & Sensor Fusion

2.1 Madgwick AHRS Filter ($\beta = 0.041$)

The orientation orientation quaternion $q = [q_1, q_2, q_3, q_4]^T$ is computed continuously using gradient descent optimization to fuse gyroscope ($\mathbf{\omega}$ in $\text{rad/s}$), accelerometer ($\mathbf{a}$ in $\text{m/s}^2$), and magnetometer ($\mathbf{m}$ in $\mu\text{T}$) inputs:

qห™=12qโŠ—ฯ‰โˆ’ฮฒโˆ‡fโˆฅโˆ‡fโˆฅ\dot{q} = \frac{1}{2} q \otimes \mathbf{\omega} - \beta \frac{\nabla f}{\|\nabla f\|}

  • Gyroscope Units: Standard Android $\text{rad/s}$ (never converted to degrees prior to filter update).
  • Gain Parameter: Fixed at $\beta = 0.041$ for optimal balance between gyro drift rejection and dynamic response.

2.2 Quaternion to Euler Angles Conversion

To translate orientation into screen space coordinates:

Roll (ฯ•)=atan2โก(2(q1q2+q3q4),1โˆ’2(q22+q32))Pitch (ฮธ)=arcsinโก(2(q1q3โˆ’q4q2))Yaw (ฯˆ)=atan2โก(2(q1q4+q2q3),1โˆ’2(q32+q42))\begin{aligned} \text{Roll } (\phi) &= \operatorname{atan2}(2(q_1 q_2 + q_3 q_4), 1 - 2(q_2^2 + q_3^2)) \\ \text{Pitch } (\theta) &= \arcsin(2(q_1 q_3 - q_4 q_2)) \\ \text{Yaw } (\psi) &= \operatorname{atan2}(2(q_1 q_4 + q_2 q_3), 1 - 2(q_3^2 + q_4^2)) \end{aligned}

๐ŸŽฏ 3. 3-Step Sensor Calibration Protocol

Calibration Step Sensor Target Procedure Mathematical Model
Step 1: Gyroscope Gyro Bias Vector Place phone on flat, vibration-free surface; collect 100 stationary samples. $\mathbf{b}g = \frac{1}{100} \sum{i=1}^{100} \mathbf{\omega}_i$
Step 2: Magnetometer Hard/Soft-Iron Offsets Perform figure-8 motion in 3D space; collect 200 samples. $\mathbf{V}{\text{hard}} = \frac{\mathbf{m}{\max} + \mathbf{m}{\min}}{2}, \quad \mathbf{S}{\text{soft}} = \frac{\text{avg_range}}{\mathbf{m}{\max} - \mathbf{m}{\min}}$
Step 3: Accelerometer Scale & Zero-g Bias Place phone in 6 static orientations (Flat Up/Down, Left, Right, Top, Bottom). Least-Squares Optimization to enforce $|\mathbf{a}| = 9.81 \text{ m/s}^2$

๐Ÿ–๏ธ 4. Motion Gesture Detection Engine

  • Click (Left Click): Gyroscope Y-axis angular velocity $> 8.0 \text{ rad/s}$ with a $300\text{ ms}$ cooldown.
  • Double Click: Second left-click gesture detected within $400\text{ ms}$ of the first.
  • Scroll Movement: Accelerometer Y-axis acceleration $> 8.0 \text{ m/s}^2$ with a $2.0 \text{ m/s}^2$ debounce and $100\text{ ms}$ cooldown.
  • Right Click: Roll tilt angle $> 45^\circ$ held continuously for $\ge 500\text{ ms}$.

๐Ÿ“ก 5. Network Protocol & ACK Specification

Messages are formatted as newline-delimited JSON (\n) transmitted over TCP port 9090 or UDP stream 9093.

5.1 Packet Types & Schemas

  • hello: Client registration handshake ({"type":"hello","device_name":"Pixel 8","version":"2.0"}).
  • move: Continuous delta orientation stream ({"type":"move","dx":12.5,"dy":-4.2}).
  • click: Left-click request ({"type":"click","id":"req_101"}).
  • scroll: Vertical scroll command ({"type":"scroll","id":"req_102","delta":-3}).
  • ack: Server acknowledgment ({"type":"ack","id":"req_101"}).

5.2 Reliable Retransmission Logic

For click, scroll, and control message types:

  • Client starts a 500 ms countdown timer upon sending.
  • If no matching ack is received within $500\text{ ms}$, the packet is retransmitted (maximum 3 retries before declaring a dropped connection).

๐Ÿช 6. Extended 5-Lifecycle Agent Hook Engine

The MCP server CLI & stdio interface includes an embedded hook engine managing 5 distinct lifecycle events:

  1. pretool (PreToolUse): Executed immediately before a specific tool runs.
  2. posttool (PostToolUse): Executed immediately after a specific tool returns.
  3. preinvoke (PreInvocation): Executed once at the start of an agent invocation request.
  4. postinvoke (PostInvocation): Executed once after the full invocation completes.
  5. stop (Stop): Graceful shutdown hook executed via signal handlers (SIGINT/SIGTERM) or atexit.
  • Configuration File: ~/.gemini/config/hooks.json
  • Audit Log File: ~/.gemini/logs/hooks.log
  • Environment Injections: HOOK_TYPE, TOOL_NAME, ARGS, RESULT.

๐Ÿง  7. Antigravity Agent Workspace Layout (.agents/)

The workspace .agents/ directory provides workspace-level context to Google Antigravity:

.agents/
โ”œโ”€โ”€ .env                    # Environment variables (AIRMOUSE_PROJECT_ROOT, ports)
โ”œโ”€โ”€ .env.template           # Template for required system environment variables
โ”œโ”€โ”€ config.json             # Runtime configuration (gemini-3.1-pro, temp 0.15, max_tokens 16384)
โ”œโ”€โ”€ instructions.md         # Master system instructions & official documentation links
โ”œโ”€โ”€ README.md               # Directory guide and documentation sitemap
โ”œโ”€โ”€ memory/
โ”‚   โ”œโ”€โ”€ known_issues.json   # Knowledge graph of resolved bugs and tuning parameters
โ”‚   โ””โ”€โ”€ project_state.json  # Dynamic state tracking (builds, servers, devices)
โ”œโ”€โ”€ rules/
โ”‚   โ””โ”€โ”€ airmouse-standards.md # Pure Kotlin/Go math rules & 500ms ACK policies
โ”œโ”€โ”€ workflows/
โ”‚   โ””โ”€โ”€ deploy.yml          # Multi-stage CI/CD pipeline script
โ””โ”€โ”€ skills/                 # 22 Domain Skill Folders with SKILL.md files

๐Ÿ“ˆ 8. Perfetto Trace Analysis (Questions Q1โ€“Q11)

The system includes Android Trace.beginSection() / endSection() markers:

  • AirMouseApp.Sensors.sensor_read
  • AirMouseApp.Filter.complementary
  • AirMouseApp.Filter.compute_delta
  • AirMouseApp.Communication.send

Metric Summary:

  • Q1 (End-to-End Motion Latency): $14.2\text{ ms}$ average latency from sensor read to TCP send.
  • Q2 (Filter Execution Time): $0.48\text{ ms}$ per Madgwick iteration.
  • Q3 (Network Retransmission Overhead): $< 0.1%$ packet loss under 5GHz Wi-Fi.
  • Q4โ€“Q11: Full metric queries documented in Gozarish_Perfetto_CA2.md and code/pc/perfetto_queries.sql.

๐ŸŒ 9. Official Google Antigravity Documentation Sitemap

Xet Storage Details

Size:
8.77 kB
ยท
Xet hash:
568e42a4a5a99502d1732948e2880281d98c5180468669f4a8ed1d667849edff

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.