Files/ComputerAssignments/CA2/code/pc/Proximity-Aware_Connectivity.md

🖱️ Air Mouse Pro – Complete Cross-Platform Remote Control System

University of Tehran – Embedded Systems Laboratory
Professional-grade, AI-powered, proximity-aware remote control for presentations, media centers, and everyday productivity

---

📖 Table of Contents

• Overview
• Features
• Architecture
• Quick Start
• Installation
• Configuration
• Protocol Specification
• AI-Powered Smoothing
• Proximity-Aware Security
• Device Management
• Statistics & Analytics
• Theming & Customization
• Troubleshooting
• Development
• License

---

🎯 Overview

Air Mouse Pro transforms your Android phone into a smart, multi‑protocol remote control for any computer (Windows, macOS, Linux). It combines:

• Six connection methods – TCP, WebSocket, UDP, Bluetooth LE, USB/Serial, Wi‑Fi Direct
• AI‑powered mouse smoothing – human‑like cursor movement using RNN models
• Proximity‑aware auto lock/unlock – using Bluetooth 6.0 Channel Sounding (centimeter‑level distance)
• Cross‑platform desktop server – written in Go, with a modern Fyne GUI
• Real‑time statistics – click counts, scrolls, gesture recognition, performance metrics

---

✨ Features

📡 Multi‑Protocol Connectivity

Protocol	Port	Use Case
TCP	8080	Low‑latency, reliable control
WebSocket	8081	Browser‑based clients
UDP discovery	8082	Automatic server detection
Bluetooth LE	-	HID proxy, BLE device emulation
USB/Serial	-	Wired connection
Wi‑Fi Direct	-	Direct peer‑to‑peer

🧠 AI‑Powered Mouse Smoothing

• Pre‑trained RNN (LSTM) model generates natural velocity curves
• User fine‑tuning – the model learns your personal movement style
• Smoothing + acceleration + gesture detection
• Falls back to traditional EMA smoothing if AI unavailable

🔐 Proximity‑Aware Security (Bluetooth 6.0)

• Channel Sounding – measures distance with ±30‑50 cm accuracy
• Automatically locks screen when phone moves beyond threshold (e.g., 4 meters)
• Automatically unlocks when you return (optional, may require password)
• Hysteresis to avoid rapid toggling
• Falls back to RSSI‑based proximity on older devices

📊 Real‑Time Statistics Dashboard

• Clicks / double‑clicks / right‑clicks / scrolls (live updates)
• Connected devices list with uptime and idle time
• Server uptime, endpoint display, AI smoothing status
• Performance metrics: CPU, memory, active goroutines

🎨 Modern GUI (Fyne)

• 15+ themes – Dark, Light, Pure Black, High Contrast, Ocean, Sunset, Forest, Purple, Cherry, Neon, Lavender, Mint, Peach, Sky
• Tabbed interface – Dashboard, Devices, Network, Settings, Logs
• QR code generation for easy pairing
• Configurable sensitivity, smoothing, acceleration, rate limiting

📱 Android App Features

• Motion control (gyroscope/accelerometer)
• Touchpad mode with gestures
• Voice commands (optional)
• Custom gesture recorder
• Bluetooth HID mode (acts as a real Bluetooth mouse)
• QR scanner for auto‑configuration

🛠️ Additional Capabilities

• Multi‑client – control from multiple phones simultaneously
• Device naming – identify each connected client
• Automatic reconnection – watchdog with heartbeat
• Log exporting – save logs for debugging
• Encrypted communication (optional AES‑GCM)

---

🏗️ Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                         ANDROID APP                                      │
├─────────────────────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐     │
│  │ Motion      │  │ Touchpad    │  │ Voice       │  │ BLE HID     │     │
│  │ Sensors     │  │ Gestures    │  │ Commands    │  │ Proxy       │     │
│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘     │
│         │                │                │                │             │
│         └────────────────┴────────────────┴────────────────┘             │
│                                   │                                      │
│                    ┌──────────────▼──────────────┐                       │
│                    │   Protocol Abstraction      │                       │
│                    │ (TCP / WS / UDP / BLE)      │                       │
│                    └──────────────┬──────────────┘                       │
└───────────────────────────────────┼──────────────────────────────────────┘
                                    │
                    ┌───────────────▼────────────────────────┐
                    │          GO DESKTOP SERVER              │
                    ├─────────────────────────────────────────┤
                    │ ┌─────────┐ ┌─────────┐ ┌───────────┐   │
                    │ │ TCP     │ │ WebSock │ │ UDP Disc  │   │
                    │ │ Server  │ │ Server  │ │ overy     │   │
                    │ └────┬────┘ └────┬────┘ └─────┬─────┘   │
                    │      │           │            │         │
                    │      └───────────┴────────────┘         │
                    │              │                          │
                    │    ┌─────────▼──────────┐               │
                    │    │  Message Router    │               │
                    │    │ & Device Registry  │               │
                    │    └─────────┬──────────┘               │
                    │              │                          │
                    │    ┌─────────▼──────────┐               │
                    │    │  Mouse Controller  │               │
                    │    │  + AI Smoothing    │               │
                    │    │  + Proximity Mgr   │               │
                    │    └─────────┬──────────┘               │
                    │              │                          │
                    │    ┌─────────▼──────────┐               │
                    │    │  Native Input      │               │
                    │    │  (Win32/CoreGr/    │               │
                    │    │   uinput)          │               │
                    │    └────────────────────┘               │
                    └─────────────────────────────────────────┘
                                    │
                    ┌───────────────▼────────────────────────┐
                    │          DESKTOP ENVIRONMENT           │
                    │  (Lock/Unlock via loginctl / DBus)      │
                    └────────────────────────────────────────┘

---

🚀 Quick Start

One‑line setup (Linux / macOS / WSL)

git clone https://github.com/yourusername/airmouse-go.git
cd airmouse-go
make install   # or: go build -o airmouse-server ./cmd/airmouse-server && ./airmouse-server

Android app

1. Install the APK from Releases or build from source
2. Grant Bluetooth and Location permissions
3. Open the app – scan the QR code shown on the desktop server (Network tab)
4. Start moving your phone!

---

📦 Installation

Desktop Server (Go)

Prerequisites

• Go 1.23+
• For Bluetooth support: bluez (Linux), win32 Bluetooth stack (Windows), or CoreBluetooth (macOS)
• For AI smoothing: ONNX Runtime shared library (auto‑fetched by go module)

Build

go mod download
go build -o airmouse-server ./cmd/airmouse-server

Run

./airmouse-server

The GUI will open. Start the server from the Dashboard tab.

Android App

From source (Android Studio)

git clone https://github.com/yourusername/airmouse-android.git
open in Android Studio → Build → Run

Pre‑built APK Download from Releases

---

⚙️ Configuration

The server stores settings in ~/.config/airmouse/config.json (Linux/macOS) or %APPDATA%\airmouse\config.json (Windows).

Example configuration

{
  "host": "0.0.0.0",
  "port": 8080,
  "websocket_port": 8081,
  "udp_port": 8082,
  "enable_tcp": true,
  "enable_websocket": true,
  "enable_udp": true,
  "enable_bluetooth": true,
  "sensitivity": 0.5,
  "theme": "dark",
  "enable_ai_smoothing": true,
  "ai_model_path": "models/mouse_smoothing.onnx",
  "enable_personalization": true,
  "personalization_buffer": 2000,
  "auto_lock_enabled": true,
  "auto_unlock_enabled": false,
  "proximity_near_threshold": 2.0,
  "proximity_far_threshold": 4.0,
  "log_level": "info"
}

Changing settings

• GUI: Settings tab – all options are live‑updated and saved automatically.
• CLI: Edit the JSON file directly and restart the server.

---

📡 Protocol Specification

The server accepts JSON‑line messages over TCP or WebSocket. Each message must end with \n.

Client → Server

Type	Payload	Description
move	{"dx": 1.5, "dy": 2.0}	Move mouse by delta pixels
click	{"button": "left"}	Left or right click
doubleclick	{}	Double click
rightclick	{}	Right click
scroll	{"delta": 1}	Scroll (positive = up)
hello	{"name": "MyPhone", "version": "2.0"}	Identify device
ping	{}	Keep‑alive
proximity	{"is_near": true, "distance": 1.23}	Distance update (Android 16+)

Server → Client

Type	Payload	Description
welcome	{"server":"AirMouse","version":"2.0"}	Sent after hello
ping	{}	Heartbeat request
pong	{}	Heartbeat reply
ack	{"id": "your_message_id"}	Acknowledgment (optional)

---

🧠 AI-Powered Smoothing

How it works

1. Pre‑trained RNN model (LSTM) – trained on 25,000+ real human cursor movements
2. ONNX Runtime – fast inference (≈5ms on modern CPU)
3. User fine‑tuning – collects your movement data and retrains a personalized model in the background

Enabling AI smoothing

• Desktop GUI: Settings tab → enable "AI Smoothing"
• Config file: "enable_ai_smoothing": true

Performance impact

• CPU overhead: <2% on a Core i5
• Memory: ~50 MB for the ONNX model
• Latency added: <10ms (often hidden by network jitter)

Fine‑tuning your model

The system automatically:

1. Buffers your movements (up to personalization_buffer samples)
2. Every personalization_interval seconds, launches the Python trainer service
3. Fine‑tunes the base model and swaps it in (if auto_swap_model is true)

To force a retrain manually, click "Force Retrain Now" in the Analytics tab.

---

🔐 Proximity-Aware Security

Requirements

Component	Minimum Requirement
Android	Android 16+ (API 36) with Bluetooth 6.0 hardware
Desktop (reflector)	Linux kernel 6.15+ (Channel Sounding HCI)
Alternative	Legacy RSSI‑based proximity (less accurate)

Setup

1. Pair your phone via Bluetooth (optional but recommended)
2. On the desktop server, enable "Auto Lock/Unlock" in Settings → Proximity
3. On the Android app, go to Proximity tab and calibrate (place phone at 0.5m, 1m, 2m, 5m)
4. Set your preferred near/far thresholds (default: 2m near, 4m far)

How it works

• The phone acts as initiator (measures distance via Channel Sounding)
• The desktop server acts as reflector (responds to sounding packets)
• Every 1–2 seconds, distance is measured and reported over WebSocket
• When distance exceeds far_threshold for >3 consecutive readings, the screen locks
• When distance drops below near_threshold for >2 readings, it unlocks (if auto_unlock_enabled)

Fallback mode

If Bluetooth 6.0 is not available, the system uses RSSI‑based distance estimation (less accurate, may vary by environment). You can still lock/unlock, but false triggers are more likely.

---

📊 Statistics & Analytics

The Dashboard tab displays:

Statistic	Description
Click count	Total left clicks
Double click count	Total double clicks
Right click count	Total right clicks
Scroll count	Total scroll events (positive/negative combined)
Connected devices	Number of active clients
Server uptime	Time since last start
AI smoothing status	Enabled / Disabled / Personalizing

Additionally, the Logs tab shows real‑time events (info, warning, error) with filtering and export.

Performance metrics (hidden in GUI, available via API)

• CPU usage (percent)
• Memory usage (MB)
• Active goroutines
• Incoming/outgoing bytes per protocol

---

🎨 Theming & Customization

Available themes

Theme Name	Description
dark (default)	Dark mode with blue accents
light	Light mode
pure_black	AMOLED‑friendly (#000000 background)
high_contrast	Accessibility‑optimised
ocean	Blue/teal scheme
sunset	Orange/amber warm tones
forest	Green/earth tones
purple	Purple/pink accents
cherry	Pink/red scheme
neon	Cyberpunk cyan/magenta
lavender	Soft purple pastel
mint	Green‑blue pastel
peach	Warm orange pastel
sky	Light blue pastel

Changing themes

• GUI: Settings tab → Theme dropdown (instant preview)
• Config: "theme": "ocean"

---

🐛 Troubleshooting

Server won't start

• Check if the port is already in use: sudo lsof -i :8080
• Ensure you have permission to bind to the port (Linux: non‑root can use ports >1024)

Android app cannot find server

• Make sure both devices are on the same Wi‑Fi network
• Disable VPN / firewall temporarily
• Try using the QR code in the Network tab
• Check UDP discovery port (8082) is not blocked

Mouse movement feels laggy

• Reduce sensitivity (0.3–0.5 is typical)
• Enable AI smoothing for human‑like velocity
• Check Wi‑Fi signal strength (RSSI > -65dBm recommended)

Bluetooth proximity doesn't work

• Verify both devices support Bluetooth 6.0 (Android 16+, Linux kernel 6.15+)
• Calibrate the Channel Sounding (Settings → Proximity → Calibrate)
• Try the RSSI fallback mode (disable "Use Channel Sounding")

AI smoothing model not loading

• Ensure models/mouse_smoothing.onnx exists (download from Releases)
• Check that ONNX Runtime library is installed (libonnxruntime.so on Linux)
• For fine‑tuning, make sure Python trainer service is running (python3 python_trainer/trainer_server.py)

---

🔧 Development

Building from source

# Desktop server
git clone https://github.com/yourusername/airmouse-go
cd airmouse-go
go mod download
go build -o airmouse-server ./cmd/airmouse-server

# Android app
git clone https://github.com/yourusername/airmouse-android
open in Android Studio → Build → Build APK

Running tests

go test -v ./...

Generating a new AI model

1. Collect mouse movement data (use data_collector.py in python_trainer/)
2. Train the model: python python_trainer/train.py --data data/mouse_log.csv
3. Export to ONNX: python python_trainer/export_onnx.py
4. Place the .onnx file in models/

Adding a new theme

1. Define colors in internal/ui/themes.go
2. Add theme name to getThemeByName()
3. Add option to settings dropdown

---

📄 License

MIT License – Copyright (c) 2025 University of Tehran, Embedded Systems Laboratory

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files, to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software.

---

🙏 Acknowledgements

• Fyne – cross‑platform GUI toolkit
• ONNX Runtime – high‑performance inference
• robotgo – mouse control (fallback)
• gorilla/websocket – WebSocket server
• hashicorp/mdns – Bonjour/Zeroconf
• gopsutil – system metrics
• All contributors and open‑source maintainers

---

Built with ❤️ at University of Tehran – Winter 2025