Hugging Face's logo

Trishti
/
InsightUX

ONNX
Model card Files
xet
 Community
InsightUX
File size: 2,637 Bytes
bcb81c1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
# 👁️ InsightUX Browser

> AI-powered eye-tracking research browser for usability analysis and user experience evaluation.

InsightUX Browser is a desktop research browser built using **PyWebView**, **MediaPipe**, and an **ONNX-based gaze estimation model**. It enables researchers to record users' gaze behavior while browsing real websites and automatically generates detailed visual analytics including attention heatmaps, dwell timelines, and ranked Areas of Interest (AOIs).

---

## ✨ Features

- 👁️ Real-time gaze estimation using a CNN (ONNX Runtime)
- 🌐 Integrated research browser built with PyWebView
- 🎯 Automatic AOI (Area of Interest) detection
- 📊 Attention ranking and dwell-time analytics
- 🔥 Screenshot-based heatmap generation
- 📈 Timeline visualization of user attention
- 💾 Session logging for gaze and DOM interactions
- 📷 Automatic screenshot capture during browsing
- 🧠 Head pose compensation and illumination normalization
- ⚡ Lightweight desktop application

---

## Project Structure

```text
InsightUX/
│
├── browser_session.py          # Main browser application
├── calibrate.py                # User calibration
├── validate.py                 # Calibration validation
├── analysis.py                 # Analytics & report generation
├── inference_pipeline.py       # ONNX inference pipeline
├── session_logger.py           # Session logging
│
├── models/
│   └── gaze_cnn_v4.onnx
│
├── preprocessing/
│   └── preprocessing_pipeline.py
│
├── sessions/
│
└── requirements.txt
```

---

## Requirements

- Python 3.11
- Webcam
- Windows 10 / Windows 11

---

## Installation

Create a virtual environment

```bash
python -m venv venv
```

Activate it

```bash
.\venv\Scripts\Activate.ps1
```

Install dependencies

```bash
pip install -r requirements.txt
```

---

## Usage

### 1. Calibrate

```bash
python calibrate.py
```

This generates:

- calibration.pkl

---

### 2. Validate (Optional)

```bash
python validate.py
```

---

### 3. Launch Browser

```bash
python browser_session.py
```

Workflow

1. Open any website
2. Click on the webpage
3. Press **S** to start tracking
4. Browse normally
5. Press **E** to stop tracking
6. Analytics report opens automatically

---

## Generated Outputs

Each session generates

- Raw gaze logs
- DOM interaction logs
- Website screenshots
- Attention heatmaps
- Ranked AOIs
- Dwell timelines
- HTML analytics report

---

## Technology Stack

- Python
- PyWebView
- MediaPipe
- OpenCV
- NumPy
- ONNX Runtime
- JavaScript
- HTML/CSS

---