docs/GUI_VISUALIZATION.md

# Real-time GUI Visualization

## Overview

The GUI visualization provides real-time, progressive updates as BA validation runs. It shows:

- **3D camera trajectories** updating frame-by-frame
- **Error metrics** plotted as they're computed
- **Statistics** updating in real-time
- **Progress indicators** showing current status

## Usage

### Basic Usage

```bash
python scripts/run_arkit_ba_validation_gui.py \
    --arkit-dir assets/examples/ARKit \
    --output-dir data/arkit_ba_validation_gui \
    --max-frames 30 \
    --frame-interval 1 \
    --device cpu
```

### GUI Components

#### Left Panel

1. **Status Panel**:

   - Current operation (e.g., "Running DA3 inference...")
   - Progress bar (indeterminate during processing)
   - Frame counter (e.g., "5/30 frames")

2. **Statistics Panel**:
   - Real-time statistics updating as data arrives
   - Mean/max rotation errors
   - Mean translation errors
   - Comparison metrics (DA3 vs ARKit, BA vs ARKit, DA3 vs BA)

#### Right Panel (Tabs)

1. **3D Trajectories Tab**:

   - Interactive 3D plot showing camera paths
   - Green: ARKit (ground truth)
   - Red: DA3 predictions
   - Blue: BA refined poses
   - Updates progressively as frames are processed
   - Can rotate, zoom, pan using matplotlib toolbar

2. **Error Metrics Tab**:

   - Rotation errors plotted over time
   - Threshold lines (2° accept, 30° reject)
   - Updates as errors are computed

3. **Comparison Tab**:
   - Side-by-side comparison of rotation and translation errors
   - Multiple methods shown together
   - Updates progressively

## Progressive Updates

The GUI updates in real-time as:

1. **Frame Extraction**: Progress bar shows frame extraction
2. **DA3 Inference**:
   - Status updates to "Running DA3 inference..."
   - Trajectories update as each frame's pose is computed
   - Progress counter increments
3. **BA Validation**:
   - Status updates to "Running BA validation..."
   - BA poses appear on trajectory plot
   - Error metrics update as computed
4. **Completion**:
   - Final statistics displayed
   - All visualizations finalized

## Thread Safety

The GUI uses a thread-safe update mechanism:

- Validation runs in a background thread
- Updates are queued and processed in the main GUI thread
- No blocking of the GUI during computation

## Features

### Real-time Updates

- Visualizations update as data arrives
- No need to wait for entire process to complete
- See results immediately

### Interactive 3D Plot

- Rotate, zoom, pan using matplotlib toolbar
- Toggle trajectories on/off (via legend)
- Export to image (via toolbar)

### Error Analysis

- See error patterns as they develop
- Identify problematic frames early
- Compare methods side-by-side

### Statistics Panel

- Real-time statistics
- Scrollable text area
- Auto-updates as new data arrives

## Example Workflow

1. **Start GUI**: Run the script, GUI window opens
2. **Watch Progress**:
   - Status shows "Processing ARKit data..."
   - Progress bar animates
3. **DA3 Inference**:
   - Trajectory plot shows ARKit path (green)
   - DA3 poses appear (red) as computed
   - Statistics update
4. **BA Validation**:
   - BA poses appear (blue)
   - Error metrics populate
   - Final statistics displayed
5. **Analysis**:
   - Rotate 3D plot to inspect trajectories
   - Check error plots for patterns
   - Review statistics panel

## Troubleshooting

### GUI Not Updating

- Check that validation thread is running (status should change)
- Verify no exceptions in console
- Ensure GUI main loop is running (window should be responsive)

### Performance Issues

- Reduce `--max-frames` for faster updates
- Use `--device cpu` if GPU is slow
- Increase `frame_interval` to process fewer frames

### Missing Updates

- Some updates may be batched for performance
- Check statistics panel for final results
- All data is available after completion

## Integration

The GUI can be integrated into other workflows:

```python
from ylff.visualization_gui import create_gui
from scripts.run_arkit_ba_validation_gui import run_validation_with_gui

# Create GUI
gui = create_gui()

# Start validation
run_validation_with_gui(
    gui,
    arkit_dir=Path("path/to/arkit"),
    output_dir=Path("output"),
    max_frames=30,
)

# GUI runs until window is closed
```

## Advantages Over Static Visualization

1. **Immediate Feedback**: See results as they're computed
2. **Early Problem Detection**: Identify issues before completion
3. **Interactive Exploration**: Rotate/zoom 3D plots in real-time
4. **Progress Monitoring**: Know exactly what's happening
5. **No Post-Processing**: Results available immediately