plan.md

# Interactive Kernel Convolution Visualizer - Project Plan

## Project Overview
Build an interactive web application that demonstrates how image convolution works in real time. The application serves both as an image-processing tool and an educational visualization platform for understanding convolution kernels, their mathematical operations, and their effects on images.

## Target Audience
- Students learning computer vision and digital image processing
- Developers learning about CNNs and kernel operations
- Educators teaching image processing concepts
- Anyone curious about how image filters work

## Core Features

### 1. Image Management
- Upload any image (drag & drop, file picker)
- Support common formats: PNG, JPEG, WebP, BMP
- Display original image with zoom/pan capabilities
- Image metadata display (dimensions, file size, color channels)

### 2. Kernel Library
Predefined kernels:
- **Blur Filters**: Box Blur (3×3, 5×5, 7×7), Gaussian Blur (3×3, 5×5)
- **Sharpen Filters**: Sharpen, Unsharp Mask
- **Edge Detection**: Sobel X, Sobel Y, Sobel Combined, Prewitt, Roberts, Laplacian
- **Emboss**: Emboss (multiple directions)
- **Special Effects**: Motion Blur, Outline, Ridge Detection
- **Custom Kernel Builder**: User-defined matrix with size selector (3×3, 5×5, 7×7, 9×9)

### 3. Convolution Visualization (Core Feature)
- **Step-by-step Animation**: Kernel matrix moves across image pixel by pixel
- **Current Patch Highlight**: Visual indication of image region under kernel
- **Element-wise Multiplication**: Show kernel values × pixel values
- **Summation Display**: Real-time calculation of output pixel value
- **Continuous Animation**: Smooth traversal across entire image
- **Progressive Output Rendering**: Output image builds up as computation completes

### 4. Interactive Controls
- Play / Pause / Restart / Step Forward / Step Backward
- Animation speed slider (0.5x - 10x)
- Zoom controls for pixel-level inspection
- Toggle overlays:
  - Kernel boundary visualization
  - Pixel value display
  - Multiplication calculation overlay
  - Summation formula display

### 5. Comparison & Analysis
- Side-by-side original vs processed view
- Split-view slider (before/after)
- Synchronized zoom/pan between views
- Multiple kernel comparison mode (tabbed results)
- Processing statistics: execution time, operations count, memory usage

### 6. Padding Strategies
- Visual demonstration of padding effects
- Options: Zero Padding, Reflect Padding, Replicate Padding, Wrap Padding
- Border handling visualization at image edges

### 7. Educational Insights
- Real-time mathematical equation display
- Kernel purpose/effect description
- Convolution formula with current values
- Channel-wise processing explanation (RGB)
- Links to learning resources

### 8. Export & Sharing
- Save processed image (PNG, JPEG)
- Export visualization as GIF/MP4
- Shareable URL with current state
- Download convolution computation log (CSV/JSON)

## Technical Architecture

### Frontend Stack
- **Framework**: Next.js 14+ (App Router) with TypeScript
- **State Management**: Zustand or React Context
- **Animation**: Framer Motion for UI, custom Canvas/WebGL for convolution viz
- **Styling**: Tailwind CSS
- **Canvas**: HTML5 Canvas API (primary), WebGL fallback for performance
- **Image Processing**: Custom JavaScript implementation (no heavy deps)

### Component Structure
```
src/
├── app/                    # Next.js App Router pages
├── components/
│   ├── canvas/            # Canvas-based visualization components
│   ├── controls/          # Playback, speed, zoom controls
│   ├── kernel/            # Kernel selector, editor, library
│   ├── comparison/        # Split view, side-by-side comparison
│   ├── education/         # Math display, kernel info panels
│   └── ui/                # Reusable UI components (buttons, sliders, etc.)
├── hooks/                 # Custom React hooks
├── lib/
│   ├── convolution/       # Core convolution algorithms
│   ├── kernels/           # Predefined kernel definitions
│   ├── image/             # Image loading, processing utilities
│   └── export/            # Export functionality
├── store/                 # State management
├── types/                 # TypeScript definitions
└── utils/                 # Helper functions
```

### Core Algorithms

#### Convolution Engine
```typescript
interface ConvolutionOptions {
  kernel: number[][];
  padding: 'zero' | 'reflect' | 'replicate' | 'wrap';
  normalize: boolean;
  stride: number;
}

function convolve(
  imageData: ImageData,
  kernel: number[][],
  options: ConvolutionOptions,
  onProgress?: (progress: ConvolutionProgress) => void
): Promise<ImageData>;
```

#### Step-by-step Generator
```typescript
interface ConvolutionStep {
  x: number;
  y: number;
  inputPatch: number[][][];
  kernel: number[][];
  multiplications: number[][][];
  sum: number;
  outputValue: number;
  channel: number;
}

function* convolutionSteps(
  imageData: ImageData,
  kernel: number[][],
  options: ConvolutionOptions
): Generator<ConvolutionStep>;
```

### Animation System
- RequestAnimationFrame-based loop
- Configurable frame rate (target 60fps)
- Step batching for performance (process N pixels per frame)
- Pause/resume without losing state
- Step-by-step mode with manual advancement

### Performance Considerations
- Web Workers for heavy computation
- Canvas pooling to reduce GC pressure
- TypedArrays for numerical computation
- Progressive rendering (don't block UI)
- Lazy evaluation for comparison views

## Development Phases

### Phase 1: Foundation (Week 1-2)
- [ ] Project setup (Next.js, TypeScript, Tailwind, ESLint, Prettier)
- [ ] Basic project structure and component architecture
- [ ] Image upload and display component
- [ ] Basic Canvas rendering setup
- [ ] Kernel data structure and predefined kernels library

### Phase 2: Core Convolution (Week 2-3)
- [ ] Implement convolution algorithm (CPU)
- [ ] Implement padding strategies
- [ ] Create step-by-step generator
- [ ] Basic convolution visualization (static)
- [ ] Output image generation

### Phase 3: Animation & Visualization (Week 3-4)
- [ ] Animation loop with play/pause/step controls
- [ ] Speed control implementation
- [ ] Current patch highlighting
- [ ] Element-wise multiplication visualization
- [ ] Summation process display
- [ ] Progressive output rendering

### Phase 4: Interactive Controls (Week 4-5)
- [ ] Zoom/pan functionality
- [ ] Overlay toggles (kernel boundary, values, math)
- [ ] Kernel selector UI with categories
- [ ] Custom kernel editor
- [ ] Padding strategy selector with visualization

### Phase 5: Comparison & Analysis (Week 5-6)
- [ ] Side-by-side comparison view
- [ ] Split slider component
- [ ] Synchronized zoom/pan
- [ ] Multi-kernel comparison tabs
- [ ] Statistics panel (time, ops, memory)

### Phase 6: Educational Features (Week 6-7)
- [ ] Real-time equation display
- [ ] Kernel information panels
- [ ] Channel processing visualization
- [ ] Tooltip/explanation system
- [ ] Learning resource links

### Phase 7: Export & Polish (Week 7-8)
- [ ] Save processed image
- [ ] Export animation (GIF/Video)
- [ ] Shareable state URLs
- [ ] Performance optimization
- [ ] Mobile responsiveness
- [ ] Accessibility improvements
- [ ] Documentation & help system

### Phase 8: Advanced Features (Week 8+)
- [ ] WebGL acceleration option
- [ ] Web Worker for computation
- [ ] Batch processing multiple images
- [ ] Kernel composition (sequential filters)
- [ ] Frequency domain visualization (FFT)
- [ ] 3D kernel visualization

## Data Structures

### Kernel Definition
```typescript
interface Kernel {
  id: string;
  name: string;
  category: 'blur' | 'sharpen' | 'edge' | 'emboss' | 'custom' | 'other';
  matrix: number[][];
  size: number;
  normalize: boolean;
  description: string;
  formula?: string;
  reference?: string;
}
```

### Convolution State
```typescript
interface ConvolutionState {
  status: 'idle' | 'running' | 'paused' | 'completed' | 'error';
  currentStep: ConvolutionStep | null;
  progress: number; // 0-1
  startTime: number;
  elapsedTime: number;
  processedPixels: number;
  totalPixels: number;
  outputImageData: ImageData | null;
}
```

### App Settings
```typescript
interface AppSettings {
  animationSpeed: number; // 0.5 - 10
  showOverlay: {
    kernelBoundary: boolean;
    pixelValues: boolean;
    multiplications: boolean;
    summation: boolean;
  };
  padding: 'zero' | 'reflect' | 'replicate' | 'wrap';
  zoomLevel: number;
  panOffset: { x: number; y: number };
  comparisonMode: 'side-by-side' | 'split' | 'overlay';
}
```

## Testing Strategy

### Unit Tests
- Convolution algorithm correctness
- Padding strategy implementations
- Kernel normalization
- Step generator accuracy
- Export functionality

### Integration Tests
- Image upload → convolution → output pipeline
- Animation playback controls
- Comparison view synchronization
- State persistence/restoration

### Visual Regression Tests
- Canvas rendering at different zoom levels
- Overlay accuracy
- Split slider positions
- Kernel matrix display

### Performance Benchmarks
- Convolution speed (pixels/second)
- Animation frame rate
- Memory usage during large image processing
- Web Worker vs main thread performance

## Accessibility
- Keyboard navigation for all controls
- Screen reader announcements for animation state
- High contrast mode support
- Reduced motion preference respect
- Focus indicators
- Alt text for all visual elements

## Browser Support
- Chrome 90+
- Firefox 88+
- Safari 14+
- Edge 90+
- Mobile browsers (iOS Safari, Chrome Android)

## Deployment
- Static export to Vercel/Netlify
- CDN for kernel definitions and assets
- PWA support for offline use
- Analytics (privacy-friendly)

## Success Metrics
- Frame rate ≥ 30fps during animation
- Convolution of 1MP image < 2 seconds
- Time to interactive < 3 seconds
- Lighthouse score > 90
- Zero critical accessibility violations

## Risk Mitigation
| Risk | Mitigation |
|------|------------|
| Performance on large images | Web Workers, progressive rendering, WebGL fallback |
| Animation smoothness | RequestAnimationFrame, frame budgeting, step batching |
| Memory leaks | Canvas pooling, proper cleanup, WeakRef for caches |
| Cross-browser canvas differences | Feature detection, polyfills, fallback rendering |
| Complex state management | Zustand with devtools, immutable updates |

## Future Extensions
- Video convolution visualization
- 3D convolution (volumetric data)
- CNN layer visualization (multiple kernels, feature maps)
- Interactive kernel design with real-time preview
- Frequency domain analysis
- Integration with ML frameworks (TensorFlow.js, ONNX.js)
- Collaborative annotation/sharing
- Plugin system for custom kernels/effects