---
title: BLT Patch View
emoji: 🎧
colorFrom: blue
colorTo: purple
sdk: gradio
app_file: app.py
python_version: 3.12
pinned: false
---

# BLT Patching Visualization

Interactive Gradio app for exploring entropy-based patching strategies in Byte Latent Transformers (BLT).

## Features

- **Entropy & Perplexity Metrics**: Switch between entropy and perplexity as the underlying metric
- **Multi-Constraint Patching**: Combine global, additive, multiplicative, additive-from-min, multiplicative-from-min, cumulative, and cumulative-max thresholds with an optional minimum-metric guard (OR logic)
- **4-Panel Visualization**: Waveform, spectrogram, entropy/perplexity plot, patch size histogram
- **Audio Playback**:
  - Compare original vs WavTokenizer reconstructed audio
  - Play audio from token position until next patch boundary
- **Statistics**: Patch counts, sizes, inference speedup calculations

## Quick Start

### 1. Precompute Data (One-time setup)

This processes the 10 audio samples in `data/test-10` and computes entropy/perplexity using the trained model:

```bash
uv run python apps/patch_view/scripts/precompute_patch_view.py
```

This will:
- Load the entropy model from `archive/checkpoints/audio_peoplespeech_50k_scratch/checkpoints/0000050000`
- Process each audio file (encode with WavTokenizer, compute entropies)
- Save JSON files to `data/test-10/precomputed/`
- Save reconstructed audio files for comparison

Expected output:
- 10 JSON files (~20KB each) with tokens, entropies, perplexities
- 10 reconstructed WAV files for audio comparison
- Total processing time: ~5-10 minutes (depends on GPU)

### 2. Launch Gradio App

```bash
uv run python -m patch_view.app
```

Open in browser: http://localhost:7860

## Usage

1. **Select Audio**: Choose from 10 precomputed audio samples
2. **Configure Patching**:
   - Metric: Entropy or Perplexity
   - Constraints: Select one or more patching rules (global, additive, multiplicative, additive-from-min, multiplicative-from-min, cumulative, cumulative-max)
   - Thresholds: Individual sliders per constraint plus an optional minimum-metric guard
3. **Apply Patching**: Click "Apply Patching" to see results
4. **Explore**:
   - View 4-panel visualization with patch boundaries
   - Compare original vs reconstructed audio
   - Play individual patches by token position

## File Structure

```
apps/patch_view/
├── README.md              # This file
├── requirements.txt       # Python dependencies
├── SIMPLIFIED_DESIGN.md   # Design document
├── app.py                 # Shim that delegates to `patch_view.app:main`
├── data/                  # Bundled test-10 snapshot for the demo
├── scripts/
│   └── precompute_patch_view.py  # Data preprocessing script
├── patch_view/
│   ├── __init__.py
│   ├── app.py             # Gradio wiring (UI only)
│   ├── audio.py           # Cached audio utilities
│   ├── constraints.py     # Slider metadata & helpers
│   ├── data_loader.py     # Precomputed dataset discovery
│   ├── legacy_patching.py # Legacy single-constraint helpers
│   ├── patching.py        # Multi-constraint evaluator
│   ├── services.py        # UI-facing service layer
│   └── visualization.py   # Plotly visualization generation
└── tests/
    └── patch_view/
        ├── conftest.py
        ├── test_audio_cache.py
        ├── test_constraints.py
        └── test_patching.py
```

## Data Format

Precomputed JSON files contain:
```json
{
  "file_id": "121-121726-0000",
  "audio_path": "data/test-10/121-121726-0000.flac",
  "reconstructed_audio_path": "data/test-10/precomputed/121-121726-0000_reconstructed.wav",
  "transcript": "...",
  "audio_duration_sec": 4.35,
  "tokens": [1234, 5678, ...],
  "entropies": [2.5, 3.1, ...],
  "perplexities": [5.66, 8.57, ...],
  "avg_entropy": 3.2,
  "avg_perplexity": 9.2
}
```

## Technical Details

- **Entropy Model**: `archive/checkpoints/audio_peoplespeech_50k_scratch` (step 50000)
- **WavTokenizer**: 4096 vocab, 40 tokens/sec, 24kHz
- **Entropy Calculation**: `H = -Σ p(x) log₂(p(x))` (bits)
- **Perplexity**: `PPL = 2^H`
- **Spectrogram**: Computed at runtime using librosa (not stored in JSON)

## Patching Strategies

### Global Threshold
Creates a new patch when metric exceeds threshold.

### Additive Monotonic
Creates a new patch when metric increase exceeds threshold:
- `delta = H(x_i) - H(x_{i-1})`
- If `delta > threshold`: new patch

### Multiplicative Monotonic
Creates a new patch when metric ratio exceeds threshold:
- `ratio = H(x_i) / H(x_{i-1})`
- If `ratio > threshold`: new patch

### Additive from Minimum
Creates a patch when the metric rises above the minimum observed in the current patch by more than the threshold.

### Multiplicative from Minimum
Creates a patch when the metric grows proportionally compared to the current patch's minimum.

### Cumulative Sum (Min)
Accumulates metric values and inserts a boundary immediately after the running sum exceeds the threshold.

### Cumulative Sum (Max)
Attempts to keep patch sums below the threshold by deferring boundaries until the next token would overflow.

### Minimum Metric Guard
Optional AND guard that delays any boundary creation until the metric itself passes a minimum value (applies alongside the OR-combined constraints).

## Statistics

The app displays:
- Total number of patches
- Average, min, max, median patch sizes (in tokens and seconds)
- Standard deviation
- **Inference speedup**: `num_tokens / num_patches`
  - Shows how many fewer transformer steps are needed vs. token-level processing

## Requirements

- Python 3.10+
- CUDA-capable GPU (for precomputation)
- ~2GB GPU memory for entropy model inference
- See `requirements.txt` for Python dependencies