VPER Metric — Visualization Performance Enhancement Rate
Where E = error rate (1 − accuracy). Visual traces reduce AI diagnostic error by 27.9% over tabular query statistics alone.
| Condition | Error Rate | Accuracy | Improvement |
|---|---|---|---|
| Tabular stats only (baseline) | 64.3% | 35.7% | — |
| + PlanViz visual trace | 46.4% | 53.6% | +17.9pp (VPER=27.9%) |
AI-VPER Benchmark Protocol
Standardized evaluation protocol for measuring multimodal AI performance on SQL plan debugging tasks.
Present AI model with tabular query statistics only: runtime, rows processed, memory usage, CPU time. Record diagnostic accuracy on 192 annotated bug reports.
Present the same AI model with the PlanViz SQL execution plan visualization attached to each bug report. Record diagnostic accuracy under identical conditions.
Compute VPER = (E_tabular − E_visual) / E_tabular. Positive VPER indicates visualization improves diagnosis. Report with 95% confidence intervals and paired t-test.
Compare AI-VPER against human expert VPER on the same 192 bugs to measure the gap between AI and human visual reasoning over SQL execution traces.
Dataset Details
| Property | Value |
|---|---|
| Total bug reports | 192 |
| Source system | Commercial column-store RDBMS (enterprise production) |
| Collection period | 2016–2025 (10 years) |
| Annotation type | Root cause label + diagnostic difficulty + resolution category |
| Privacy protection | k-anonymity ≥ 5; customer identifiers removed; SQL generalized |
| License | CC BY 4.0 |
| Format | CSV + XLSX |
Access & Citation
📦 Download Dataset ⬇ Direct CSV Downloadtitle = {VisualTraceBench: A Human-Performance Benchmark for Multimodal AI SQL Plan Debugging},
author = {Anonymous},
year = {2026},
url = {https://huggingface.co/datasets/viperoptic/VisualTraceBench}
}