Hugging Face's logo

Spaces:
viperoptic
/
VisualTraceBench-Demo
No application file

App Files Community
VisualTraceBench-Demo / index.html
viperoptic's picture
viperoptic
Initial deploy: VisualTraceBench demo
c9a906b verified
Raw History Blame Contribute Delete
9.89 kB
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>VisualTraceBench — SQL Plan Debugging Benchmark</title>
<style>
* { box-sizing: border-box; margin: 0; padding: 0; }
body { font-family: 'Segoe UI', system-ui, sans-serif; background: #f8fafc; color: #1e293b; }
header { background: linear-gradient(135deg, #1a73e8, #0d47a1); color: white; padding: 40px 24px; text-align: center; }
header h1 { font-size: 2rem; font-weight: 700; margin-bottom: 8px; }
header p { font-size: 1.05rem; opacity: 0.9; max-width: 700px; margin: 0 auto; }
.badge { display: inline-block; background: rgba(255,255,255,0.2); border-radius: 20px; padding: 4px 14px; font-size: 0.85rem; margin-top: 12px; }
main { max-width: 960px; margin: 0 auto; padding: 32px 16px; }
.stats-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(180px, 1fr)); gap: 16px; margin: 28px 0; }
.stat-card { background: white; border-radius: 12px; padding: 24px; text-align: center; box-shadow: 0 2px 8px rgba(0,0,0,0.07); border-top: 4px solid #1a73e8; }
.stat-card .value { font-size: 2.2rem; font-weight: 700; color: #1a73e8; }
.stat-card .label { font-size: 0.85rem; color: #64748b; margin-top: 4px; }
.section { background: white; border-radius: 12px; padding: 28px; margin-bottom: 20px; box-shadow: 0 2px 8px rgba(0,0,0,0.07); }
.section h2 { font-size: 1.25rem; color: #0d47a1; margin-bottom: 16px; border-bottom: 2px solid #e2e8f0; padding-bottom: 10px; }
.vper-highlight { background: #e8f0fe; border-left: 4px solid #1a73e8; padding: 16px 20px; border-radius: 0 8px 8px 0; margin: 16px 0; }
.vper-highlight .formula { font-family: monospace; font-size: 1.1rem; color: #1a73e8; font-weight: 600; }
.vper-highlight p { color: #374151; margin-top: 8px; font-size: 0.95rem; }
table { width: 100%; border-collapse: collapse; font-size: 0.9rem; }
th { background: #f1f5f9; text-align: left; padding: 10px 12px; color: #475569; font-weight: 600; }
td { padding: 10px 12px; border-bottom: 1px solid #f1f5f9; }
tr:hover td { background: #f8fafc; }
.tag { display: inline-block; background: #dbeafe; color: #1d4ed8; border-radius: 12px; padding: 2px 10px; font-size: 0.78rem; margin: 2px; }
.btn { display: inline-block; background: #1a73e8; color: white; padding: 12px 24px; border-radius: 8px; text-decoration: none; font-weight: 600; margin: 8px 8px 8px 0; transition: background 0.2s; }
.btn:hover { background: #0d47a1; }
.btn.secondary { background: white; color: #1a73e8; border: 2px solid #1a73e8; }
.btn.secondary:hover { background: #e8f0fe; }
.protocol-step { display: flex; gap: 16px; margin-bottom: 16px; align-items: flex-start; }
.step-num { background: #1a73e8; color: white; border-radius: 50%; width: 32px; height: 32px; display: flex; align-items: center; justify-content: center; font-weight: 700; flex-shrink: 0; font-size: 0.9rem; }
.step-content strong { display: block; color: #0d47a1; margin-bottom: 4px; }
.step-content p { color: #475569; font-size: 0.9rem; }
footer { text-align: center; padding: 32px; color: #94a3b8; font-size: 0.85rem; }
.result-bar { background: #e2e8f0; border-radius: 4px; height: 20px; margin-top: 6px; overflow: hidden; }
.result-bar-fill { height: 100%; border-radius: 4px; display: flex; align-items: center; padding-left: 8px; font-size: 0.78rem; color: white; font-weight: 600; }
</style>
</head>
<body>
<header>
<h1>VisualTraceBench</h1>
<p>A Human-Performance Benchmark for Multimodal AI SQL Plan Debugging with PlanViz Visual Traces</p>
<span class="badge">ICLR 2027 Datasets &amp; Benchmarks Track — Under Review</span>
</header>
<main>
<!-- Key Stats -->
<div class="stats-grid">
<div class="stat-card">
<div class="value">192</div>
<div class="label">Production Bug Reports</div>
</div>
<div class="stat-card">
<div class="value">27.9%</div>
<div class="label">VPER Improvement</div>
</div>
<div class="stat-card">
<div class="value">2016–2025</div>
<div class="label">Collection Period</div>
</div>
<div class="stat-card">
<div class="value">p&lt;0.001</div>
<div class="label">Statistical Significance</div>
</div>
<div class="stat-card">
<div class="value">k≥5</div>
<div class="label">k-Anonymity (GDPR)</div>
</div>
</div>
<!-- VPER Metric -->
<div class="section">
<h2>VPER Metric — Visualization Performance Enhancement Rate</h2>
<div class="vper-highlight">
<div class="formula">VPER = (E_tabular − E_visual) / E_tabular = 27.9%</div>
<p>Where E = error rate (1 − accuracy). Visual traces reduce AI diagnostic error by 27.9% over tabular query statistics alone.</p>
</div>
<table>
<thead><tr><th>Condition</th><th>Error Rate</th><th>Accuracy</th><th>Improvement</th></tr></thead>
<tbody>
<tr>
<td>Tabular stats only (baseline)</td>
<td>64.3%</td>
<td>35.7%</td>
<td>—</td>
</tr>
<tr>
<td><strong>+ PlanViz visual trace</strong></td>
<td><strong>46.4%</strong></td>
<td><strong>53.6%</strong></td>
<td style="color:#16a34a; font-weight:600;">+17.9pp (VPER=27.9%)</td>
</tr>
</tbody>
</table>
<div style="margin-top:16px;">
<div style="margin-bottom:8px; font-size:0.9rem; color:#475569;">Tabular baseline accuracy</div>
<div class="result-bar"><div class="result-bar-fill" style="width:35.7%; background:#94a3b8;">35.7%</div></div>
<div style="margin-top:10px; margin-bottom:8px; font-size:0.9rem; color:#475569;">Visual trace accuracy</div>
<div class="result-bar"><div class="result-bar-fill" style="width:53.6%; background:#1a73e8;">53.6%</div></div>
</div>
</div>
<!-- AI-VPER Protocol -->
<div class="section">
<h2>AI-VPER Benchmark Protocol</h2>
<p style="color:#475569; margin-bottom:20px;">Standardized evaluation protocol for measuring multimodal AI performance on SQL plan debugging tasks.</p>
<div class="protocol-step">
<div class="step-num">1</div>
<div class="step-content">
<strong>Tabular Baseline (E_tabular)</strong>
<p>Present AI model with tabular query statistics only: runtime, rows processed, memory usage, CPU time. Record diagnostic accuracy on 192 annotated bug reports.</p>
</div>
</div>
<div class="protocol-step">
<div class="step-num">2</div>
<div class="step-content">
<strong>Visual Augmentation (E_visual)</strong>
<p>Present the same AI model with the PlanViz SQL execution plan visualization attached to each bug report. Record diagnostic accuracy under identical conditions.</p>
</div>
</div>
<div class="protocol-step">
<div class="step-num">3</div>
<div class="step-content">
<strong>VPER Computation</strong>
<p>Compute VPER = (E_tabular − E_visual) / E_tabular. Positive VPER indicates visualization improves diagnosis. Report with 95% confidence intervals and paired t-test.</p>
</div>
</div>
<div class="protocol-step">
<div class="step-num">4</div>
<div class="step-content">
<strong>Human Baseline Calibration</strong>
<p>Compare AI-VPER against human expert VPER on the same 192 bugs to measure the gap between AI and human visual reasoning over SQL execution traces.</p>
</div>
</div>
</div>
<!-- Dataset Info -->
<div class="section">
<h2>Dataset Details</h2>
<table>
<thead><tr><th>Property</th><th>Value</th></tr></thead>
<tbody>
<tr><td>Total bug reports</td><td>192</td></tr>
<tr><td>Source system</td><td>Commercial column-store RDBMS (enterprise production)</td></tr>
<tr><td>Collection period</td><td>2016–2025 (10 years)</td></tr>
<tr><td>Annotation type</td><td>Root cause label + diagnostic difficulty + resolution category</td></tr>
<tr><td>Privacy protection</td><td>k-anonymity ≥ 5; customer identifiers removed; SQL generalized</td></tr>
<tr><td>License</td><td>CC BY 4.0</td></tr>
<tr><td>Format</td><td>CSV + XLSX</td></tr>
</tbody>
</table>
<div style="margin-top:16px;">
<span class="tag">SQL Performance</span>
<span class="tag">Regression Detection</span>
<span class="tag">Plan Visualization</span>
<span class="tag">Multimodal AI</span>
<span class="tag">Enterprise Software</span>
<span class="tag">Benchmarking</span>
<span class="tag">GDPR-compliant</span>
</div>
</div>
<!-- Links -->
<div class="section">
<h2>Access &amp; Citation</h2>
<a class="btn" href="https://huggingface.co/datasets/viperoptic/VisualTraceBench" target="_blank">📦 Download Dataset</a>
<a class="btn secondary" href="https://huggingface.co/datasets/viperoptic/VisualTraceBench/resolve/main/data/VisualTraceBench_v1.0.csv" target="_blank">⬇ Direct CSV Download</a>
<div style="margin-top:20px; background:#f8fafc; border-radius:8px; padding:16px; font-family:monospace; font-size:0.85rem; color:#374151;">
@dataset{visualtracebench2026,<br>
&nbsp;&nbsp;title = {VisualTraceBench: A Human-Performance Benchmark for Multimodal AI SQL Plan Debugging},<br>
&nbsp;&nbsp;author = {Anonymous},<br>
&nbsp;&nbsp;year = {2026},<br>
&nbsp;&nbsp;url = {https://huggingface.co/datasets/viperoptic/VisualTraceBench}<br>
}
</div>
</div>
</main>
<footer>
<p>VisualTraceBench · Under review at ICLR 2027 Datasets &amp; Benchmarks Track · CC BY 4.0</p>
<p style="margin-top:6px;">Dataset: <a href="https://huggingface.co/datasets/viperoptic/VisualTraceBench" style="color:#1a73e8;">huggingface.co/datasets/viperoptic/VisualTraceBench</a></p>
</footer>
</body>
</html>