Blame for index.html

<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Self-Healing Text-to-SQL</title>
<style>

:root{

  --bg:#f7f6f3; --card:#ffffff; --ink:#1a1a1a; --muted:#5f5c57; --line:#e4e1db;

  --accent:#b4552d; --good:#2d7a4f; --bad:#a63d3d; --code:#f0eee9;

}

@media (prefers-color-scheme: dark){ :root:not([data-theme="light"]){

  --bg:#171614; --card:#201f1c; --ink:#ece9e4; --muted:#a39e96; --line:#35332e;

  --accent:#e08050; --good:#5fbf8a; --bad:#d97f7f; --code:#2a2925;

}}

:root[data-theme="dark"]{

  --bg:#171614; --card:#201f1c; --ink:#ece9e4; --muted:#a39e96; --line:#35332e;

  --accent:#e08050; --good:#5fbf8a; --bad:#d97f7f; --code:#2a2925;

}

*{box-sizing:border-box}

body{margin:0;background:var(--bg);color:var(--ink);

  font:16px/1.6 Georgia,'Times New Roman',serif;}

.wrap{max-width:860px;margin:0 auto;padding:48px 16px 80px}

h1{font-size:2rem;line-height:1.2;margin:0 0 4px}

h2{font-size:1.25rem;margin:40px 0 12px;border-bottom:1px solid var(--line);padding-bottom:6px}

.sub{color:var(--muted);margin:0 0 28px}

.badge{display:inline-block;background:var(--accent);color:#fff;border-radius:4px;

  padding:1px 8px;font:600 .75rem/1.6 system-ui,sans-serif;vertical-align:middle}

p{margin:12px 0}

pre{background:var(--code);border:1px solid var(--line);border-radius:8px;

  padding:14px;overflow-x:auto;font:13px/1.5 ui-monospace,Consolas,monospace}

table{border-collapse:collapse;width:100%;font:14px/1.5 system-ui,sans-serif;margin:16px 0}

th,td{border:1px solid var(--line);padding:7px 10px;text-align:left}

th{background:var(--code)}

td.num{text-align:right;font-variant-numeric:tabular-nums}

.good{color:var(--good);font-weight:600}.bad{color:var(--bad);font-weight:600}

a{color:var(--accent)}

ul{padding-left:22px}

.card{background:var(--card);border:1px solid var(--line);border-radius:10px;

  padding:4px 18px;margin:18px 0}

.foot{color:var(--muted);font-size:.85rem;margin-top:48px;border-top:1px solid var(--line);padding-top:14px}

</style>
</head>
<body>
<div class="wrap">
<h1>Self-Healing Text-to-SQL <span class="badge">PS #8</span></h1>
<p class="sub">Lexsi Labs Hackathon · Track 3 (Agentic Fine-tuning) · AgentTune + AuditKIT ·
Cohere Labs open models</p>

<p>A fully automated self-healing loop for a <strong>Tiny Aya Global (3.35B)</strong>
text-to-SQL agent on Spider: failures are detected by <em>executing</em> the SQL,
classified into SQL root causes, corrected by <strong>Aya Expanse 32B</strong>
(every fix verified by execution before it becomes training data; the healer never
sees the gold query), retrained adapter-only through <strong>AgentTune</strong>, and
deployment is decided by <strong>AgentTune's DeploymentGate</strong> on a held-out
split. Outcomes independently re-verified with <strong>AuditKIT</strong>.
No human wrote or fixed a single SQL query.</p>

<h2>The loop</h2>
<pre>
student (Tiny Aya 3.35B) writes SQL ──▶ execution checker runs it vs gold ──▶ failures
     ▲                                                                          │
     │                                        classify (SQL taxonomy, Aya Expanse 32B)
 LoRA adapter (AgentTune                                                        │
 DPO / BCO retrain)                     generate fix (Aya Expanse 32B, never sees gold)
     ▲                                                                          │
     │                                        execute fix ── correct? keep : retry ≤3
 158 execution-verified pairs ◀─────────────────────────────────────────────────┘
     │
 AgentTune DeploymentGate (held-out gate split) ──▶ deploy / keep old
     │
 final report on unseen test split (+ AuditKIT re-verification)
</pre>

<h2>Headline results</h2>
<div class="card">
<p><strong>The loop ran end to end, and the gate mattered twice.</strong>
158 of 253 pool failures (62.5%) became execution-verified corrective training
pairs. The DeploymentGate correctly <span class="bad">BLOCKED</span> four regressing
adapters (naive DPO hits likelihood displacement on minimal-edit SQL pairs, plus
catastrophic forgetting), then <span class="good">APPROVED</span> the fifth
(BCO&nbsp;+&nbsp;success-replay) on a 400-question held-out gate. The deployed
adapter improves the unseen test split <strong>45.6% → 47.6%</strong> execution
accuracy and <strong>68.2% → 73.0%</strong> valid-SQL rate.</p>
</div>

<table>
<tr><th>Stage</th><th>Metric</th><th class="num">Value</th></tr>
<tr><td>Baseline, <strong>test (500, unseen)</strong></td><td>execution accuracy</td><td class="num">45.6%</td></tr>
<tr><td>Heal stage</td><td>verified-fix yield</td><td class="num good">158 / 253 (62.5%)</td></tr>
<tr><td>DPO adapter, gate (n=100)</td><td>execution accuracy</td><td class="num">58.0% vs 62.0% <span class="bad">BLOCKED</span></td></tr>
<tr><td>BCO + replay, gate (n=400)</td><td>execution accuracy</td><td class="num">59.25% vs 59.0% <span class="good">APPROVED</span></td></tr>
<tr><td><strong>BCO + replay, test (500, unseen)</strong></td><td>execution accuracy</td><td class="num good"><strong>47.6% (+2.0pp, fixed 34 / broke 24)</strong></td></tr>
<tr><td>BCO + replay, test</td><td>valid-SQL rate</td><td class="num good">73.0% (vs 68.2%)</td></tr>
<tr><td>AuditKIT re-verification, test ×2</td><td>agreement with harness</td><td class="num good">500 / 500 both runs</td></tr>
</table>
<p>Full before/after tables (per-status breakdowns, McNemar exact tests, all five
adapters, gate decision JSONs) are in the article and the runs repo below.</p>

<h2>Artifacts</h2>
<ul>
<li>Dataset (splits + 149 Spider DBs):
  <a href="https://huggingface.co/datasets/HideIron/selfheal-sql-spider">HideIron/selfheal-sql-spider</a></li>
<li>Every run log, heal pairs, gate decisions, metrics:
  <a href="https://huggingface.co/datasets/HideIron/selfheal-sql-runs">HideIron/selfheal-sql-runs</a></li>
<li>Adapters:
  <a href="https://huggingface.co/HideIron/tiny-aya-global-sql-selfheal">DPO</a> ·
  <a href="https://huggingface.co/HideIron/tiny-aya-global-sql-selfheal-bco">BCO</a> ·
  <a href="https://huggingface.co/HideIron/tiny-aya-global-sql-selfheal-bco-replay">BCO&nbsp;+&nbsp;replay</a></li>
<li>Code: browse the <em>Files</em> tab of this Space (<code>src/selfheal_sql/</code>);
  reproduce instructions in <code>README.md</code></li>
</ul>

<h2>Why the gate is the point</h2>
<p>PS #8 asks for redeployment <em>gated on measured accuracy</em>. Preference-tuning
on minimal-edit SQL pairs hits <em>likelihood displacement</em> (chosen and rejected
share ~90% of tokens; pushing the rejected down drags the chosen with it), and
training only on failures causes forgetting of the 347 already-correct answers. The
gate caught every resulting regression on held-out data and refused to ship. Then,
once BCO + success-replay removed both causes and the gate was enlarged to 400
questions for statistical power, it approved deployment, and the test split confirmed
the gain on databases nothing in the loop ever saw. The gate's decisions ranked the
adapters correctly on unseen data. Full analysis in the article.</p>

<div class="foot">Solo project · models: Tiny Aya Global (student) and Aya Expanse 32B
(healer), both open-weights Cohere Labs models from the hackathon's allowed list ·
Spider is CC BY-SA 4.0 · AgentTune/AuditKIT © Lexsi Labs (DeploymentGate vendored
verbatim @ 1dfd054, provenance in <code>third_party/</code>).</div>
</div>
</body>
</html>