README.md

metadata

title: Second Loop · 2 · External Grounding
emoji: 🛡️
colorFrom: gray
colorTo: green
sdk: static
app_file: index.html
pinned: true
license: mit
short_description: Lifting self-correction 50%→92.9% under a noisy notebook

External Grounding — interactive demo

Interactive visualization of Experiment 2–3 (the guardian) of the Second Loop project.

This Space loads no model. Everything is a static page driven by data.json — the verbatim output of the original experimental run, all 14 working traps of it.

The exhibit

A frozen Qwen2.5-3B-Instruct has a confidently memorized wrong answer to fourteen questions, and its correction notebook is fed from a noisy source (some verified facts, some unreliable look-alikes). Drag the lever through six guardian versions and watch the share of correct answers climb:

stage	guardian	corrected
sick	no defense	50.0% · 7/14
1.0	same-family clone arbiter	64.3% · 9/14
2.0	live Wikipedia retrieval	71.4% · 10/14
2.1	brute strengthening	71.4% · 10/14
2.2	three targeted fixes	85.7% · 12/14
2.3	final calibration	100% · 14/14 †

† The final 100% includes reformulating one genuinely ambiguous trap (#46). Without it, 92.9% · 13/14.

What the numbers say (the honest middle)

• The clone ceiling equals the arbiter's own accuracy. Guardian 1.0 caps at 64.3% (9/14) — exactly the share of traps on which the 7B arbiter itself kept the correct fact (also 9/14). A verifier cannot lift a system above its own knowledge, and a same-family clone shares the subject's blind spots by construction.
• A raw external source cures the clone's blind spots. Swapping the clone for live Wikipedia lifts 64.3% → 71.4%, and it does so on precisely the facts the clone got wrong — Mauna Kea, and the fruit-flies-before-Laika trap (#37).
• Brute strengthening does nothing. Guardian 2.1 is a generic "make retrieval stronger" pass and moves the score not at all: 71.4% → 71.4%. This is the one flat step in the arc. It is not idle underneath — it fixes some traps and breaks others (the readout shows +fixed / −broken) — but the net is zero. What works is targeted operating-point engineering, not more force.
• Several traps regress before they settle. Venus (#46) goes correct → wrong → correct → wrong → wrong → correct across the six stages — the path to 100% is not monotonic, and that is shown openly, not smoothed over.

Only Guardian 2.2 (verbatim-quote check, namesake relevance gate, soft threshold) gets past the flat step, to 85.7%, and Guardian 2.3 (calibration) closes it at 100%. An independent Qwen2.5-7B reader/judge with Wikipedia adjudicated the v2 stages.

On the 100%. The subject model is frozen, the notebook is a static file on disk and decoding is greedy, so a rerun reproduces an identical state. The final 100% is partly a property of that determinism, and it was reached with one trap (#46) reformulated mid-arc after the external source revealed it was ambiguous between the sidereal and solar day. What we stand behind is the failure-and-repair arc, not the round number.

Data and attribution

Subject model Qwen2.5-3B-Instruct; arbiters Qwen2.5-7B-Instruct (same-family clone) and Wikipedia retrieval + 7B reader/judge (both Apache-2.0, Alibaba Cloud). Wikipedia content © its authors (CC BY-SA). Run on a single RTX 3090 Ti. No model weights are redistributed here — only aggregate verdicts and counts. Demo code and data: MIT.

Source code, raw per-stage JSON results, and methodology document: https://github.com/SergheiBrinza/external-grounding