Spaces:
Running
Running
Align museum with the canonical n=14 run: arc 50.0 -> 64.3 -> 71.4 -> 71.4 -> 85.7 -> 100.0
Browse files
README.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
| 1 |
---
|
| 2 |
-
title: 2 · External Grounding
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: static
|
| 7 |
app_file: index.html
|
| 8 |
pinned: true
|
|
@@ -16,38 +16,51 @@ Interactive visualization of Experiment 2–3 (the *guardian*) of the
|
|
| 16 |
[Second Loop](https://github.com/SergheiBrinza/external-grounding) project.
|
| 17 |
|
| 18 |
This Space loads **no model**. Everything is a static page driven by `data.json`
|
| 19 |
-
— the verbatim output of the original experimental run.
|
| 20 |
|
| 21 |
## The exhibit
|
| 22 |
|
| 23 |
-
A frozen Qwen2.5-3B-Instruct has a confidently memorized **wrong** answer to
|
| 24 |
questions, and its correction notebook is fed from a **noisy** source (some verified
|
| 25 |
facts, some unreliable look-alikes). Drag the lever through six guardian versions and
|
| 26 |
watch the share of correct answers climb:
|
| 27 |
|
| 28 |
| stage | guardian | corrected |
|
| 29 |
|---|---|---|
|
| 30 |
-
| sick | no defense | 50.0% ·
|
| 31 |
-
| 1.0 | same-family clone arbiter |
|
| 32 |
-
| 2.0 | live Wikipedia retrieval |
|
| 33 |
-
| 2.1 |
|
| 34 |
-
| 2.2 | three targeted fixes |
|
| 35 |
-
| 2.3 | final calibration | 100% ·
|
| 36 |
|
| 37 |
## What the numbers say (the honest middle)
|
| 38 |
|
| 39 |
-
- **The
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
- **Several traps regress before they settle.** Venus (#46) goes
|
| 45 |
`correct → wrong → correct → wrong → wrong → correct` across the six stages — the path
|
| 46 |
to 100% is not monotonic, and that is shown openly, not smoothed over.
|
| 47 |
|
| 48 |
-
Only Guardian 2.2 (verbatim-quote check, namesake relevance gate, soft threshold)
|
| 49 |
-
the
|
| 50 |
-
Qwen2.5-7B reader/judge with Wikipedia adjudicated the v2 stages.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
|
| 52 |
## Data and attribution
|
| 53 |
|
|
|
|
| 1 |
---
|
| 2 |
+
title: Second Loop · 2 · External Grounding
|
| 3 |
+
emoji: 🛡️
|
| 4 |
+
colorFrom: gray
|
| 5 |
+
colorTo: green
|
| 6 |
sdk: static
|
| 7 |
app_file: index.html
|
| 8 |
pinned: true
|
|
|
|
| 16 |
[Second Loop](https://github.com/SergheiBrinza/external-grounding) project.
|
| 17 |
|
| 18 |
This Space loads **no model**. Everything is a static page driven by `data.json`
|
| 19 |
+
— the verbatim output of the original experimental run, all 14 working traps of it.
|
| 20 |
|
| 21 |
## The exhibit
|
| 22 |
|
| 23 |
+
A frozen Qwen2.5-3B-Instruct has a confidently memorized **wrong** answer to fourteen
|
| 24 |
questions, and its correction notebook is fed from a **noisy** source (some verified
|
| 25 |
facts, some unreliable look-alikes). Drag the lever through six guardian versions and
|
| 26 |
watch the share of correct answers climb:
|
| 27 |
|
| 28 |
| stage | guardian | corrected |
|
| 29 |
|---|---|---|
|
| 30 |
+
| sick | no defense | 50.0% · 7/14 |
|
| 31 |
+
| 1.0 | same-family clone arbiter | 64.3% · 9/14 |
|
| 32 |
+
| 2.0 | live Wikipedia retrieval | 71.4% · 10/14 |
|
| 33 |
+
| 2.1 | brute strengthening | 71.4% · 10/14 |
|
| 34 |
+
| 2.2 | three targeted fixes | 85.7% · 12/14 |
|
| 35 |
+
| 2.3 | final calibration | 100% · 14/14 |
|
| 36 |
|
| 37 |
## What the numbers say (the honest middle)
|
| 38 |
|
| 39 |
+
- **The clone ceiling equals the arbiter's own accuracy.** Guardian 1.0 caps at
|
| 40 |
+
64.3% (9/14) — exactly the share of traps on which the 7B arbiter itself kept the
|
| 41 |
+
correct fact (also 9/14). A verifier cannot lift a system above its own knowledge,
|
| 42 |
+
and a same-family clone shares the subject's blind spots by construction.
|
| 43 |
+
- **A raw external source cures the clone's blind spots.** Swapping the clone for live
|
| 44 |
+
Wikipedia lifts 64.3% → 71.4%, and it does so on precisely the facts the clone got
|
| 45 |
+
wrong — Mauna Kea, and the fruit-flies-before-Laika trap (#37).
|
| 46 |
+
- **Brute strengthening does nothing.** Guardian 2.1 is a generic "make retrieval
|
| 47 |
+
stronger" pass and moves the score not at all: 71.4% → 71.4%. This is the one flat
|
| 48 |
+
step in the arc. It is not idle underneath — it fixes some traps and breaks others
|
| 49 |
+
(the readout shows `+fixed / −broken`) — but the net is zero. What works is targeted
|
| 50 |
+
operating-point engineering, not more force.
|
| 51 |
- **Several traps regress before they settle.** Venus (#46) goes
|
| 52 |
`correct → wrong → correct → wrong → wrong → correct` across the six stages — the path
|
| 53 |
to 100% is not monotonic, and that is shown openly, not smoothed over.
|
| 54 |
|
| 55 |
+
Only Guardian 2.2 (verbatim-quote check, namesake relevance gate, soft threshold) gets
|
| 56 |
+
past the flat step, to 85.7%, and Guardian 2.3 (calibration) closes it at 100%. An
|
| 57 |
+
independent Qwen2.5-7B reader/judge with Wikipedia adjudicated the v2 stages.
|
| 58 |
+
|
| 59 |
+
**On the 100%.** The subject model is frozen, the notebook is a static file on disk and
|
| 60 |
+
decoding is greedy, so a rerun reproduces an identical state. The final 100% is partly a
|
| 61 |
+
property of that determinism, and it was reached with one trap (#46) reformulated
|
| 62 |
+
mid-arc after the external source revealed it was ambiguous between the sidereal and
|
| 63 |
+
solar day. What we stand behind is the failure-and-repair arc, not the round number.
|
| 64 |
|
| 65 |
## Data and attribution
|
| 66 |
|