Laborator commited on
Commit
ae2e712
·
verified ·
1 Parent(s): ab717c1

Align museum with the canonical n=14 run: arc 50.0 -> 64.3 -> 71.4 -> 71.4 -> 85.7 -> 100.0

Browse files
Files changed (1) hide show
  1. README.md +33 -20
README.md CHANGED
@@ -1,8 +1,8 @@
1
  ---
2
- title: 2 · External Grounding
3
- emoji: 🔁
4
- colorFrom: indigo
5
- colorTo: purple
6
  sdk: static
7
  app_file: index.html
8
  pinned: true
@@ -16,38 +16,51 @@ Interactive visualization of Experiment 2–3 (the *guardian*) of the
16
  [Second Loop](https://github.com/SergheiBrinza/external-grounding) project.
17
 
18
  This Space loads **no model**. Everything is a static page driven by `data.json`
19
- — the verbatim output of the original experimental run.
20
 
21
  ## The exhibit
22
 
23
- A frozen Qwen2.5-3B-Instruct has a confidently memorized **wrong** answer to twelve
24
  questions, and its correction notebook is fed from a **noisy** source (some verified
25
  facts, some unreliable look-alikes). Drag the lever through six guardian versions and
26
  watch the share of correct answers climb:
27
 
28
  | stage | guardian | corrected |
29
  |---|---|---|
30
- | sick | no defense | 50.0% · 6/12 |
31
- | 1.0 | same-family clone arbiter | 66.7% · 8/12 |
32
- | 2.0 | live Wikipedia retrieval | 66.7% · 8/12 |
33
- | 2.1 | more retrieval | 66.7% · 8/12 |
34
- | 2.2 | three targeted fixes | 91.7% · 11/12 |
35
- | 2.3 | final calibration | 100% · 12/12 |
36
 
37
  ## What the numbers say (the honest middle)
38
 
39
- - **The 66.7% plateau is real.** Three different guardians (1.0, 2.0, 2.1) all stop at
40
- the same ceiling. Guardian 1.0's clone arbiter shares the subject's blind spots.
41
- - **The plateau is not stagnation — it's churn.** Each step fixes some traps while
42
- breaking others (the readout shows `+fixed / −broken`); net change is zero across the
43
- plateau.
 
 
 
 
 
 
 
44
  - **Several traps regress before they settle.** Venus (#46) goes
45
  `correct → wrong → correct → wrong → wrong → correct` across the six stages — the path
46
  to 100% is not monotonic, and that is shown openly, not smoothed over.
47
 
48
- Only Guardian 2.2 (verbatim-quote check, namesake relevance gate, soft threshold) breaks
49
- the ceiling at 91.7%, and Guardian 2.3 (calibration) closes it at 100%. An independent
50
- Qwen2.5-7B reader/judge with Wikipedia adjudicated the v2 stages.
 
 
 
 
 
 
51
 
52
  ## Data and attribution
53
 
 
1
  ---
2
+ title: Second Loop · 2 · External Grounding
3
+ emoji: 🛡️
4
+ colorFrom: gray
5
+ colorTo: green
6
  sdk: static
7
  app_file: index.html
8
  pinned: true
 
16
  [Second Loop](https://github.com/SergheiBrinza/external-grounding) project.
17
 
18
  This Space loads **no model**. Everything is a static page driven by `data.json`
19
+ — the verbatim output of the original experimental run, all 14 working traps of it.
20
 
21
  ## The exhibit
22
 
23
+ A frozen Qwen2.5-3B-Instruct has a confidently memorized **wrong** answer to fourteen
24
  questions, and its correction notebook is fed from a **noisy** source (some verified
25
  facts, some unreliable look-alikes). Drag the lever through six guardian versions and
26
  watch the share of correct answers climb:
27
 
28
  | stage | guardian | corrected |
29
  |---|---|---|
30
+ | sick | no defense | 50.0% · 7/14 |
31
+ | 1.0 | same-family clone arbiter | 64.3% · 9/14 |
32
+ | 2.0 | live Wikipedia retrieval | 71.4% · 10/14 |
33
+ | 2.1 | brute strengthening | 71.4% · 10/14 |
34
+ | 2.2 | three targeted fixes | 85.7% · 12/14 |
35
+ | 2.3 | final calibration | 100% · 14/14 |
36
 
37
  ## What the numbers say (the honest middle)
38
 
39
+ - **The clone ceiling equals the arbiter's own accuracy.** Guardian 1.0 caps at
40
+ 64.3% (9/14) — exactly the share of traps on which the 7B arbiter itself kept the
41
+ correct fact (also 9/14). A verifier cannot lift a system above its own knowledge,
42
+ and a same-family clone shares the subject's blind spots by construction.
43
+ - **A raw external source cures the clone's blind spots.** Swapping the clone for live
44
+ Wikipedia lifts 64.3% → 71.4%, and it does so on precisely the facts the clone got
45
+ wrong — Mauna Kea, and the fruit-flies-before-Laika trap (#37).
46
+ - **Brute strengthening does nothing.** Guardian 2.1 is a generic "make retrieval
47
+ stronger" pass and moves the score not at all: 71.4% → 71.4%. This is the one flat
48
+ step in the arc. It is not idle underneath — it fixes some traps and breaks others
49
+ (the readout shows `+fixed / −broken`) — but the net is zero. What works is targeted
50
+ operating-point engineering, not more force.
51
  - **Several traps regress before they settle.** Venus (#46) goes
52
  `correct → wrong → correct → wrong → wrong → correct` across the six stages — the path
53
  to 100% is not monotonic, and that is shown openly, not smoothed over.
54
 
55
+ Only Guardian 2.2 (verbatim-quote check, namesake relevance gate, soft threshold) gets
56
+ past the flat step, to 85.7%, and Guardian 2.3 (calibration) closes it at 100%. An
57
+ independent Qwen2.5-7B reader/judge with Wikipedia adjudicated the v2 stages.
58
+
59
+ **On the 100%.** The subject model is frozen, the notebook is a static file on disk and
60
+ decoding is greedy, so a rerun reproduces an identical state. The final 100% is partly a
61
+ property of that determinism, and it was reached with one trap (#46) reformulated
62
+ mid-arc after the external source revealed it was ambiguous between the sidereal and
63
+ solar day. What we stand behind is the failure-and-repair arc, not the round number.
64
 
65
  ## Data and attribution
66