card: generate candidates and let the verifier pick (+2.31 pp); majority voting measured as a loss

README.md

@@ -192,11 +192,47 @@ The score alone is not actionable, so the response carries the measured conseque
 | **Escalate the lowest 20% to a stronger model** | **+1.34 pp** end-to-end |
 | **Send the lowest 20% to human review** | catches **47.2%** of all errors — 2.4× random |
 
+### Generate several candidates and let the verifier pick
+
+When the score is low, the most effective next step is not to escalate — it is to **produce more
+candidate answers and select between them**. Measured on 346 questions with five candidates each
+(1,730 candidates), all scored through this endpoint:
+
+| Policy | Accuracy | Output tokens | vs. one attempt |
+|---|---|---|---|
+| One attempt | 49.13% | 72 | — |
+| Majority vote over 3 | 46.82% | 212 | **−2.31 pp** |
+| Majority vote over 5 | 46.53% | 353 | **−2.60 pp** |
+| **Pick best of 5 with this model** | **51.45%** | 353 | **+2.31 pp** |
+| Pick best of 5 with JEV | 53.47% | 353 | +4.34 pp |
+| Pick one of 5 at random | 46.82% | 353 | −2.31 pp |
+| *Oracle — any correct candidate counts* | *63.87%* | *353* | *+14.74 pp* |
+
+**The same five candidates swing by 5 points depending on how one is chosen.** Majority voting is
+worse than not resampling at all: when a model prefers a wrong answer, more samples make that wrong
+consensus more certain. A verifier that ranks the candidates is what turns extra samples into
+accuracy.
+
+**Spend the budget only where it is needed.** Generating extra candidates only for low-scoring
+first attempts captures most of the gain at a fraction of the cost:
+
+| Triggered on | Accuracy | Output tokens | vs. one attempt |
+|---|---|---|---|
+| 10% of items | 49.71% | 80 | +0.58 pp |
+| **30% of items** | **50.87%** | **132** | **+1.73 pp** |
+| 100% of items | 51.45% | 353 | +2.31 pp |
+
+**At a 30% trigger rate you get three quarters of the benefit for 1.8× the tokens**, where always
+generating costs 4.9× for 1.3× the benefit.
+
+*Scope: one generator (GPT-4o-mini), one item set, five candidates. The oracle row shows the
+headroom that remains — a correct candidate is present far more often than any policy recovers it.*
+
 **One thing that does not work:**
 
-🔴 **Do not resample the same model.** Measured: re-drawing and taking a 3-way majority fixes
-**6.7%** of wrong answers while breaking **13.1%** of right ones. It is a net loss, and it is the
-first thing most pipelines try.
+🔴 **Do not take a majority vote over resamples.** Measured: five resamples with majority voting score **46.53%** where a single attempt scores **49.13%**. More candidates make a wrong consensus more certain unless something picks between them — see the table above.
+
+
 
 *Escalation pays for itself through precision, not recall. Re-answering repairs about 38% of wrong
 answers and damages about 30% of right ones, so a gate is only worth its budget if it mostly calls