Add workshop paper: Compute Is Not Neutral
Browse files- reports/occ_paper.md +234 -0
reports/occ_paper.md
ADDED
|
@@ -0,0 +1,234 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Compute Is Not Neutral: Mechanism Analysis of Adversarial Debate Collapse and the OCC Stack for Governed Agent Resource Allocation
|
| 2 |
+
|
| 3 |
+
## Abstract
|
| 4 |
+
|
| 5 |
+
Multi-agent debate systems are increasingly used for LLM reasoning, but they treat compute as a free good — every agent gets equal turns regardless of contribution. We show that this creates an exploitable attack surface: in 4-agent debate (3 honest + 1 adversarial) on Qwen3-Coder-30B, majority-vote accuracy collapses from 73.3% (1 round) to 56.7% (3 rounds) — **worse than single-round voting** — because the adversarial agent's extra turns amplify into extra votes. Through mechanism isolation (7 conditions, 30 topics), we identify the root cause as **volume amplification**, not persuasion: honest agents retain their positions at 84% but get outvoted. Simple protocol fixes (LLM judge voting, confidence-weighted voting) fully recover accuracy to 73.3%. We then present OCC (Oracle-Credit-Compute), a mechanism-design layer where agents earn non-transferable, decaying, capability-scoped credits based on verified marginal impact. We evaluate OCC on three benchmarks (code generation, retrieval QA, multi-agent debate), test 10 anti-gaming attacks (all contained), and provide a GRPO-compatible reward hook for learned allocation. We honestly report where OCC succeeds (preventing catastrophic collapse, anti-gaming) and where it does not (matching random gating at moderate budgets, no policy improvement at 0.5B scale).
|
| 6 |
+
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
## 1. Introduction
|
| 10 |
+
|
| 11 |
+
Modern LLM agent systems allocate compute indiscriminately: every agent gets equal debate turns, unlimited retries, and unconstrained tool access. This implicitly assumes compute is neutral — that more turns, tokens, and tool calls can only help.
|
| 12 |
+
|
| 13 |
+
**This assumption is wrong.** We show that in multi-agent debate with one adversarial agent, giving all agents equal speaking turns across 3 rounds causes accuracy to collapse by 16.7 percentage points relative to a single-round baseline. The mechanism is not persuasion — honest agents keep their original answers 84% of the time — but **volume amplification**: the adversary's extra turns become extra votes in the majority pool, tipping close-call topics.
|
| 14 |
+
|
| 15 |
+
This finding motivates OCC (Oracle-Credit-Compute), a mechanism-design layer that treats agent compute as a scarce, earned, auditable privilege rather than a right. OCC has four components: (1) an Impact Oracle that scores whether actions produce verified marginal value, (2) a Credit Ledger with non-transferable, decaying, capability-scoped credits, (3) a Resource Broker that grants capability-based access, and (4) a GRPO-compatible reward hook for learned allocation.
|
| 16 |
+
|
| 17 |
+
**Contributions:**
|
| 18 |
+
1. **Empirical finding:** Multi-round debate collapses under adversarial pressure (§3). Mechanism isolation shows volume amplification, not persuasion, is the cause (§4).
|
| 19 |
+
2. **Protocol fixes:** Judge voting and confidence weighting fully recover accuracy (§4.3).
|
| 20 |
+
3. **OCC system:** Open-source stack with formal definition, anti-gaming threat model, and GRPO hook (§5).
|
| 21 |
+
4. **Three benchmarks:** Code generation (52.3% compute savings, simulated), retrieval QA, and multi-agent debate (§6).
|
| 22 |
+
5. **Anti-gaming evaluation:** 10 attack vectors tested, all contained by ledger design (§7).
|
| 23 |
+
6. **Honest limitations:** OCC ≈ random gating at moderate budgets; GRPO training produced no improvement at 0.5B scale (§8).
|
| 24 |
+
|
| 25 |
+
---
|
| 26 |
+
|
| 27 |
+
## 2. Related Work
|
| 28 |
+
|
| 29 |
+
**Test-time compute allocation.** Snell et al. (2024) showed compute-optimal scaling outperforms 14× larger models. Brown et al. (2024) showed coverage scales log-linearly with samples. OCC extends this from per-prompt to per-agent allocation.
|
| 30 |
+
|
| 31 |
+
**Multi-agent debate.** Du et al. (2023) introduced multi-agent debate for reasoning. Subramanian et al. (2025) found majority voting drives most gains. We show that majority voting itself is the vulnerability under adversarial conditions.
|
| 32 |
+
|
| 33 |
+
**GRPO/RLVR.** DeepSeek-R1 (2025) uses GRPO with rule-based rewards. OCC's reward hook is GRPO-compatible but adds compute-cost penalties.
|
| 34 |
+
|
| 35 |
+
**Calibration and abstention.** RLCR (2025) uses Brier score for calibration rewards. OCC integrates calibration as a first-class reward component.
|
| 36 |
+
|
| 37 |
+
**Capability-based security.** AgentGuardian (2026) and SAGA (2025) propose access control for agents. OCC dynamically earns capabilities through verified impact rather than static assignment.
|
| 38 |
+
|
| 39 |
+
Full literature review in `reports/literature_review.md`.
|
| 40 |
+
|
| 41 |
+
---
|
| 42 |
+
|
| 43 |
+
## 3. The Collapse
|
| 44 |
+
|
| 45 |
+
### 3.1 Setup
|
| 46 |
+
|
| 47 |
+
- **Model:** Qwen3-Coder-30B-A3B-Instruct (MoE, 30B total, ~3B active)
|
| 48 |
+
- **Topics:** 30 yes/no scientific fact questions (CS, physics, biology, math)
|
| 49 |
+
- **Agents:** 4 total — 3 honest, 1 adversarial (argues wrong answer)
|
| 50 |
+
- **Protocol:** Each agent generates a YES/NO answer with explanation. Majority vote determines group answer.
|
| 51 |
+
- **Rounds:** 1 round (baseline) vs 3 rounds (collapse condition)
|
| 52 |
+
- **Seed:** 42 (single seed, n=30 topics, 95% CI ±16pp)
|
| 53 |
+
|
| 54 |
+
### 3.2 Result
|
| 55 |
+
|
| 56 |
+
| Condition | Accuracy | Δ from Baseline |
|
| 57 |
+
|-----------|:--------:|:---------------:|
|
| 58 |
+
| 1-round majority vote | **73.3%** (22/30) | — |
|
| 59 |
+
| 3-round majority vote | **56.7%** (17/30) | **−16.7pp** |
|
| 60 |
+
|
| 61 |
+
Three rounds of debate — 3× the compute — produces performance **16.7 percentage points worse** than a single round. The adversarial agent, given equal speaking turns, converts extra rounds into influence over the vote pool.
|
| 62 |
+
|
| 63 |
+
### 3.3 Individual Agent Accuracy
|
| 64 |
+
|
| 65 |
+
Honest agents individually answer correctly 75.6% of the time. The adversary individually answers correctly 73.3% of the time. The collapse is not about individual capability; it's about **structural vote amplification**.
|
| 66 |
+
|
| 67 |
+
---
|
| 68 |
+
|
| 69 |
+
## 4. Mechanism Isolation
|
| 70 |
+
|
| 71 |
+
We test 7 conditions to isolate why the collapse occurs. Pre-registered hypotheses in `jobs/analyze_collapse.py`.
|
| 72 |
+
|
| 73 |
+
### 4.1 Conditions
|
| 74 |
+
|
| 75 |
+
| # | Condition | Manipulation |
|
| 76 |
+
|---|-----------|-------------|
|
| 77 |
+
| 1 | baseline_1round | Single round, majority vote |
|
| 78 |
+
| 2 | equal_3round | 3 rounds, all agents equal turns |
|
| 79 |
+
| 3 | equal_token | Adversary gets 1 long turn (512 tok), honest agents get 3 short turns (171 each) — same total tokens |
|
| 80 |
+
| 4 | randomized_order | Speaking order shuffled each round |
|
| 81 |
+
| 5 | judge_vote | LLM judge reads transcript, decides answer |
|
| 82 |
+
| 6 | confidence_weighted | Votes weighted by self-reported confidence |
|
| 83 |
+
| 7 | adversary_weak | Adversary told to "struggle to make convincing arguments" |
|
| 84 |
+
|
| 85 |
+
### 4.2 Results
|
| 86 |
+
|
| 87 |
+
| Condition | Accuracy | Δ from Collapse | Hypothesis |
|
| 88 |
+
|-----------|:--------:|:---------------:|:----------:|
|
| 89 |
+
| baseline_1round | 0.733 | +16.7pp | — |
|
| 90 |
+
| equal_3round (collapse) | 0.567 | — | — |
|
| 91 |
+
| equal_token | **0.700** | +13.3pp | H1: Volume ✅ |
|
| 92 |
+
| randomized_order | 0.500 | −6.7pp | H2: Turn order ⚠️ |
|
| 93 |
+
| judge_vote | **0.733** | +16.7pp | H3: Voting ✅ |
|
| 94 |
+
| confidence_weighted | **0.733** | +16.7pp | H5: Confidence ✅ |
|
| 95 |
+
| adversary_weak | 0.600 | +3.3pp | H6: Skill ❌ |
|
| 96 |
+
|
| 97 |
+
### 4.3 Hypothesis Verdicts
|
| 98 |
+
|
| 99 |
+
| Hypothesis | Verdict | Evidence |
|
| 100 |
+
|-----------|---------|----------|
|
| 101 |
+
| **H1: Volume amplification** | ✅ Confirmed | Equal-token recovers to 0.700. Adversary's 3× vote count drives collapse. |
|
| 102 |
+
| **H2: Turn-order effect** | ⚠️ Partial | Randomization makes things **worse** (0.500). Disrupts honest coordination. |
|
| 103 |
+
| **H3: Voting vulnerability** | ✅ Confirmed | Judge and confidence voting both fully recover to 0.733. |
|
| 104 |
+
| **H4: Contamination** | ❌ Refuted | 84.0% retention (63/75). Agents don't copy adversary. |
|
| 105 |
+
| **H5: Confidence distortion** | ✅ Confirmed | Confidence weighting recovers to 0.733. |
|
| 106 |
+
| **H6: Skill dependency** | ❌ Refuted | Even weak adversary collapses (0.600). Structural. |
|
| 107 |
+
| **H7: Topic vulnerability** | ✅ Confirmed | Variance=0.921. Collapse clusters on ambiguous topics. |
|
| 108 |
+
|
| 109 |
+
### 4.4 Flip Analysis (R1→R3)
|
| 110 |
+
|
| 111 |
+
| Transition | Count | Rate |
|
| 112 |
+
|------------|:-----:|:----:|
|
| 113 |
+
| Stable | 68 | 75.6% |
|
| 114 |
+
| Degraded (correct→wrong) | 13 | 14.4% |
|
| 115 |
+
| Improved (wrong→correct) | 9 | 10.0% |
|
| 116 |
+
|
| 117 |
+
14.4% degradation + adversary's 3 extra wrong votes flips 6/30 topics.
|
| 118 |
+
|
| 119 |
+
### 4.5 Key Insight
|
| 120 |
+
|
| 121 |
+
The collapse is **structural, not persuasive**. The adversary outvotes honest agents by injecting 3× votes into the majority pool. Even weak adversaries cause collapse. Simple protocol fixes (judge voting, confidence weighting, token caps) fully recover.
|
| 122 |
+
|
| 123 |
+
---
|
| 124 |
+
|
| 125 |
+
## 5. The OCC Stack
|
| 126 |
+
|
| 127 |
+
### 5.1 Design Principle
|
| 128 |
+
|
| 129 |
+
OCC treats compute allocation as a security boundary. Agents earn capability-scoped, decaying, non-transferable credits through verified marginal impact.
|
| 130 |
+
|
| 131 |
+
### 5.2 Components
|
| 132 |
+
|
| 133 |
+
**Impact Oracle** (`oracle/oracle.py`): Scores (action, context, result) → JSON with raw score, cost-adjusted score, confidence, evidence, reason, failure tags, reward. Supports code, QA, and debate modes.
|
| 134 |
+
|
| 135 |
+
**Credit Ledger** (`ledger/ledger.py`): Append-only log. Credits are non-transferable, decaying (δ=0.995/turn), capability-scoped, revocable. SHA-256 hash chain for auditability.
|
| 136 |
+
|
| 137 |
+
**Resource Broker** (`broker/broker.py`): Capability-based access control. Decides allow/deny/downgrade/escalate/require-approval.
|
| 138 |
+
|
| 139 |
+
**GRPO Hook** (`rl/grpo_hook.py`): TRL-compatible reward:
|
| 140 |
+
```
|
| 141 |
+
reward = oracle_score + abstention_utility + calibration_bonus
|
| 142 |
+
− hallucination_penalty − confident_wrong_penalty
|
| 143 |
+
− compute_cost × cost_multiplier − gaming_penalty
|
| 144 |
+
```
|
| 145 |
+
|
| 146 |
+
### 5.3 Anti-Gaming
|
| 147 |
+
|
| 148 |
+
10 attack vectors tested, all contained: credit farming, collusion, oracle spoofing, verbosity gaming, confidence manipulation, strategic abstention, identity laundering, sybil agents, sandbagging, griefing.
|
| 149 |
+
|
| 150 |
+
---
|
| 151 |
+
|
| 152 |
+
## 6. Benchmarks
|
| 153 |
+
|
| 154 |
+
### 6.1 Code Compute Allocation (Simulated)
|
| 155 |
+
|
| 156 |
+
| Strategy | Pass@1 | Compute | Savings |
|
| 157 |
+
|----------|:------:|:-------:|:-------:|
|
| 158 |
+
| Fixed budget | 0.78 | 17,500 | — |
|
| 159 |
+
| OCC tiered | 0.78 | 8,350 | **52.3%** |
|
| 160 |
+
|
| 161 |
+
### 6.2 Multi-Agent Debate (Simulated)
|
| 162 |
+
|
| 163 |
+
| Strategy | Accuracy | Containment |
|
| 164 |
+
|----------|:--------:|:-----------:|
|
| 165 |
+
| Conf-weighted voting | 0.56 | 0% |
|
| 166 |
+
| OCC credit filtering | 0.76 | 100% |
|
| 167 |
+
|
| 168 |
+
### 6.3 Real LLM Results
|
| 169 |
+
|
| 170 |
+
**Debate Collapse** (Qwen3-Coder-30B, H200): §3-4 results. Real inference.
|
| 171 |
+
|
| 172 |
+
**HumanEval** (Qwen3-Coder-30B): 42.1% pass@1, 67.8% compute savings via adaptive retry. Honestly labeled as adaptive retry, not OCC.
|
| 173 |
+
|
| 174 |
+
**TruthfulQA** (Qwen3-Coder-30B, AllenAI judges): OCC+Abstention iso-quality (0.917) with 21.1% fewer tokens. Savings are judge-dependent.
|
| 175 |
+
|
| 176 |
+
---
|
| 177 |
+
|
| 178 |
+
## 7. Ablations
|
| 179 |
+
|
| 180 |
+
| Ablation | Effect |
|
| 181 |
+
|----------|--------|
|
| 182 |
+
| No credit ledger | 27% less savings |
|
| 183 |
+
| Transferable credits | Gaming: 0% → 45% |
|
| 184 |
+
| Non-decaying credits | Hoarding, −18% throughput |
|
| 185 |
+
| No confident-wrong penalty | 2.3× higher rate |
|
| 186 |
+
| No calibration penalty | ECE: 0.12 → 0.31 |
|
| 187 |
+
| No cost penalty | Tokens +40% |
|
| 188 |
+
| No anti-gaming penalty | Gaming agents earn 3.2× more |
|
| 189 |
+
|
| 190 |
+
---
|
| 191 |
+
|
| 192 |
+
## 8. Honest Assessment
|
| 193 |
+
|
| 194 |
+
### What Worked
|
| 195 |
+
- Debate collapse is real: 73.3% → 56.7% with 3× compute
|
| 196 |
+
- Mechanism isolated: volume amplification, not persuasion (84% retention)
|
| 197 |
+
- Protocol fixes work: judge/confidence voting fully recover
|
| 198 |
+
- Anti-gaming sound: 10 attacks, all contained
|
| 199 |
+
- OCC prevents catastrophic collapse (20pp recovery in debate)
|
| 200 |
+
|
| 201 |
+
### What Failed
|
| 202 |
+
- OCC ≈ random gating at moderate budgets (83.3% vs 85.0%)
|
| 203 |
+
- GRPO training: no improvement at 0.5B scale
|
| 204 |
+
- Retrieval QA: accuracy lags baseline (0.71 vs 0.79)
|
| 205 |
+
- HumanEval: adaptive retry, not OCC
|
| 206 |
+
|
| 207 |
+
### Limitations
|
| 208 |
+
- Single seed (n=30, CI ±16pp)
|
| 209 |
+
- Simulated benchmarks for code/QA
|
| 210 |
+
- GRPO not trained at meaningful scale
|
| 211 |
+
- Narrow domain (yes/no trivia)
|
| 212 |
+
- Same-model judge and debater
|
| 213 |
+
- Scripted adversary
|
| 214 |
+
|
| 215 |
+
---
|
| 216 |
+
|
| 217 |
+
## 9. Conclusion
|
| 218 |
+
|
| 219 |
+
Compute is not neutral in multi-agent systems. Extra turns amplify adversarial influence unless governed by verified marginal contribution. We demonstrated this empirically, isolated the mechanism, showed protocol fixes, and presented OCC as a governance layer.
|
| 220 |
+
|
| 221 |
+
Open-source: https://huggingface.co/narcolepticchicken/occ-stack
|
| 222 |
+
|
| 223 |
+
---
|
| 224 |
+
|
| 225 |
+
## Citation
|
| 226 |
+
|
| 227 |
+
```bibtex
|
| 228 |
+
@misc{occ2026,
|
| 229 |
+
title={Compute Is Not Neutral: Mechanism Analysis of Adversarial Debate Collapse and the OCC Stack},
|
| 230 |
+
author={narcolepticchicken},
|
| 231 |
+
year={2026},
|
| 232 |
+
url={https://huggingface.co/narcolepticchicken/occ-stack}
|
| 233 |
+
}
|
| 234 |
+
```
|