Buckets:

Mercity
/

Desicion-Model-Custom

Files

xet

Mercity/Desicion-Model-Custom / think-decide /REPORT.md

Pranav2748

7 days ago

preview code

download

raw

16.8 kB

# Can diffusion steps replace token thinking in a decision model?

Think-then-Decide, 6 November 2026. One A100 80GB, ~8 hours. Code, logs, predictions and checkpoints are in the
bucket `Mercity/Desicion-Model-Custom` under `think-decide/`. Lab notebook: `NOTEBOOK.md`; every change to the plan:
`DEVIATIONS.md`.

## TL;DR

- **No latent "thinking" design worked.** Six designs that iterate on latent vectors were tried, from the original
  plan's thought slots to a separate latent reasoner read by a decision head. All six score the same with 32 steps as
  with 1 step (gains between -0.04 and +0.24 points, every 95% CI includes 0).
- **The one design that gains from extra passes is answer-level diffusion (MQ).** All of a task's sub-questions
  go on one canvas, and the model commits its most confident answer each pass, so earlier answers help later ones.
  Result: +0.48 points [+0.21, +0.76] on the same questions, rising to +1.5 on the questions answered last. The
  ceiling is +1.75 (all other answers given), so MQ captures about 27% of it.
- **The gains are small next to token thinking.** In the bucket's previous project, the 4B model gained about 48
  points on the same procedural tasks by writing reasoning tokens. Here, iteration buys about half a point.
- **Diagnosis.** Iteration only helps when the positions being refined are supervised (answers or sub-answers).
  Unsupervised latent vectors get ignored, whether they sit inside the backbone, in a separate latent reasoner, or
  behind a gate. Both literature reviews reached the same conclusion.

![Gain from extra passes for every approach](results/fig1_gain_overview.png)

*Figure 1. Accuracy gain from extra passes vs one pass, same questions, paired bootstrap over items (95% CI). The
two Core rows use Core's reasoning suites; all other rows use the procedural testbed. \*MQ-soft's gain is relative
to its own one-pass accuracy, which is 1.5 points below MQ's; in absolute terms MQ is better (see section 3.3).*

## 1. Question and setup

The user's question: can the token-by-token thinking of an LLM be replaced by diffusion steps? The test was a small
diffusion decision model doing a few extra denoising passes "in its head" before deciding, compared with deciding
in one pass.

- **Backbone.** S1 (`SEU-ZZH/Shared-YesNo-Qwen3-0.6B-S1`, from jev-dllm) is Qwen3-0.6B converted to a masked
  diffusion LM with bidirectional attention, fine-tuned to make decisions in one pass. Each option gets a [MASK];
  its score is logit(Yes) - logit(No) at that mask, followed by a softmax over the options.
- **Benchmarks.**
  - **Core:** jev-dllm's 17-suite benchmark, 6,744 decisions. All items were scored; S1's own protocol drops
    221 long ContractNLI items.
  - **Procedural testbed:** `tasksource/procedural-typed-decisions`, 18 multi-step task families × 5 difficulty
    levels. Not in S1's training data. Token thinking helped the previous project's 4B model by about 48 points on
    this distribution.
- **Statistics.** Every gain is measured within one model (K passes vs 1 pass) on the same items, with a paired
  bootstrap (10,000 resamples) stratified by suite or task family.
## 2. Everything that was run

| ID | What | Data | Result |
|---|---|---|---|
| E0 | Harness parity: jev-dllm's evaluator and ours, on S1 | Core | reproduces published S1; group scores equal |
| E1 | S1 + decision head, 1 epoch (control) | S1's 40k train | Core reasoning 64.99 (S1 64.07) |
| E2 | Plan: 32 thought slots, gated latent loop, K in {1,2,4,8} | 40k | **flat** (+0.08 [-0.27, +0.43]); gate stuck at 0.002 |
| E3 | E2 + CODI-style distillation from S1 reading notes | 40k + 5.4k teacher targets | **flat** (-0.04 [-0.58, +0.59]) |
| P0 | S1 zero-shot | procedural | 42.4% |
| P1 | S1 + head, 1 question per state, K=1 | 32k procedural | **88.7%** |
| P2o | E2's loop, gate opened at 0.014 | 32k | killed at step 520: 4 passes = 1 pass on dev (83.97 = 83.97) |
| P2s | Soft-token feedback into the slots, no gate | 32k | killed at step 310: 4 passes = 1 pass on dev (71.80 = 71.80) |
| **MQ** | **All sub-questions on one canvas; diffusion over answers** | 32k states (105k questions) | **sequential +0.48 [+0.21, +0.76]**; 1 pass 90.7 on target questions |
| MQ-soft | MQ + soft-belief self-conditioning | 32k states | multi-pass +0.8 over its own 1 pass, but below MQ overall |
| LDR A-D | Separate latent reasoner (32 vectors, K steps) read by a decision head | 32k procedural | **flat** in all four variants |

Not run, by decision: E5 (notes written as tokens by diffusion; dropped at the user's request), general-mix
E2o/E2s (superseded), E4 latency (not measured), and the full-data MQ run (the user called time).

## 3. Results

### 3.1 The pre-registered plan on Core (E0-E3)

| Model | K | Reasoning | Lookup | Other | Core17 | Core14 |
|---|---|---|---|---|---|---|
| S1 | 1 | 64.07 | 73.96 | 61.49 | 66.07 | 64.05 |
| E1 (head, no thinking) | 1 | 64.99 | 74.10 | 60.50 | 66.08 | 64.15 |
| E2 (latent loop) | 1 / 8 | 64.67 / 64.76 | 74.45 / 74.35 | 59.82 / 59.94 | 65.83 / 65.88 | 63.93 / 63.97 |
| E3 (+ distillation) | 1 / 8 | 64.51 / 64.47 | 75.31 / 75.34 | 59.56 / 59.63 | 65.94 / 65.96 | 64.02 / 64.01 |

Core17 is the equal-weight mean accuracy over all 17 suites. Core14 drops three suites that S1 saw in training.

| Pre-registered hypothesis | Test | Result [95% CI] | Verdict |
|---|---|---|---|
| H1: accuracy rises with K | E2, K=8 vs K=1, reasoning | +0.08 [-0.27, +0.43] | not supported |
| H2: the gain comes from thinking, not the head or extra epoch | E2 (best K) vs E1 | -0.31 [-1.51, +0.91] | not supported |
| H3: a teacher target helps | E3 (best K) vs E2 (best K) | -0.59 [-1.84, +0.60] | not supported |

![Core K curves](results/fig5_core_k.png)

**Why it failed: the gate deadlocks.** The feedback is x = e_mask + g·W·LN(h), with g starting at 0.
- W's gradient is proportional to g, so W cannot learn while the gate is shut.
- g's gradient points along W·LN(h): a last-layer state pushed into input-embedding space, which is meaningless
  until W has learned something.
So neither can move first. The gate ended at 0.002 in E2 and 0.001 in E3, while about 0.014 is needed for the
feedback to match the mask embedding's size. Opening the gate (P2o) or removing it (P2s) did not help either: the
model left the channel unused.
![Gate over training](results/fig6_gate.png)
**Why E3's teacher could not help.** Frozen S1 barely uses notes, even notes written by Qwen3.5-9B with the gold
answer in view: 93.6% accuracy with a note vs 93.7% without, over 9,370 training items. E3's distillation target
was therefore close to S1's own states.
### 3.2 The procedural testbed (P0-P2)
| Model | All | L0 (easy) | L1 | L2 | L3 | L4 (hard) |
|---|---|---|---|---|---|---|
| P0: S1 zero-shot | 42.37 | 53.25 | 47.52 | 40.65 | 36.21 | 34.21 |
| P1: trained, 1 question, 1 pass | 88.68 | 96.28 | 91.93 | 87.94 | 84.01 | 83.20 |
Levels are difficulty: level 0 is short and few-step, level 4 long and many-step. One epoch of training on 32k
items (9% of the dataset) takes S1 from 42% to 89%. The errors left over are concentrated at levels 3-4.
### 3.3 Answer diffusion (MQ): the only positive result
**Design.** Every question of a state goes on one canvas, in S1's own format, each with its answer masks:
```
State: ...
Question 1: Which policy governs?   P1: [MASK]   P2: [MASK]
Question 2: Is access allowed?      Answer: [MASK]
Question 3: How risky is it?        level 0: [MASK]  level 1: [MASK]  level 2: [MASK]
```
- **Training** is masked diffusion over answers: each question's gold Yes/No tokens are revealed with
  probability 1-t (t ~ U(0,1)), and the loss is computed on the hidden ones.
- **Inference** has three modes, all on the same model and questions:
  - *parallel:* all answers in 1 pass;
  - *sequential:* each pass commits the most confident open question, then re-reads;
  - *oracle:* all other answers are given correctly.
- There are no new parameters and no gate. What carries information between steps is committed answer tokens,
  which are supervised.
![Procedural accuracy](results/fig2_procedural.png)
| MQ readout (11,582 test questions) | All | Target q | L3 | L4 | vs parallel [95% CI] |
|---|---|---|---|---|---|
| parallel (1 pass) | 88.45 | 90.67 | 83.07 | 82.75 | - |
| **sequential (1 answer per pass)** | **88.98** | **91.10** | 83.82 | 83.25 | **+0.48 [+0.21, +0.76]** |
| oracle (others given) | 90.84 | 92.28 | 87.29 | 86.81 | +1.75 [+1.45, +2.06] |
- **The gain grows with commit order.** Questions answered later have more committed answers to use. Against
  parallel they gain +0.6 at rank 1, +0.8 at rank 2 and +1.5 at rank 3.
- **The clearest gains come from families whose questions chain explicitly.** Policy applicability goes 97.5 to
  99.0 and belief tracking 98.8 to 100.0. The largest untapped headroom is record eligibility: 78.1 sequential vs
  87.5 oracle.
- **Training on all the questions helps one-pass accuracy too.** On the same target questions MQ scores 90.67 in
  one pass vs P1's 88.68.
**Does accuracy grow with the number of steps?** Yes, up to about 3, then it plateaus. These are the same
questions with only the step budget changed; each pass commits ceil(open/passes left) answers.
![MQ step curve](results/fig3_mq_steps.png)
**MQ-soft (continuous diffusion).** Nothing is committed: each pass feeds back soft Yes/No beliefs as blended
embeddings.
- Continuous soft passes gain about as much as discrete commits, relative to the same model's one-pass score:
  +0.81 [+0.50, +1.14] for soft passes vs +0.80 for commits.
- But soft self-conditioning training lowered one-pass accuracy by 1.5 points. MQ-soft's best result (87.86 on
  all questions) therefore stays below plain MQ's (88.98).
- My automatic rule for the large run compared gains instead of absolute accuracy, so it picked MQ-soft. That run
  was stopped once this was noticed.
### 3.4 Latent-vector reasoners (LDR): the user's target architecture
**Design.** The backbone runs once and its token states are cached. A small weight-tied reasoner (38M parameters:
self-attention over 32 latents and the answer-belief slots, cross-attention to the cached states, MLP) refines the
latents for K steps. A decision head then reads them and adds a correction to the option states before the Yes/No
readout.
- Zero-initialised output, so K=0 equals the encoder alone.
- Random depth K ~ U{1..12}, truncated backprop through the last 3 steps, EMA weights.
- Noise in the initial latents, averaged over 4 seeds at evaluation.
- Recipe from TRM, HRM and Huginn, per the third literature review.
| Variant | K=0 | K=1 | K=8 | K=32 | K=32 - K=0 [95% CI] |
|---|---|---|---|---|---|
| A: frozen P1 encoder | 88.68 | 88.71 | 88.71 | 88.71 | +0.12 [-0.26, +0.55] |
| B: frozen MQ encoder, all sub-questions | 88.03 | 88.00 | 87.98 | 87.98 | -0.01 [-0.19, +0.18] |
| C: encoder trained jointly (from S1) | 85.05 | 84.96 | 85.05 | 85.05 | 0.00 [-0.39, +0.39] |
| D: frozen raw S1 (huge headroom) | 42.43 | 42.62 | 42.49 | 42.46 | +0.24 [-0.27, +0.80] |
![Latent reasoners](results/fig4_ldr_flat.png)
D is the telling case. With 46 points of headroom on the table, the latent reasoner learned essentially nothing in
~550 steps: its gradient norm stayed around 0.01-0.3 and the loss did not fall. In C, joint training also left the
encoder worse than P1 (85.1 vs 88.7). At this model size and training budget (~500 steps per run), latent iteration
did not learn any useful computation.
## 4. Diagnosis: why latent thinking did not work here
1. **Unsupervised latents get ignored.** This held in every placement tried (inside the backbone, behind a gate,
   without a gate, in a separate reasoner). The literature agrees: Coconut fails without its curriculum, filler
   tokens need dense supervision, latents collapse in SIM-CoT, and He, Welleck and Fried (2602.03769) find that
   extra unsupervised positions hurt.
2. **One pass already solves most of each decision.** P1 gets 89% in one pass. The remaining errors need
   information that a re-read of the same states cannot create; the supervised sub-answers can (MQ's oracle,
   +1.75).
3. **Retrofitting recurrence takes far more training than we had.** Published retrofits use billions of tokens
   (McLeish et al. 2511.07384: ~50B; ETD: 50B; LoopUS: 3B). Each run here saw ~10-30M. Loopholing reports that
   fine-tuning-only retrofits failed.
4. **The feedback space matters.** Last-layer states fed back into the input deadlock under a zero gate. Token-space
   feedback (soft beliefs, committed answers) works immediately; it is what MQ and MQ-soft use.
## 5. Cost of the one thing that worked
- MQ's sequential mode costs one backbone pass per committed question: n passes for n questions (3.25 on average),
  because a bidirectional model cannot cache earlier work. On a 700-token state with 3 questions, that is about 2,100
  token-passes of a 0.6B model in 3 sequential rounds.
- Token thinking in the previous project's 4B model wrote about 700-1,000 reasoning tokens per question, in as many
  sequential decode rounds of a model about 7x larger.
- So MQ is far cheaper per decision but buys about +0.5 points, against about +48 points for token thinking on the
  same task family.
- The latent reasoners' steps are cheap (32 latents through two small blocks), but they bought nothing.
- Wall-clock latency was not measured (E4 not run).
## 6. Caveats
- **Single seed per arm.** Uncertainty comes from bootstrapping over test items only.
- **The plan's decision head was nearly inert.** It added its correction to the pre-norm option state, which has
  norm ~1,700 vs ~46 after the norm. E1-E3's differences from S1 therefore come from the extra backbone epoch, not
  the head. Found at 20:00; the latent reasoners use a post-norm head.
- **Harness parity.** Group scores match jev-dllm's evaluator exactly; 1.4% of argmaxes still differ (bf16
  numerics), the same for every arm.
- **P2o and P2s were stopped early** (steps 520 and 310 of 1,000), with dev K=4 = K=1 at every check.
- **The large MQ run was never completed.** A 216k-state MQ-soft run was stopped at 18% (wrong pick rule, section
  3.3); the full-data MQ run was not started.
- **E3's notes were generated with the gold answer visible**, then leak-filtered (93.7% kept). Frozen S1 could not
  use them, which limits what E3 could show.
## 7. What I would do next
1. **Scale MQ to the full 360k states** (about 4 h on one A100). This answers whether the sequential gain grows
   (better use of committed answers) or shrinks (one-pass accuracy catches up). Train-time noise on the answer
   channel and loss-weighting hard questions (MGDM-style) are cheap additions.
2. **Give the latent reasoner a fair budget before ruling it out.** Cache the frozen encoder states once, then train
   the reasoner for 30+ epochs at a higher learning rate. That takes seconds per epoch instead of minutes. If K
   still does not matter, call latent iteration a no-go at 0.6B.
3. **Supervise the latents.** Decode each latent step to a sub-answer or a short reasoning step (SIM-CoT, TRM's y
   channel). This is where all of the positive evidence points.
## 8. Engineering notes
- **Speed.** torch.compile on the decoder layers gave 1.6x. A padding-aware micro-batch split cut padded compute
  from 1.3-3.0x to 1.17x of real tokens. fp32 master weights with bf16 autocast (S1's own trainer kept pure bf16
  weights, which loses small updates).
- **Memory.** Training peaked at 74.9 GiB of 80 (94%), with an out-of-memory fallback that restores gradients and
  retries with checkpointing; 0 retries were needed.
- **No truncation anywhere.** Every Core item was scored. Procedural training used max length 8,192, so no item was
  dropped; a 2,048 cap would have dropped 2.5%, nearly all level 3-4. Note generation flagged capped outputs (none
  occurred) rather than cutting them.
- **Data.** Core 6,744 items. S1's training set: 40k. Procedural: 78.5k prepared (32k used per arm), 1,751 dev,
  3,484 test states (11,582 questions). MQ used the 32k states' 105k questions; the dataset has 360k training states.
## 9. Files
- `src/`: `common.py` (renderer, loaders), `model.py` (ThinkDecide), `train.py`, `eval.py`, `distill.py`,
  `teacher_cache.py`, `gen_notes.py`, `prep_procedural.py`, `mq.py` (answer diffusion), `ldr.py` / `ldr2.py`
  (latent reasoners), `bootstrap.py`, `analyze*.py`
- `runs/<arm>/`: config, training log, predictions, report, weights. bf16 checkpoints: `e1`, `e2`, `e3`, `p1`,
  `mq`, `mqsoft`; `ldr2_c/encoder.safetensors` and the `reasoner_ema.pt` files.
- `results/`: `results.md` (Core), `results_proc.md`, `results_mq.md`, `summary_stats.json`, figures
- `logs/`: every training and eval log; `NOTEBOOK.md`, `DEVIATIONS.md`, `TODO.md`

Xet Storage Details

Size:: 16.8 kB
Xet hash:: 17acaeecd75ae12e06ef915012b3a8f90c81e3fa2ac952251ffc50f5d7110fa8

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.

	# Can diffusion steps replace token thinking in a decision model?

	Think-then-Decide, 6 November 2026. One A100 80GB, ~8 hours. Code, logs, predictions and checkpoints are in the
	bucket `Mercity/Desicion-Model-Custom` under `think-decide/`. Lab notebook: `NOTEBOOK.md`; every change to the plan:
	`DEVIATIONS.md`.

	## TL;DR

	- No latent "thinking" design worked. Six designs that iterate on latent vectors were tried, from the original
	plan's thought slots to a separate latent reasoner read by a decision head. All six score the same with 32 steps as
	with 1 step (gains between -0.04 and +0.24 points, every 95% CI includes 0).
	- The one design that gains from extra passes is answer-level diffusion (MQ). All of a task's sub-questions
	go on one canvas, and the model commits its most confident answer each pass, so earlier answers help later ones.
	Result: +0.48 points [+0.21, +0.76] on the same questions, rising to +1.5 on the questions answered last. The
	ceiling is +1.75 (all other answers given), so MQ captures about 27% of it.
	- The gains are small next to token thinking. In the bucket's previous project, the 4B model gained about 48
	points on the same procedural tasks by writing reasoning tokens. Here, iteration buys about half a point.
	- Diagnosis. Iteration only helps when the positions being refined are supervised (answers or sub-answers).
	Unsupervised latent vectors get ignored, whether they sit inside the backbone, in a separate latent reasoner, or
	behind a gate. Both literature reviews reached the same conclusion.

	![Gain from extra passes for every approach](results/fig1_gain_overview.png)

	*Figure 1. Accuracy gain from extra passes vs one pass, same questions, paired bootstrap over items (95% CI). The
	two Core rows use Core's reasoning suites; all other rows use the procedural testbed. \*MQ-soft's gain is relative
	to its own one-pass accuracy, which is 1.5 points below MQ's; in absolute terms MQ is better (see section 3.3).*

	## 1. Question and setup

	The user's question: can the token-by-token thinking of an LLM be replaced by diffusion steps? The test was a small
	diffusion decision model doing a few extra denoising passes "in its head" before deciding, compared with deciding
	in one pass.

	- Backbone. S1 (`SEU-ZZH/Shared-YesNo-Qwen3-0.6B-S1`, from jev-dllm) is Qwen3-0.6B converted to a masked
	diffusion LM with bidirectional attention, fine-tuned to make decisions in one pass. Each option gets a [MASK];
	its score is logit(Yes) - logit(No) at that mask, followed by a softmax over the options.
	- Benchmarks.
	- Core: jev-dllm's 17-suite benchmark, 6,744 decisions. All items were scored; S1's own protocol drops
	221 long ContractNLI items.
	- Procedural testbed: `tasksource/procedural-typed-decisions`, 18 multi-step task families × 5 difficulty
	levels. Not in S1's training data. Token thinking helped the previous project's 4B model by about 48 points on
	this distribution.
	- Statistics. Every gain is measured within one model (K passes vs 1 pass) on the same items, with a paired
	bootstrap (10,000 resamples) stratified by suite or task family.

	## 2. Everything that was run

	\| ID \| What \| Data \| Result \|
	\|---\|---\|---\|---\|
	\| E0 \| Harness parity: jev-dllm's evaluator and ours, on S1 \| Core \| reproduces published S1; group scores equal \|
	\| E1 \| S1 + decision head, 1 epoch (control) \| S1's 40k train \| Core reasoning 64.99 (S1 64.07) \|
	\| E2 \| Plan: 32 thought slots, gated latent loop, K in {1,2,4,8} \| 40k \| flat (+0.08 [-0.27, +0.43]); gate stuck at 0.002 \|
	\| E3 \| E2 + CODI-style distillation from S1 reading notes \| 40k + 5.4k teacher targets \| flat (-0.04 [-0.58, +0.59]) \|
	\| P0 \| S1 zero-shot \| procedural \| 42.4% \|
	\| P1 \| S1 + head, 1 question per state, K=1 \| 32k procedural \| 88.7% \|
	\| P2o \| E2's loop, gate opened at 0.014 \| 32k \| killed at step 520: 4 passes = 1 pass on dev (83.97 = 83.97) \|
	\| P2s \| Soft-token feedback into the slots, no gate \| 32k \| killed at step 310: 4 passes = 1 pass on dev (71.80 = 71.80) \|
	\| MQ \| All sub-questions on one canvas; diffusion over answers \| 32k states (105k questions) \| sequential +0.48 [+0.21, +0.76]; 1 pass 90.7 on target questions \|
	\| MQ-soft \| MQ + soft-belief self-conditioning \| 32k states \| multi-pass +0.8 over its own 1 pass, but below MQ overall \|
	\| LDR A-D \| Separate latent reasoner (32 vectors, K steps) read by a decision head \| 32k procedural \| flat in all four variants \|

	Not run, by decision: E5 (notes written as tokens by diffusion; dropped at the user's request), general-mix
	E2o/E2s (superseded), E4 latency (not measured), and the full-data MQ run (the user called time).

	## 3. Results

	### 3.1 The pre-registered plan on Core (E0-E3)

	\| Model \| K \| Reasoning \| Lookup \| Other \| Core17 \| Core14 \|
	\|---\|---\|---\|---\|---\|---\|---\|
	\| S1 \| 1 \| 64.07 \| 73.96 \| 61.49 \| 66.07 \| 64.05 \|
	\| E1 (head, no thinking) \| 1 \| 64.99 \| 74.10 \| 60.50 \| 66.08 \| 64.15 \|
	\| E2 (latent loop) \| 1 / 8 \| 64.67 / 64.76 \| 74.45 / 74.35 \| 59.82 / 59.94 \| 65.83 / 65.88 \| 63.93 / 63.97 \|
	\| E3 (+ distillation) \| 1 / 8 \| 64.51 / 64.47 \| 75.31 / 75.34 \| 59.56 / 59.63 \| 65.94 / 65.96 \| 64.02 / 64.01 \|

	Core17 is the equal-weight mean accuracy over all 17 suites. Core14 drops three suites that S1 saw in training.

	\| Pre-registered hypothesis \| Test \| Result [95% CI] \| Verdict \|
	\|---\|---\|---\|---\|
	\| H1: accuracy rises with K \| E2, K=8 vs K=1, reasoning \| +0.08 [-0.27, +0.43] \| not supported \|
	\| H2: the gain comes from thinking, not the head or extra epoch \| E2 (best K) vs E1 \| -0.31 [-1.51, +0.91] \| not supported \|
	\| H3: a teacher target helps \| E3 (best K) vs E2 (best K) \| -0.59 [-1.84, +0.60] \| not supported \|

	![Core K curves](results/fig5_core_k.png)

	Why it failed: the gate deadlocks. The feedback is x = e_mask + g·W·LN(h), with g starting at 0.
	- W's gradient is proportional to g, so W cannot learn while the gate is shut.
	- g's gradient points along W·LN(h): a last-layer state pushed into input-embedding space, which is meaningless
	until W has learned something.

	So neither can move first. The gate ended at 0.002 in E2 and 0.001 in E3, while about 0.014 is needed for the
	feedback to match the mask embedding's size. Opening the gate (P2o) or removing it (P2s) did not help either: the
	model left the channel unused.

	![Gate over training](results/fig6_gate.png)

	Why E3's teacher could not help. Frozen S1 barely uses notes, even notes written by Qwen3.5-9B with the gold
	answer in view: 93.6% accuracy with a note vs 93.7% without, over 9,370 training items. E3's distillation target
	was therefore close to S1's own states.

	### 3.2 The procedural testbed (P0-P2)

	\| Model \| All \| L0 (easy) \| L1 \| L2 \| L3 \| L4 (hard) \|
	\|---\|---\|---\|---\|---\|---\|---\|
	\| P0: S1 zero-shot \| 42.37 \| 53.25 \| 47.52 \| 40.65 \| 36.21 \| 34.21 \|
	\| P1: trained, 1 question, 1 pass \| 88.68 \| 96.28 \| 91.93 \| 87.94 \| 84.01 \| 83.20 \|

	Levels are difficulty: level 0 is short and few-step, level 4 long and many-step. One epoch of training on 32k
	items (9% of the dataset) takes S1 from 42% to 89%. The errors left over are concentrated at levels 3-4.

	### 3.3 Answer diffusion (MQ): the only positive result

	Design. Every question of a state goes on one canvas, in S1's own format, each with its answer masks:

	```
	State: ...
	Question 1: Which policy governs? P1: [MASK] P2: [MASK]
	Question 2: Is access allowed? Answer: [MASK]
	Question 3: How risky is it? level 0: [MASK] level 1: [MASK] level 2: [MASK]
	```

	- Training is masked diffusion over answers: each question's gold Yes/No tokens are revealed with
	probability 1-t (t ~ U(0,1)), and the loss is computed on the hidden ones.
	- Inference has three modes, all on the same model and questions:
	- parallel: all answers in 1 pass;
	- sequential: each pass commits the most confident open question, then re-reads;
	- oracle: all other answers are given correctly.
	- There are no new parameters and no gate. What carries information between steps is committed answer tokens,
	which are supervised.

	![Procedural accuracy](results/fig2_procedural.png)

	\| MQ readout (11,582 test questions) \| All \| Target q \| L3 \| L4 \| vs parallel [95% CI] \|
	\|---\|---\|---\|---\|---\|---\|
	\| parallel (1 pass) \| 88.45 \| 90.67 \| 83.07 \| 82.75 \| - \|
	\| sequential (1 answer per pass) \| 88.98 \| 91.10 \| 83.82 \| 83.25 \| +0.48 [+0.21, +0.76] \|
	\| oracle (others given) \| 90.84 \| 92.28 \| 87.29 \| 86.81 \| +1.75 [+1.45, +2.06] \|

	- The gain grows with commit order. Questions answered later have more committed answers to use. Against
	parallel they gain +0.6 at rank 1, +0.8 at rank 2 and +1.5 at rank 3.
	- The clearest gains come from families whose questions chain explicitly. Policy applicability goes 97.5 to
	99.0 and belief tracking 98.8 to 100.0. The largest untapped headroom is record eligibility: 78.1 sequential vs
	87.5 oracle.
	- Training on all the questions helps one-pass accuracy too. On the same target questions MQ scores 90.67 in
	one pass vs P1's 88.68.

	Does accuracy grow with the number of steps? Yes, up to about 3, then it plateaus. These are the same
	questions with only the step budget changed; each pass commits ceil(open/passes left) answers.

	![MQ step curve](results/fig3_mq_steps.png)

	MQ-soft (continuous diffusion). Nothing is committed: each pass feeds back soft Yes/No beliefs as blended
	embeddings.
	- Continuous soft passes gain about as much as discrete commits, relative to the same model's one-pass score:
	+0.81 [+0.50, +1.14] for soft passes vs +0.80 for commits.
	- But soft self-conditioning training lowered one-pass accuracy by 1.5 points. MQ-soft's best result (87.86 on
	all questions) therefore stays below plain MQ's (88.98).
	- My automatic rule for the large run compared gains instead of absolute accuracy, so it picked MQ-soft. That run
	was stopped once this was noticed.

	### 3.4 Latent-vector reasoners (LDR): the user's target architecture

	Design. The backbone runs once and its token states are cached. A small weight-tied reasoner (38M parameters:
	self-attention over 32 latents and the answer-belief slots, cross-attention to the cached states, MLP) refines the
	latents for K steps. A decision head then reads them and adds a correction to the option states before the Yes/No
	readout.
	- Zero-initialised output, so K=0 equals the encoder alone.
	- Random depth K ~ U{1..12}, truncated backprop through the last 3 steps, EMA weights.
	- Noise in the initial latents, averaged over 4 seeds at evaluation.
	- Recipe from TRM, HRM and Huginn, per the third literature review.

	\| Variant \| K=0 \| K=1 \| K=8 \| K=32 \| K=32 - K=0 [95% CI] \|
	\|---\|---\|---\|---\|---\|---\|
	\| A: frozen P1 encoder \| 88.68 \| 88.71 \| 88.71 \| 88.71 \| +0.12 [-0.26, +0.55] \|
	\| B: frozen MQ encoder, all sub-questions \| 88.03 \| 88.00 \| 87.98 \| 87.98 \| -0.01 [-0.19, +0.18] \|
	\| C: encoder trained jointly (from S1) \| 85.05 \| 84.96 \| 85.05 \| 85.05 \| 0.00 [-0.39, +0.39] \|
	\| D: frozen raw S1 (huge headroom) \| 42.43 \| 42.62 \| 42.49 \| 42.46 \| +0.24 [-0.27, +0.80] \|

	![Latent reasoners](results/fig4_ldr_flat.png)

	D is the telling case. With 46 points of headroom on the table, the latent reasoner learned essentially nothing in
	~550 steps: its gradient norm stayed around 0.01-0.3 and the loss did not fall. In C, joint training also left the
	encoder worse than P1 (85.1 vs 88.7). At this model size and training budget (~500 steps per run), latent iteration
	did not learn any useful computation.

	## 4. Diagnosis: why latent thinking did not work here

	1. Unsupervised latents get ignored. This held in every placement tried (inside the backbone, behind a gate,
	without a gate, in a separate reasoner). The literature agrees: Coconut fails without its curriculum, filler
	tokens need dense supervision, latents collapse in SIM-CoT, and He, Welleck and Fried (2602.03769) find that
	extra unsupervised positions hurt.
	2. One pass already solves most of each decision. P1 gets 89% in one pass. The remaining errors need
	information that a re-read of the same states cannot create; the supervised sub-answers can (MQ's oracle,
	+1.75).
	3. Retrofitting recurrence takes far more training than we had. Published retrofits use billions of tokens
	(McLeish et al. 2511.07384: ~50B; ETD: 50B; LoopUS: 3B). Each run here saw ~10-30M. Loopholing reports that
	fine-tuning-only retrofits failed.
	4. The feedback space matters. Last-layer states fed back into the input deadlock under a zero gate. Token-space
	feedback (soft beliefs, committed answers) works immediately; it is what MQ and MQ-soft use.

	## 5. Cost of the one thing that worked

	- MQ's sequential mode costs one backbone pass per committed question: n passes for n questions (3.25 on average),
	because a bidirectional model cannot cache earlier work. On a 700-token state with 3 questions, that is about 2,100
	token-passes of a 0.6B model in 3 sequential rounds.
	- Token thinking in the previous project's 4B model wrote about 700-1,000 reasoning tokens per question, in as many
	sequential decode rounds of a model about 7x larger.
	- So MQ is far cheaper per decision but buys about +0.5 points, against about +48 points for token thinking on the
	same task family.
	- The latent reasoners' steps are cheap (32 latents through two small blocks), but they bought nothing.
	- Wall-clock latency was not measured (E4 not run).

	## 6. Caveats

	- Single seed per arm. Uncertainty comes from bootstrapping over test items only.
	- The plan's decision head was nearly inert. It added its correction to the pre-norm option state, which has
	norm ~1,700 vs ~46 after the norm. E1-E3's differences from S1 therefore come from the extra backbone epoch, not
	the head. Found at 20:00; the latent reasoners use a post-norm head.
	- Harness parity. Group scores match jev-dllm's evaluator exactly; 1.4% of argmaxes still differ (bf16
	numerics), the same for every arm.
	- P2o and P2s were stopped early (steps 520 and 310 of 1,000), with dev K=4 = K=1 at every check.
	- The large MQ run was never completed. A 216k-state MQ-soft run was stopped at 18% (wrong pick rule, section
	3.3); the full-data MQ run was not started.
	- E3's notes were generated with the gold answer visible, then leak-filtered (93.7% kept). Frozen S1 could not
	use them, which limits what E3 could show.

	## 7. What I would do next

	1. Scale MQ to the full 360k states (about 4 h on one A100). This answers whether the sequential gain grows
	(better use of committed answers) or shrinks (one-pass accuracy catches up). Train-time noise on the answer
	channel and loss-weighting hard questions (MGDM-style) are cheap additions.
	2. Give the latent reasoner a fair budget before ruling it out. Cache the frozen encoder states once, then train
	the reasoner for 30+ epochs at a higher learning rate. That takes seconds per epoch instead of minutes. If K
	still does not matter, call latent iteration a no-go at 0.6B.
	3. Supervise the latents. Decode each latent step to a sub-answer or a short reasoning step (SIM-CoT, TRM's y
	channel). This is where all of the positive evidence points.

	## 8. Engineering notes

	- Speed. torch.compile on the decoder layers gave 1.6x. A padding-aware micro-batch split cut padded compute
	from 1.3-3.0x to 1.17x of real tokens. fp32 master weights with bf16 autocast (S1's own trainer kept pure bf16
	weights, which loses small updates).
	- Memory. Training peaked at 74.9 GiB of 80 (94%), with an out-of-memory fallback that restores gradients and
	retries with checkpointing; 0 retries were needed.
	- No truncation anywhere. Every Core item was scored. Procedural training used max length 8,192, so no item was
	dropped; a 2,048 cap would have dropped 2.5%, nearly all level 3-4. Note generation flagged capped outputs (none
	occurred) rather than cutting them.
	- Data. Core 6,744 items. S1's training set: 40k. Procedural: 78.5k prepared (32k used per arm), 1,751 dev,
	3,484 test states (11,582 questions). MQ used the 32k states' 105k questions; the dataset has 360k training states.

	## 9. Files

	- `src/`: `common.py` (renderer, loaders), `model.py` (ThinkDecide), `train.py`, `eval.py`, `distill.py`,
	`teacher_cache.py`, `gen_notes.py`, `prep_procedural.py`, `mq.py` (answer diffusion), `ldr.py` / `ldr2.py`
	(latent reasoners), `bootstrap.py`, `analyze*.py`
	- `runs/<arm>/`: config, training log, predictions, report, weights. bf16 checkpoints: `e1`, `e2`, `e3`, `p1`,
	`mq`, `mqsoft`; `ldr2_c/encoder.safetensors` and the `reasoner_ema.pt` files.
	- `results/`: `results.md` (Core), `results_proc.md`, `results_mq.md`, `summary_stats.json`, figures
	- `logs/`: every training and eval log; `NOTEBOOK.md`, `DEVIATIONS.md`, `TODO.md`