# Five-round self-evolution: final 64-sample evaluation

This is the supplemental final-checkpoint evaluation for the single-seed five-round run. It evaluates the base model and round-5 `plain`, `spd_hard`, and `spectral_soft` checkpoints on all 500 held-out MBPP tasks with 64 common-policy samples per task (32,000 samples per model).

## Results

| Model | pass@1 | pass@4 | pass@8 | pass@16 | pass@64 | total coverage@4 | total coverage@16 | total coverage@64 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| base | 0.379 | 0.551 | 0.611 | 0.657 | 0.720 | 1.267 | 4.040 | 12.802 |
| plain | 0.414 | 0.565 | 0.619 | 0.656 | 0.704 | 1.166 | 3.159 | 8.506 |
| spd_hard | 0.418 | 0.566 | 0.619 | 0.656 | 0.702 | 1.153 | 3.103 | 8.388 |
| spectral_soft | 0.403 | 0.562 | 0.619 | 0.662 | 0.726 | 1.244 | 3.758 | 11.510 |

Correct-matched implementation coverage conditions on drawing the stated number of correct samples:

| Model | correct coverage@4 (eligible) | @8 (eligible) | @16 (eligible) | @64 (eligible) |
| --- | ---: | ---: | ---: | ---: |
| base | 3.358 (322/500) | 5.987 (294/500) | 10.423 (251/500) | 19.875 (24/500) |
| plain | 2.856 (321/500) | 4.690 (305/500) | 7.324 (263/500) | 8.686 (51/500) |
| spd_hard | 2.810 (321/500) | 4.590 (303/500) | 7.136 (264/500) | 7.981 (53/500) |
| spectral_soft | 3.143 (323/500) | 5.451 (302/500) | 9.091 (259/500) | 12.176 (34/500) |

## Main findings

- Relative to the intended hard-projection control (`spd_hard`), `spectral_soft` restores implementation coverage: total coverage improves by +0.091 [+0.066, +0.116] at draw budget 4 and +3.122 [+2.686, +3.596] at budget 64. Correct-matched coverage improves by +0.334 [+0.290, +0.379] at four correct draws.
- `spectral_soft` also improves pass@64 over `spd_hard` by +0.024 [+0.006, +0.044]. Its pass@1 is lower by -0.014 [-0.019, -0.010]. The correctness delta fails the preregistered absolute 0.01 noninferiority margin (`noninferior=false`).
- Relative to `plain`, `spectral_soft` gains +3.004 [+2.568, +3.494] total coverage@64 and +0.295 [+0.249, +0.343] correct-matched coverage@4, with pass@1 -0.011 [-0.016, -0.006].
- Relative to the base model, `spectral_soft` improves pass@1 by +0.024 [+0.016, +0.032] and has an inconclusive pass@64 difference of +0.006 [-0.018, +0.030]. It does not fully retain base diversity: total coverage@64 differs by -1.292 [-1.786, -0.780].

## Interpretation and limits

- The result supports the mechanism claim that soft spectral attenuation preserves substantially more implementation diversity than the hard projection after five rounds. It does not meet the stated correctness noninferiority gate against `plain` or `spd_hard` at the 0.01 margin.
- `spd_hard` is an aggressive hard-projection control; its lower coverage is expected and is evidence about the cost of hard removal, rather than an optimization failure.
- Correct-matched coverage@64 has small and model-dependent eligibility (24–53 tasks; only 30 paired tasks for `spectral_soft` versus `spd_hard`). Treat it as a sparse conditional diagnostic. Budgets 4, 8, and 16 are more stable.
- This run uses one training seed (43). Bootstrap intervals resample the 500 evaluation tasks and do not measure training-seed variation.
- AST fingerprints are implementation proxies, not audited semantic algorithm labels. Local execution used the recorded Python 3.11 task-test harness; all four models have identical recorded evaluator and task-harness provenance.

## Reproducibility

- Training: 291 MBPP training tasks × 16 raw completions per task × 5 self-evolution rounds; one LoRA epoch per round.
- Evaluation: seed 43, temperature 0.8, top-p 0.95, 64 samples per each of 500 held-out tasks.
- Exact aggregate values and task-bootstrap intervals are in `summary.csv`; compact paired results are in `comparisons_compact.json`; source-file hashes are in `manifest.json`.