Update report with source-evidence selector

workspace/reports/2026-07-03.md

@@ -30,6 +30,7 @@
 - Added `scripts/slurm/eval_perdim_trainmax_dominance.sbatch` to aggregate per-dimension train-max train/validation/test rollout rows and evaluate train-clean dominance variants on held-out test once the rollout arrays complete.
 - Extended `scripts/eval_learned_dominance_selector.py` with logged `--feature-set` and `--target` variants, including success-weighted and tangent-code diagnostics, while keeping the original utility-margin default unchanged.
 - Extended `scripts/eval_learned_dominance_selector.py` again with deployment-visible `context` and `context_tangent` feature sets using target task id, instruction hash, source task id, and source-target task match. These features do not read evaluator-only val/test outcomes.
+- Extended `scripts/eval_learned_dominance_selector.py` and `scripts/eval_nonlinear_dominance_selector.py` with deployment-clean `source_evidence` feature sets and `--source-index`. These features use train-only source chart positive/negative tangent statistics and generated-to-source support distances for CTT candidates, without reading validation/test outcomes.
 - Extended `scripts/eval_dominance_selector.py` with `--score-source checkpoint`, cached chart embeddings/base scores, and single-thread torch scoring so a standalone train-only utility-energy checkpoint can be used for dominance evaluation.
 - Updated `scripts/train_utility_energy.py` to save `chart_dim` in checkpoints.
 - Added `scripts/slurm/train_utility_energy_selector.sbatch` to train full train-only utility energy and evaluate validation-calibrated dominance on held-out test measured rows.
@@ -471,6 +472,16 @@
   - `runs/ctt_base_context_obs_nonlinear_dominance_basic_success_envclip_train_to_test`: selected success `0.2708`, coverage `0.4583`, selector gap `0.2500`.
   - `runs/ctt_base_context_obs_nonlinear_dominance_context_tangent_success_envclip_train_to_test`: selected success `0.2986`, coverage `0.4653`, selector gap `0.2083`.
   - interpretation: nonlinear model capacity does not rescue the current deployment-visible selector features under `env_clip`; the best nonlinear selected success `0.3056` is below the ridge env-clip row `0.3194`.
+- Train-clean source-evidence `env_clip` dominance diagnostics completed locally:
+  - `runs/ctt_base_context_obs_learned_dominance_source_envclip_train_to_test`: selected success `0.3333`, coverage `0.2361`, fallback `0.7639`, proposal-oracle success `0.4792`, success selector gap `0.2014`.
+  - `runs/ctt_base_context_obs_learned_dominance_context_source_envclip_train_to_test`: selected success `0.3125`, coverage `0.2014`, selector gap `0.2222`.
+  - `runs/ctt_base_context_obs_learned_dominance_context_tangent_source_envclip_train_to_test`: selected success `0.2778`, coverage `0.1875`, selector gap `0.2361`.
+  - `runs/ctt_base_context_obs_learned_dominance_source_success_weighted_envclip_train_to_test`: selected success `0.3194`, coverage `0.1667`, selector gap `0.2153`.
+  - `runs/ctt_base_context_obs_learned_dominance_source_success_envclip_train_to_test`: selected success `0.2917`, coverage `0.3333`, selector gap `0.2361`.
+  - `runs/ctt_base_context_obs_learned_dominance_tangent_source_envclip_train_to_test`: selected success `0.2708`, coverage `0.1944`, selector gap `0.2431`.
+  - `runs/ctt_base_context_obs_learned_dominance_context_source_success_weighted_envclip_train_to_test`: selected success `0.3125`, coverage `0.2292`, selector gap `0.2222`.
+  - `runs/ctt_base_context_obs_nonlinear_dominance_source_positive_envclip_train_to_test`: selected success `0.2569`, coverage `0.3958`, selector gap `0.2431`.
+  - interpretation: CTT-specific source evidence is useful only in the simple ridge utility-margin form. It improves the best train-clean env-clip selector from `0.3194` to `0.3333`, but still leaves a `0.2014` selector gap to the `0.4792` proposal oracle.
 
 ## Failed Jobs And Log Excerpts
 
@@ -544,6 +555,7 @@ Interpretation: residual CTT beats local-atlas on mean positive distance while s
 | test + base-context-observation train nonlinear dominance | 3 | 144 | 0.2917 | 0.3056 | 0.5139 | 0.7292 | 0.5347 | 0.2639 | 0.2569 | fail: selected/support/selector |
 | test + base-context-observation env_clip train learned dominance | 3 | 144 | 0.2917 | 0.3194 | 0.4792 | 0.7292 | 0.5069 | 0.2917 | 0.2153 | fail: selected/support/selector |
 | test + base-context-observation env_clip train nonlinear dominance | 3 | 144 | 0.2917 | 0.3056 | 0.4792 | 0.7292 | 0.5069 | 0.2917 | 0.2222 | fail: selected/support/selector |
+| test + base-context-observation env_clip train source-evidence dominance | 3 | 144 | 0.2917 | 0.3333 | 0.4792 | 0.7292 | 0.5069 | 0.2917 | 0.2014 | fail: selected/support/selector |
 | test + utility-energy checkpoint seed0 | 1 | 144 | 0.2847 | 0.2708 | 0.5069 | 0.7292 | 0.5278 | 0.2639 | 0.2917 | fail: selected/support/selector |
 | test + utility-energy checkpoint seed1 | 1 | 144 | 0.2847 | 0.2847 | 0.5069 | 0.7292 | 0.5278 | 0.2639 | 0.2847 | fail: selected/support/selector |
 | test + utility-energy checkpoint seed2 | 1 | 144 | 0.2847 | 0.2361 | 0.5069 | 0.7292 | 0.5278 | 0.2639 | 0.3056 | fail: selected/support/selector |
@@ -553,7 +565,7 @@ Interpretation: residual CTT beats local-atlas on mean positive distance while s
 | test + base-context-observation tanh learned context-tangent success dominance | 3 | 144 | 0.3333 | 0.3542 | 0.3819 | 0.7292 | 0.4167 | 0.3681 | 0.1528 | fail: support/selector |
 | test + base-context-observation tanh train learned context-success dominance | 3 | 144 | 0.3333 | 0.3819 | 0.3819 | 0.7292 | 0.4167 | 0.3681 | 0.1250 | fail: support/selector |
 
-Interpretation: the test proposal oracle exceeds 50% support for the clipped visual-stat row, and visual-stat CTT plus validation-calibrated learned context dominance gives the best clipped-convention held-out selected success (`0.3264`). The cleaner clipped-convention train-calibrated selector reaches `0.3125`, which still improves over base but does not beat the validation-calibrated diagnostic. Nonlinear train-calibrated selectors do not improve this number. The object-layout proxy pass does not survive measured validation rollout and is worse than the RGB-stat row on proposal oracle/selected success. No-clipping preserves measurable support but exposes widespread action-bound violations; global scaled replay fixes base bound validity but collapses proposal support. The bounded tanh convention eliminates action-bound violations on validation/test, while per-dimension train-max replay fixes base validity but further collapses proposal support and still leaves generated action-bound violations. With train-clean context-success dominance, bounded tanh reaches `0.3819` selected success on held-out test and matches the measured proposal oracle in micro success, but the row-level selector gap remains `0.1250`, its proposal oracle is also only `0.3819`, and the support gap to the hidden chart oracle remains `0.3681`. The explicit `env_clip` convention is action-bound-clean and preserves more bounded proposal support than tanh or per-dimension scaling, reaching `0.4792` held-out proposal-oracle success, but score-only selection is below base and the best train-clean env-clip dominance row reaches only `0.3194` selected success. The nonlinear env-clip sweep tops out at `0.3056`, so adding model flexibility to the current features is not enough. The current bounded-action story is therefore an action-convention fix plus a selector/dominance failure. The next paper-critical work is richer learned visual-language/object-centric chart representation and a learned decoder/selector that exploits the preserved proposal support, not another deterministic proxy descriptor.
+Interpretation: the test proposal oracle exceeds 50% support for the clipped visual-stat row, and visual-stat CTT plus validation-calibrated learned context dominance gives the best clipped-convention held-out selected success (`0.3264`). The cleaner clipped-convention train-calibrated selector reaches `0.3125`, which still improves over base but does not beat the validation-calibrated diagnostic. Nonlinear train-calibrated selectors do not improve this number. The object-layout proxy pass does not survive measured validation rollout and is worse than the RGB-stat row on proposal oracle/selected success. No-clipping preserves measurable support but exposes widespread action-bound violations; global scaled replay fixes base bound validity but collapses proposal support. The bounded tanh convention eliminates action-bound violations on validation/test, while per-dimension train-max replay fixes base validity but further collapses proposal support and still leaves generated action-bound violations. With train-clean context-success dominance, bounded tanh reaches `0.3819` selected success on held-out test and matches the measured proposal oracle in micro success, but the row-level selector gap remains `0.1250`, its proposal oracle is also only `0.3819`, and the support gap to the hidden chart oracle remains `0.3681`. The explicit `env_clip` convention is action-bound-clean and preserves more bounded proposal support than tanh or per-dimension scaling, reaching `0.4792` held-out proposal-oracle success. Source-evidence dominance exploits the CTT fact that every proposal comes from a measured train-positive source chart and improves train-clean env-clip selected success to `0.3333`, but still leaves a `0.2014` selector gap. The current bounded-action story is therefore an action-convention fix plus a partial CTT-specific selector improvement, not a final deployment method. The next paper-critical work is richer learned visual-language/object-centric chart representation and a learned decoder/selector that exploits the preserved proposal support, not another deterministic proxy descriptor.
 
 Safety metric status: regenerated measured rollout reports now expose `base_safety_label_known`, `generated_safety_label_coverage_at_8`, and `selected_safety_label_known_at_8`. For the visual-stat, object-layout, held-out test, and train-calibration rollout rows generated before the action-bound hook, each coverage number is `0.0000`; the legacy validation aggregate without `candidate_outcomes` reports `n/a`. For action-bound smoke/no-clipping/scaled artifacts, coverage is `1.0000` because `safety_violation_source=action_bounds`. These are action-space validity labels only, not collision/contact labels.
 
@@ -621,6 +633,7 @@ Action-bound audit status: new rollout artifacts now log action-bound safety lab
 - Added explicit `env_clip` validation/test aggregate tables via `\input{../runs/ctt_base_context_obs_val_envclip_rollout_comparison/table}` and `\input{../runs/ctt_base_context_obs_test_envclip_rollout_comparison/table}`. The draft treats `env_clip` as the strongest bounded-action support diagnostic so far: action-bound labels are complete and clean, proposal support is preserved better than tanh/per-dimension scaling, but score-only selected success remains below base.
 - Added the train-calibrated `env_clip` learned-dominance table via `\input{../runs/ctt_base_context_obs_learned_dominance_basic_envclip_train_to_test/table}`. The draft states the split precisely: `env_clip` fixes the bounded-action support confound, while the current dominance features still leave a large selector gap.
 - Added a short nonlinear `env_clip` selector note to the paper: the best train-clean nonlinear row reaches only `30.56%` held-out selected success, below the ridge env-clip row, so the current blocker is feature/representation quality rather than simply linear selector capacity.
+- Added the train-calibrated `env_clip` source-evidence selector table via `\input{../runs/ctt_base_context_obs_learned_dominance_source_envclip_train_to_test/table}`. The paper explains why train-only source-chart evidence is deployment-clean for CTT and reports the improved held-out selected success `33.33%`, while keeping the selector-gap failure explicit.
 - Verification after the bounded-tanh/dominance paper patch:
   - `.venv/bin/python -m py_compile scripts/eval_ctt_generated_rollout.py tests/test_ctt.py scripts/eval_dominance_selector.py scripts/eval_learned_dominance_selector.py`
   - `.venv/bin/python -m pytest tests/test_ctt.py tests/test_metrics.py tests/test_cil_metrics.py tests/test_dominance_selector.py tests/test_chart_features.py -q`: 36 passed.
@@ -709,7 +722,7 @@ Action-bound audit status: new rollout artifacts now log action-bound safety lab
 
 1. Treat `env_clip` as the current declared bounded-action diagnostic convention: it is action-bound-clean and support-preserving enough to expose the selector/dominance blocker.
 2. Replace deterministic RGB/object descriptors with learned visual-language or task/object tokens that distinguish target, distractor, gripper/contact region, and task stage.
-3. Train a dominance/utility selector that uses richer deployment-visible chart features under `env_clip`; the current basic ridge row reaches only `0.3194` selected success against a `0.4792` proposal oracle, and the nonlinear sweep does not improve it.
+3. Train a dominance/utility selector that uses richer deployment-visible chart features under `env_clip`; source-evidence ridge improves to `0.3333` selected success against a `0.4792` proposal oracle, but the remaining `0.2014` selector gap is still too large.
 4. Improve the decoder/action representation beyond post-hoc scalar or per-dimension scaling; both collapse support, while tanh trades support for validity.
 5. Stop treating deterministic object-layout as a likely win: it passed proxy but failed measured support relative to RGB-stat.
 6. Rerun measured support/selection once after the learned representation/selector fix under exactly one declared deployment action convention.