auto-sync 2026-07-04T01:28:02Z workspace (part 3)

workspace/reports/2026-07-03.md

@@ -33,6 +33,7 @@
 - Extended `scripts/eval_learned_dominance_selector.py` and `scripts/eval_nonlinear_dominance_selector.py` with deployment-clean `source_evidence` feature sets and `--source-index`. These features use train-only source chart positive/negative tangent statistics and generated-to-source support distances for CTT candidates, without reading validation/test outcomes.
 - Extended `scripts/eval_learned_dominance_selector.py` with `--threshold-scope task`, a Mondrian execute/fallback threshold over visible `task_id` buckets. The model weights and task thresholds are fit on calibration rows only; held-out eval outcomes are used only for reporting.
 - Extended `scripts/eval_learned_dominance_selector.py` with `--fit-objective {pointwise,pairwise,hybrid_pairwise}`. The pairwise objectives train the ridge selector on within-chart measured causal comparisons from calibration rows only, then calibrate execute/fallback thresholds on candidate-row predictions. Added regression coverage in `tests/test_dominance_selector.py` for pairwise design rows outnumbering candidate rows.
+- Extended `scripts/eval_learned_dominance_selector.py` with deployment-visible chart-compatibility feature sets and `--selector-chart-feature-mode`. The new path loads target/source chart features without hidden outcomes, compares whole/base/RGB-stat/object-layout segments, logs selector/source split hashes, and has regression coverage in `tests/test_dominance_selector.py`.
 - Fixed `scripts/summarize_ctt_runs.py` so summary artifacts include `ctt*_learned_dominance*/metrics.json` runs such as `ctt_base_context_obs_learned_dominance_*`, not only the older `ctt_learned_dominance*` naming pattern.
 - Extended `scripts/eval_dominance_selector.py` with `--score-source checkpoint`, cached chart embeddings/base scores, and single-thread torch scoring so a standalone train-only utility-energy checkpoint can be used for dominance evaluation.
 - Updated `scripts/train_utility_energy.py` to save `chart_dim` in checkpoints.
@@ -568,7 +569,7 @@ Interpretation: residual CTT beats local-atlas on mean positive distance while s
 | test + base-context-observation tanh learned context-tangent success dominance | 3 | 144 | 0.3333 | 0.3542 | 0.3819 | 0.7292 | 0.4167 | 0.3681 | 0.1528 | fail: support/selector |
 | test + base-context-observation tanh train learned context-success dominance | 3 | 144 | 0.3333 | 0.3819 | 0.3819 | 0.7292 | 0.4167 | 0.3681 | 0.1250 | fail: support/selector |
 
-Interpretation: the test proposal oracle exceeds 50% support for the clipped visual-stat row, and visual-stat CTT plus validation-calibrated learned context dominance gives the best clipped-convention held-out selected success (`0.3264`). The cleaner clipped-convention train-calibrated selector reaches `0.3125`, which still improves over base but does not beat the validation-calibrated diagnostic. Nonlinear train-calibrated selectors do not improve this number. The object-layout proxy pass does not survive measured validation rollout and is worse than the RGB-stat row on proposal oracle/selected success. No-clipping preserves measurable support but exposes widespread action-bound violations; global scaled replay fixes base bound validity but collapses proposal support. The bounded tanh convention eliminates action-bound violations on validation/test, while per-dimension train-max replay fixes base validity but further collapses proposal support and still leaves generated action-bound violations. With train-clean context-success dominance, bounded tanh reaches `0.3819` selected success on held-out test and matches the measured proposal oracle in micro success, but the row-level selector gap remains `0.1250`, its proposal oracle is also only `0.3819`, and the support gap to the hidden chart oracle remains `0.3681`. The explicit `env_clip` convention is action-bound-clean and preserves more bounded proposal support than tanh or per-dimension scaling, reaching `0.4792` held-out proposal-oracle success. Source-evidence dominance exploits the CTT fact that every proposal comes from a measured train-positive source chart and improves train-clean env-clip selected success to `0.3333`, but still leaves a `0.2014` selector gap. The current bounded-action story is therefore an action-convention fix plus a partial CTT-specific selector improvement, not a final deployment method. The next paper-critical work is richer learned visual-language/object-centric chart representation and a learned decoder/selector that exploits the preserved proposal support, not another deterministic proxy descriptor.
+Interpretation: the test proposal oracle exceeds 50% support for the pre-K16 clipped visual-stat row, and visual-stat CTT plus validation-calibrated learned context dominance gives the best K=8 clipped-convention held-out selected success (`0.3264`). The cleaner clipped-convention train-calibrated selector reaches `0.3125`, which still improves over base but does not beat the validation-calibrated diagnostic. Nonlinear train-calibrated selectors do not improve this number. The object-layout proxy pass does not survive measured validation rollout and is worse than the RGB-stat row on proposal oracle/selected success. No-clipping preserves measurable support but exposes widespread action-bound violations; global scaled replay fixes base bound validity but collapses proposal support. The bounded tanh convention eliminates action-bound violations on validation/test, while per-dimension train-max replay fixes base validity but further collapses proposal support and still leaves generated action-bound violations. With train-clean context-success dominance, bounded tanh reaches `0.3819` selected success on held-out test and matches the measured proposal oracle in micro success, but the row-level selector gap remains `0.1250`, its proposal oracle is also only `0.3819`, and the support gap to the hidden chart oracle remains `0.3681`. The explicit `env_clip` convention is action-bound-clean and preserves more bounded proposal support than tanh or per-dimension scaling, reaching `0.4792` held-out proposal-oracle success. Source-evidence dominance exploits the CTT fact that every proposal comes from a measured train-positive source chart and improves train-clean env-clip selected success to `0.3333`, but still leaves a `0.2014` selector gap. The current bounded-action story is therefore an action-convention fix plus a partial CTT-specific selector improvement, not a final deployment method. The next paper-critical work is richer learned visual-language/object-centric chart representation and a learned decoder/selector that exploits the preserved proposal support, not another deterministic proxy descriptor.
 
 Safety metric status: regenerated measured rollout reports now expose `base_safety_label_known`, `generated_safety_label_coverage_at_8`, and `selected_safety_label_known_at_8`. For the visual-stat, object-layout, held-out test, and train-calibration rollout rows generated before the action-bound hook, each coverage number is `0.0000`; the legacy validation aggregate without `candidate_outcomes` reports `n/a`. For action-bound smoke/no-clipping/scaled artifacts, coverage is `1.0000` because `safety_violation_source=action_bounds`. These are action-space validity labels only, not collision/contact labels.
 
@@ -714,6 +715,11 @@ Action-bound audit status: new rollout artifacts now log action-bound safety lab
     - `runs/ctt_base_context_obs_learned_dominance_basic_success_weighted_hybrid_task_envclip_k16_train_to_test`: selected success `0.3125`, coverage `0.4028`, selector gap `0.2847`
     - `runs/ctt_base_context_obs_learned_dominance_tangent_success_weighted_hybrid_task_envclip_k16_train_to_test`: selected success `0.3056`, coverage `0.4653`, selector gap `0.2778`
     - `runs/ctt_base_context_obs_learned_dominance_source_success_weighted_hybrid_task_envclip_k16_train_to_test`: selected success `0.2917`, coverage `0.5694`, selector gap `0.2986`
+  - chart-compatibility K=16 ridge diagnostics:
+    - `runs/ctt_base_context_obs_learned_dominance_chartcompat_obs_utility_task_envclip_k16_train_to_test`: selected success `0.3542`, coverage `0.6597`, selector gap `0.2431`
+    - `runs/ctt_base_context_obs_learned_dominance_chartcompat_obs_success_weighted_task_envclip_k16_train_to_test`: selected success `0.3333`, coverage `0.7222`, selector gap `0.2569`
+    - `runs/ctt_base_context_obs_learned_dominance_chartcompat_obj_success_weighted_task_envclip_k16_train_to_test`: selected success `0.3125`, coverage `0.5486`, selector gap `0.2847`
+    - `runs/ctt_base_context_obs_learned_dominance_chartcompat_success_weighted_task_envclip_k16_train_to_test`: selected success `0.3264`, coverage `0.5486`, selector gap `0.2639`
 
 ### Best Metric Table
 
@@ -722,18 +728,20 @@ Action-bound audit status: new rollout artifacts now log action-bound safety lab
 | validation K=16 env_clip | 207 | 0.2754 | 0.2464 | 0.4638 | 0.6667 | 0.5362 | 0.2174 | 0.2174 | 0.0000 |
 | train-cal K=16 env_clip | 432 | 0.2778 | 0.2477 | 0.5370 | 0.6597 | 0.5463 | 0.1759 | 0.2894 | 0.0000 |
 | held-out test K=16 env_clip | 144 | 0.2917 | 0.2778 | 0.5694 | 0.7292 | 0.5486 | 0.2014 | 0.2917 | 0.0000 |
-| test K=16 best train-clean selector | 144 | 0.2917 | 0.3264 | 0.5694 | 0.7292 | 0.5486 | 0.2014 | 0.2708 | 0.0000 |
+| test K=16 best train-clean selector | 144 | 0.2917 | 0.3542 | 0.5694 | 0.7292 | 0.5486 | 0.2014 | 0.2431 | 0.0000 |
 
 Interpretation: K=16 is the first action-bound-clean held-out support result
 that clears the proposal-oracle target from the advisor comment: the measured
 test proposal oracle is 56.94% and OutcomePTR@16 is 54.86%. This is strong
 support evidence for the CTT/Atlas thesis. It is not a selected-action
 deployment success: score-only selection remains below base, and the best
-train-clean K=16 selector reaches only 32.64% with a 27.08-point selector gap.
-The small selector gain comes from a success-weighted ridge model with
-task-specific Mondrian thresholds. Source-evidence, nonlinear, pairwise
-within-chart ranking, hybrid pointwise-plus-pairwise ranking, and other
-task-scoped variants do not close the gap; the selector bottleneck remains.
+train-clean K=16 selector now reaches 35.42% with a 24.31-point selector gap.
+The gain comes from train-only RGB-stat target--source chart compatibility
+with a utility-margin ridge model and task-specific Mondrian thresholds. The
+previous basic success-weighted task-threshold row reached 32.64%, and the
+RGB-stat success-weighted row reaches 33.33%. Object-layout, source-evidence,
+nonlinear, pairwise within-chart ranking, and hybrid pointwise-plus-pairwise
+variants do not resolve the selector bottleneck.
 
 ### Paper Changes
 
@@ -742,13 +750,17 @@ task-scoped variants do not close the gap; the selector bottleneck remains.
 - Added held-out K=16 measured rollout table via
   `\input{../runs/ctt_base_context_obs_test_envclip_k16_rollout_comparison/table}`.
 - Added K=16 train-calibrated selector table via
-  `\input{../runs/ctt_base_context_obs_learned_dominance_basic_success_weighted_task_envclip_k16_train_to_test/table}`.
+  `\input{../runs/ctt_base_context_obs_learned_dominance_chartcompat_obs_utility_task_envclip_k16_train_to_test/table}`.
 - Updated the interpretation around the advisor spine: the current CTT
   implementation can now find positive do-action support, while utility
   energy/calibrated dominance remains the blocker.
 - Added a pairwise/listwise selector diagnostic paragraph to `latex/main.tex`:
   train-clean within-chart causal-ranking objectives do not improve over the
   current pointwise task-scoped ridge selector.
+- Added the chart-compatibility K=16 selector interpretation to
+  `latex/main.tex`: train-only RGB-stat target--source compatibility improves
+  selected success to `0.3542`, but the remaining selector gap keeps it a
+  diagnostic rather than a solved deployment method.
 
 ### Leakage and Data Status
 
@@ -834,7 +846,7 @@ task-scoped variants do not close the gap; the selector bottleneck remains.
 
 1. Treat K=16 `env_clip` as the current declared bounded-action diagnostic convention: it is action-bound-clean and now support-strong on held-out test (`0.5694` proposal oracle, `0.5486` OutcomePTR@16), exposing the selector/dominance blocker sharply.
 2. Replace deterministic RGB/object descriptors with learned visual-language or task/object tokens that distinguish target, distractor, gripper/contact region, and task stage.
-3. Train a dominance/utility selector that uses richer deployment-visible chart features under K=16 `env_clip`; the best train-clean row is only `0.3264` selected success against a `0.5694` proposal oracle, leaving a `0.2708` selector gap.
+3. Train a dominance/utility selector that uses richer deployment-visible chart features under K=16 `env_clip`; the best train-clean row is now `0.3542` selected success against a `0.5694` proposal oracle, leaving a `0.2431` selector gap.
 4. Improve the decoder/action representation beyond post-hoc scalar or per-dimension scaling; both collapse support, while tanh trades support for validity.
 5. Stop treating deterministic object-layout as a likely win: it passed proxy but failed measured support relative to RGB-stat.
 6. Rerun measured support/selection after the learned representation/selector fix under exactly the K=16 `env_clip` action convention before changing the support generator again.

workspace/reports/draft_audit_ctt.md

@@ -85,8 +85,8 @@ improves support and score-only selected success relative to the original
 residual CTT test rollout, but it still does not beat base. The best new
 selector diagnostic is validation-calibrated learned context dominance over the
 visual-stat rows: selected test success = 0.3264 at 50.69% coverage. This is
-the strongest current held-out selected-success row, but still far below the
-deployment gate. The draft must keep it as a measured support/selector
+the strongest K=8 visual-stat held-out selected-success row, but still far
+below the deployment gate. The draft must keep it as a measured support/selector
 diagnostic, not a final deployment or object-centric representation claim.
 The cleaner train-calibrated follow-up uses 432 train measured rollout rows
 with same-chart and same-state source retrieval excluded. Its best learned
@@ -121,10 +121,23 @@ weights can be trained from within-chart measured causal comparisons rather
 than only pointwise candidate margins. On held-out K=16 `env_clip` rows, the
 pairwise-only basic selector reaches 0.2917 selected success, and the best
 hybrid pointwise-plus-pairwise row reaches only 0.3125. Both are below the
-current best pointwise task-scoped row at 0.3264. This strengthens the audit
-conclusion: the selector gap is not closed by merely changing the ranking loss;
-the missing piece is richer deployment-visible chart/outcome representation or
-a stronger calibrated utility model.
+current best chart-compatibility row at 0.3542. This strengthens the audit
+conclusion: merely changing the ridge ranking loss is insufficient; the missing
+piece is richer deployment-visible chart/outcome representation or a stronger
+calibrated utility model.
+The chart-compatibility follow-up is the first K=16 train-clean selector
+improvement that changes the best selected-success row. It adds
+deployment-visible target--source chart feature comparisons without loading
+hidden validation/test outcomes. The strongest run,
+`runs/ctt_base_context_obs_learned_dominance_chartcompat_obs_utility_task_envclip_k16_train_to_test`,
+uses RGB-stat chart compatibility and a utility-margin ridge objective,
+reaching 0.3542 selected held-out test success at 0.6597 coverage. The same
+proposal set has base success 0.2917, proposal-oracle success 0.5694, and a
+0.2431 success selector gap. This is a useful representation-side selector
+diagnostic, but it is still hand-built RGB statistics rather than learned
+object-centric CTT geometry. Object-layout-only reaches 0.3125, combined
+RGB-stat/object-layout ties 0.3264, and RGB-stat success-weighted reaches
+0.3333, so deterministic object-layout features are not yet the missing method.
 The current continuation adds a leakage-clean deterministic object-layout chart
 token path, but it is not yet promoted as a method result. The exporter
 `scripts/export_chart_object_embeddings.py` writes 64D RGB foreground
@@ -318,6 +331,11 @@ Current artifact evidence:
   rows. The best row is
   `runs/ctt_base_context_obs_learned_dominance_context_val_to_test` with
   selected success 0.3264.
+- `runs/ctt_base_context_obs_learned_dominance_chartcompat_obs_utility_task_envclip_k16_train_to_test`
+  contains the strongest current K=16 train-clean selector diagnostic:
+  selected success 0.3542, coverage 0.6597, proposal oracle 0.5694, and
+  success selector gap 0.2431. It uses train-only target--source RGB-stat
+  chart compatibility and remains a selector diagnostic.
 - `runs/ctt_base_context_obs_{val,test}_tanh_rollout_comparison` contain the
   full bounded-decoder tanh measured validation/test diagnostics. They are
   action-bound-clean but support-negative relative to clipped/no-clipping
@@ -552,29 +570,34 @@ Current missing:
   as deployment evidence.
 - Real robot near-miss recovery audit.
 - A dominance model that materially reduces the selected-success selector gap. The current
-  learned ridge row only marginally beats the base, and the full train-only
-  utility-energy checkpoints do not beat base. Coarse context metadata also
-  does not beat the best learned ridge row.
+  best train-clean chart-compatibility ridge row improves selected success to
+  0.3542 against a 0.2917 base, but still leaves a 0.2431 gap to the 0.5694
+  proposal oracle. The full train-only utility-energy checkpoints do not beat
+  base, and coarse context metadata does not beat the learned ridge rows.
 - Learned visual-language/object feature export and selector transfer. The
   deterministic RGB-stat export is implemented, leakage-audited, and improves
-  measured support plus validation-calibrated selected success, but the best
-  clipped-convention selected success is still only 0.3264. The bounded tanh
-  train-clean selector diagnostic reaches 0.3819 selected success and matches
-  the tanh proposal oracle in micro success, but still has a 0.1250 row-level
-  selector gap and lower proposal support than the clipped visual-stat row.
-  These are still hand-built visual/statistical tokens rather than the
-  object-centric representation needed for a strong method claim.
+  measured support plus selected success. Under K=16 `env_clip`, RGB-stat
+  target--source chart compatibility reaches 0.3542 selected held-out test
+  success, but still has a 0.2431 selector gap to the 0.5694 proposal oracle.
+  The bounded tanh train-clean selector diagnostic reaches 0.3819 selected
+  success and matches the tanh proposal oracle in micro success, but still has
+  a 0.1250 row-level selector gap and lower proposal support than the clipped
+  visual-stat row. These are still hand-built visual/statistical tokens rather
+  than the object-centric representation needed for a strong method claim.
 - K=16 `env_clip` support changes the current gate diagnosis. It is
   action-bound-clean and reaches held-out test proposal-oracle success 0.5694
   with OutcomePTR@16 0.5486, support gap 0.2014, and zero action-bound unsafe
   rate. This is the strongest evidence so far that CTT transport can find
   measured positive do-action support from train-positive source tangents. It
   is still not a selected-action deployment result: score-only selection is
-  0.2778, below the 0.2917 base action. Adding a train-clean
-  success-weighted ridge selector with task-specific Mondrian thresholds
-  improves selected success only to 0.3264, with a 0.2708 selector gap to the
-  proposal oracle. Nonlinear K=16 selectors and source-evidence/task-threshold
-  variants do not materially improve this.
+  0.2778, below the 0.2917 base action. Adding train-only RGB-stat
+  target--source chart compatibility with a utility-margin ridge objective
+  improves selected success to 0.3542, with a 0.2431 selector gap to the
+  proposal oracle. This is better than the earlier 0.3264 basic
+  success-weighted task-threshold row, but not enough for a method-success
+  claim. Nonlinear K=16 selectors, source-evidence/task-threshold variants,
+  deterministic object-layout compatibility, and pairwise/hybrid ridge
+  objectives do not materially improve the diagnosis.
 
 ## Immediate Fix Order
 
@@ -597,11 +620,11 @@ Next:
    selection on the K=16 `env_clip` rows.
 2. Move from deterministic RGB statistics to learned visual-language or
    object-centric chart tokens; the current K=16 support row reaches 0.5694
-   proposal oracle, but the best train-clean selector reaches only 0.3264.
+   proposal oracle, but the best train-clean selector reaches only 0.3542.
 3. Replace the validation-calibrated ridge/context selector with a stronger
    deployment-clean train-only dominance model. The new pairwise/hybrid ridge
    diagnostic is negative: within-chart causal-ranking loss alone does not beat
-   the current pointwise task-scoped K=16 selector.
+   the current chart-compatibility K=16 selector.
 4. Fix action representation/decoding rather than applying a global scalar:
    no-clipping jobs `15117739`--`15117744` show widespread bound violations,
    while scaled0.215 fixes base bounds but collapses generated support and