auto-sync 2026-07-03T17:25:27Z workspace (part 4)

workspace/reports/2026-07-03.md

@@ -28,6 +28,10 @@
 - Added `scripts/slurm/reexport_rgb_ref_cil_charts.sbatch` to re-export a non-destructive `data/cil_charts_rgb_refs/{train,val,test}` chart DB and run leakage plus feature-source audits after rendering succeeds.
 - Added `scripts/export_chart_observation_embeddings.py` to decode deployment-visible RGB observation refs into deterministic 32D `rgb_jpeg_stats_v1` embeddings without reading outcomes, labels, or hidden branches.
 - Extended `cil/chart_features.py` with `base_context_obs`, a leakage-audited visual-stat diagnostic mode that appends the 32D observation embedding to the base/context chart token.
+- Added `scripts/export_chart_object_embeddings.py` to decode deployment-visible RGB observation refs into deterministic 64D `rgb_object_layout_v1` foreground component/layout embeddings without reading outcomes, labels, or hidden branches.
+- Extended `cil/chart_features.py` with `base_context_obj` and `base_context_obs_obj` so CTT can test object-layout-only and RGB-stat-plus-object-layout chart tokens.
+- Updated `scripts/audit_chart_feature_sources.py` to audit `object_embedding_path` coverage and report feature dimensions for the new object-layout modes.
+- Updated selector/comparison scripts so new metrics artifacts expose top-level `data_hash` and `split_hash`.
 - Updated CTT loaders to attach `_chart_root` metadata so relative observation-embedding paths resolve correctly in train, proxy, rollout, utility-energy, and dominance code paths.
 - Added `scripts/slurm/train_ctt_feature_proxy.sbatch` for feature-mode CTT proxy sweeps; it trains residual CTT and refreshes the proxy comparison/summary artifacts for a specified chart feature mode.
 - Fixed validation/test dominance selectors to load chart maps with the checkpoint's recorded chart feature mode. This prevents 112D `base` features from being fed into 162D `base_context_obs` checkpoints when row-scored rollouts still need checkpoint base scores.
@@ -83,6 +87,17 @@
   - `15115082`: `base_context_obs` seed 1, test 48 positive-support charts, K=8
   - `15115083`: `base_context_obs` seed 2, test 48 positive-support charts, K=8
   - purpose: check whether the validation support-side improvement transfers to the held-out test split.
+- Submitted object-layout CTT proxy arrays:
+  - `15116834_[0-2]`: residual CTT, `--chart-feature-mode base_context_obj`, validation proxy, K=16
+  - `15116833_[0-2]`: residual CTT, `--chart-feature-mode base_context_obs_obj`, validation proxy, K=16
+  - final scheduler state: all six array tasks completed with exit code `0:0` and empty stderr.
+  - purpose: test whether object-layout chart geometry improves proxy support before any rollout claim.
+- Submitted measured object-layout CTT validation rollout jobs:
+  - `15116890`: `base_context_obj` seed 0, validation 69 positive-support charts, K=8
+  - `15116891`: `base_context_obj` seed 1, validation 69 positive-support charts, K=8
+  - `15116892`: `base_context_obj` seed 2, validation 69 positive-support charts, K=8
+  - final scheduler state: all three jobs completed with exit code `0:0`.
+  - purpose: object-layout proxy passed the support/safety gate by mean positive distance, so the next evidence must be measured OutcomePTR rather than another proxy claim.
 
 ## Jobs Completed
 
@@ -194,10 +209,34 @@
   - val: 6,704/6,704 rows with embeddings, 419 unique observation refs
   - test: 6,560/6,560 rows with embeddings, 410 unique observation refs
   - interpretation: this is a simple RGB-stat feature export, not a learned visual-language or object-centric representation.
+- RGB object-layout embedding export completed: `runs/chart_object_embeddings_rgb_refs`.
+  - train: 32,704/32,704 rows with object-layout embeddings, 2,044 unique observation refs
+  - val: 6,704/6,704 rows with object-layout embeddings, 419 unique observation refs
+  - test: 6,560/6,560 rows with object-layout embeddings, 410 unique observation refs
+  - extractor: deterministic `rgb_object_layout_v1`, 64D foreground component/layout descriptor, no outcomes read.
 - RGB-ref leakage audit completed: `runs/leakage_audit_rgb_refs`, status `pass`, 0 violations.
 - RGB-ref chart-feature audit completed: `runs/chart_feature_audit_rgb_refs`.
   - train/val/test: 100% `observation_ref`, 100% `observation_embedding_path`, 100% scene id, and 100% instruction coverage
   - feature dims: `base=112`, `base_context=130`, `base_context_obs=162`
+- Object-layout RGB-ref leakage audit completed: `runs/leakage_audit_rgb_refs_object`, status `pass`, 0 violations.
+- Object-layout chart-feature audit completed: `runs/chart_feature_audit_rgb_refs_object`.
+  - train/val/test: 100% `object_embedding_path`, 100% `observation_embedding_path`, 100% `observation_ref`
+  - feature dims: `base=112`, `base_context=130`, `base_context_obs=162`, `base_context_obj=194`, `base_context_obs_obj=226`
+- Local object-layout CTT smoke completed:
+  - `runs/ctt_residual_base_context_obj_smoke_seed0` and `_val_proxy`: train/eval shape path passes on 64 train charts and 8 validation target charts.
+  - `runs/ctt_residual_base_context_obs_obj_smoke_seed0` and `_val_proxy`: train/eval shape path passes on 64 train charts and 8 validation target charts.
+  - interpretation: smoke validates feature loading and proxy path only; full three-seed proxy jobs completed next.
+- Object-layout CTT proxy diagnostics completed for three seeds:
+  - `runs/ctt_residual_base_context_obj_seed{0,1,2}` and `_val_proxy`: PPTC@0.20 `0.2271`, PPTC@0.40 `0.6425`, NegativeNear@0.20 `0.0380`, mean positive distance `0.4340`, diversity `0.2417`, proxy gate `pass`.
+  - `runs/ctt_residual_base_context_obs_obj_seed{0,1,2}` and `_val_proxy`: PPTC@0.20 `0.2077`, PPTC@0.40 `0.6425`, NegativeNear@0.20 `0.0285`, mean positive distance `0.4429`, diversity `0.2448`, proxy gate `pass`.
+  - interpretation: object-layout-only slightly improves mean positive distance over the RGB-stat row (`0.4340` vs `0.4347`) while staying inside the NegativeNear safety slack, but it does not beat RGB-stat on PPTC. This is enough to submit measured validation rollout, not enough for an OutcomePTR claim.
+- Measured object-layout CTT validation rollout jobs `15116890`, `15116891`, and `15116892` completed with exit code `0:0`.
+  - seed artifacts: `runs/ctt_residual_base_context_obj_rollout_val69_seed{0,1,2}`
+  - aggregate artifact: `runs/ctt_base_context_obj_val_rollout_comparison`
+  - rows: 207 validation measured rows = 69 charts x 3 train seeds, K=8
+  - selected success `0.2029`, base success `0.2754`, proposal oracle `0.3816`, hidden oracle `0.6667`
+  - OutcomePTR@8 `0.5024`, success support gap `0.2947`, success selector gap `0.1787`
+  - interpretation: proxy gate pass did not translate into better measured support. Object-layout-only is worse than RGB-stat on proposal oracle (`0.3816` vs `0.4058`) and selected success (`0.2029` vs `0.2415`), so it is a negative representation diagnostic.
 - Base-context-observation CTT proxy diagnostics completed for three seeds:
   - train artifacts: `runs/ctt_residual_base_context_obs_seed{0,1,2}`
   - proxy artifacts: `runs/ctt_residual_base_context_obs_seed{0,1,2}_val_proxy`
@@ -298,6 +337,7 @@
 - Local `nohup` HF daemon exited after auth check without a clear error. Slurm daemon `15106830` started but failed HF auth from the compute node after a 60s timeout, so it was canceled and replaced with local `setsid` daemon PID `615094`, which passed auth and began syncing.
 - Visual-stat chart-token proxy array `15114949_[1-2]` completed successfully; no failure log.
 - First local visual-stat dominance/learned-selector attempts failed with `RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x112 and 162x128)` because row-scored selectors loaded chart maps in default `base` mode while base-score computation used 162D `base_context_obs` checkpoints. Fixed by making dominance scorers expose the checkpoint chart feature mode to chart-map loading, then reran successfully.
+- First object-layout local smoke attempt failed with `libgomp: Thread creation failed: Resource temporarily unavailable` under default thread settings. Reran successfully with `OMP_NUM_THREADS=1`, `OPENBLAS_NUM_THREADS=1`, `MKL_NUM_THREADS=1`, and `NUMEXPR_NUM_THREADS=1`; the Slurm wrapper already sets single-thread CPU env vars.
 
 ## Best Metric Table
 
@@ -308,9 +348,11 @@
 | CTT residual full | 3 | 207 | 0.1981 | 0.6087 | 0.0296 | 0.4509 | 0.2703 | proxy pass |
 | CTT residual base-context | 1 | 69 | 0.1739 | 0.6232 | 0.0182 | 0.4429 | 0.2471 | proxy pass |
 | CTT residual base-context-observation | 3 | 207 | 0.2464 | 0.6425 | 0.0343 | 0.4347 | 0.2397 | proxy pass |
+| CTT residual base-context-object-layout | 3 | 207 | 0.2271 | 0.6425 | 0.0380 | 0.4340 | 0.2417 | proxy pass |
+| CTT residual base-context-observation-object-layout | 3 | 207 | 0.2077 | 0.6425 | 0.0285 | 0.4429 | 0.2448 | proxy pass |
 | CTT gated residual full | 3 | 207 | 0.2319 | 0.6135 | 0.0527 | 0.4337 | 0.1164 | proxy fail |
 
-Interpretation: residual CTT beats local-atlas on mean positive distance while staying within the one-point NegativeNear@0.20 slack; it does not beat local-atlas on PPTC. The one-seed base-context diagnostic slightly improves mean positive distance and PPTC@0.40 over the seed-0 base-action token but remains weaker than local-atlas PPTC. The three-seed base-context-observation diagnostic confirms the new RGB-stat path is wired and leakage-audited; it is the strongest CTT proxy row on PPTC@0.20/PPTC@0.40/mean positive distance while staying inside the safety slack, but still below local-atlas PPTC. Gated CTT improves mean positive distance but fails the safety slack in the three-seed aggregate. These are proxy support metrics, not OutcomePTR or measured success.
+Interpretation: residual CTT beats local-atlas on mean positive distance while staying within the one-point NegativeNear@0.20 slack; it does not beat local-atlas on PPTC. The one-seed base-context diagnostic slightly improves mean positive distance and PPTC@0.40 over the seed-0 base-action token but remains weaker than local-atlas PPTC. The three-seed base-context-observation diagnostic remains the strongest CTT proxy row on PPTC@0.20, while the object-layout-only row is slightly strongest on mean positive distance (`0.4340`) and stays within the safety slack. Gated CTT improves mean positive distance but fails the safety slack in the three-seed aggregate. These are proxy support metrics, not OutcomePTR or measured success.
 
 ## Measured Rollout Gate Table
 
@@ -318,6 +360,7 @@ Interpretation: residual CTT beats local-atlas on mean positive distance while s
 | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | --- |
 | validation | 3 | 207 | 0.2850 | 0.2415 | 0.3768 | 0.6667 | 0.4589 | 0.2947 | 0.1353 | fail: selected/oracle/support/selector |
 | validation + base-context-observation | 3 | 207 | 0.2754 | 0.2415 | 0.4058 | 0.6667 | 0.5024 | 0.2754 | 0.1643 | fail: selected/oracle/support/selector |
+| validation + base-context-object-layout | 3 | 207 | 0.2754 | 0.2029 | 0.3816 | 0.6667 | 0.5024 | 0.2947 | 0.1787 | fail: selected/oracle/support/selector |
 | test | 3 | 144 | 0.2847 | 0.2222 | 0.5069 | 0.7292 | 0.5278 | 0.2639 | 0.2847 | fail: selected/support/selector |
 | test + base-context-observation | 3 | 144 | 0.2917 | 0.2708 | 0.5139 | 0.7292 | 0.5347 | 0.2639 | 0.2431 | fail: selected/support/selector |
 | test + dominance auto | 3 | 144 | 0.2847 | 0.2500 | 0.5069 | 0.7292 | 0.5278 | 0.2639 | 0.2917 | fail: selected/support/selector |
@@ -334,7 +377,7 @@ Interpretation: residual CTT beats local-atlas on mean positive distance while s
 | test + learned dominance | 3 | 144 | 0.2847 | 0.3056 | 0.5069 | 0.7292 | 0.5278 | 0.2639 | 0.2569 | fail: selected/support/selector |
 | test + base-context-observation learned context dominance | 3 | 144 | 0.2917 | 0.3264 | 0.5139 | 0.7292 | 0.5347 | 0.2639 | 0.2431 | fail: selected/support/selector |
 
-Interpretation: the test proposal oracle clears the internal 50% support target, and visual-stat CTT plus validation-calibrated learned context dominance gives the best current held-out selected success (`0.3264`). The cleaner train-calibrated selector reaches `0.3125`, which still improves over base but does not beat the validation-calibrated diagnostic. Nonlinear train-calibrated selectors do not improve this number. All rows remain far below the selected-success gate and leave large selector gaps. Standalone train-only utility-energy checkpoints do not transfer well enough across three seeds. The next paper-critical work is richer visual/object-centric chart representation plus a stronger train-only utility/dominance model, not another proxy-only generator result.
+Interpretation: the test proposal oracle clears the internal 50% support target, and visual-stat CTT plus validation-calibrated learned context dominance gives the best current held-out selected success (`0.3264`). The cleaner train-calibrated selector reaches `0.3125`, which still improves over base but does not beat the validation-calibrated diagnostic. Nonlinear train-calibrated selectors do not improve this number. The object-layout proxy pass does not survive measured validation rollout and is worse than the RGB-stat row on proposal oracle/selected success. All rows remain far below the selected-success gate and leave large selector gaps. Standalone train-only utility-energy checkpoints do not transfer well enough across three seeds. The next paper-critical work is richer learned visual-language/object-centric chart representation plus a stronger train-only utility/dominance model, not another deterministic proxy descriptor.
 
 ## Leakage Audit Status
 
@@ -344,6 +387,7 @@ Interpretation: the test proposal oracle clears the internal 50% support target,
 - Cross-split chart/state overlap audit: pass.
 - Expert branches remain excluded from deployment candidates by index contract.
 - `data/cil_charts_rgb_refs/{train,val,test}`: same split/audience contract, RGB-ref leakage audit pass with 0 violations, and 100% observation-ref plus 100% RGB-stat embedding coverage in `runs/chart_feature_audit_rgb_refs`.
+- `data/cil_charts_rgb_refs/{train,val,test}` after object-layout export: same split/audience contract, leakage audit pass with 0 violations in `runs/leakage_audit_rgb_refs_object`, and 100% object-layout embedding coverage in `runs/chart_feature_audit_rgb_refs_object`.
 
 ## Data Accounting Status
 
@@ -354,6 +398,7 @@ Interpretation: the test proposal oracle clears the internal 50% support target,
 - Split hash remains `aaa857b92cbe5d455009f3ca3380e8b124908aa6c4c99fc1da408f738ed2dcf6`.
 - `runs/chart_feature_audit` documents the original `data/cil_charts` inputs: instruction and scene ids exist, but observation embeddings/raw refs do not.
 - `runs/chart_feature_audit_rgb_refs` documents the new non-destructive RGB-ref export: all splits have observation refs and deterministic 32D RGB-stat embeddings. This supports visual-stat diagnostics only; it does not yet support a learned visual-language/object-centric claim.
+- `runs/chart_feature_audit_rgb_refs_object` documents the deterministic 64D RGB object-layout export. The full proxy rows and measured validation rollout are complete; the measured result is negative relative to RGB-stat support.
 
 ## Paper Changes
 
@@ -371,6 +416,9 @@ Interpretation: the test proposal oracle clears the internal 50% support target,
 - Added chart feature source audit evidence and base-context proxy diagnostic language. The paper treats `base_context` as a coarse metadata diagnostic and separates the original chart DB from the new RGB-ref diagnostic chart DB.
 - Added RGB-ref/embedding audit evidence, the three-seed `base_context_obs` proxy row, measured validation/test rollout aggregates, and visual-stat dominance selector diagnostics. The draft describes this as a deterministic visual-stat support plus selector diagnostic that improves held-out selected success, but still not object-centric, learned visual-language, or enough for selected-action deployment evidence.
 - Added `cil/chart_features.py`, `scripts/audit_chart_feature_sources.py`, `scripts/export_chart_observation_embeddings.py`, and `scripts/slurm/train_ctt_feature_proxy.sbatch` to the reproducibility artifact list.
+- Added `scripts/export_chart_object_embeddings.py` to the reproducibility artifact list as a deterministic object-layout chart-token exporter.
+- Updated the validation proxy gate discussion to account for object-layout rows now present in `runs/ctt_val_proxy_comparison/table`: RGB-stat remains best on PPTC@0.20, object-layout-only is slightly best on mean positive distance, and measured rollout shows the object-layout row is negative relative to RGB-stat support.
+- Added the measured object-layout validation rollout table via `\input{../runs/ctt_base_context_obj_val_rollout_comparison/table}` and described it as a negative representation diagnostic.
 - Reframed the title/abstract around `Counterfactual Action Atlas`; CTT is now the current transport generator inside the Atlas, not the whole novelty.
 - Tightened the abstract to avoid overclaiming a successful learned generator: the paper now says the Atlas measures local do-action geometry and audits whether deployment-clean generators reach positive support.
 - Added the train-calibration leakage rule to the split contract: when target charts are from train, retrieval excludes same chart id and same restored-state hash.
@@ -384,8 +432,8 @@ Interpretation: the test proposal oracle clears the internal 50% support target,
 
 ## Next Actions
 
-1. Replace the current weak train-only utility/dominance model with richer visual-language/object-centric chart tokens; the best clean train-calibrated selector reaches only `0.3125` selected success.
-2. Move beyond deterministic RGB statistics toward learned visual-language/object-centric chart tokens; the best current visual-stat validation-calibrated row reaches `0.3264` selected success but remains far below the gate.
-3. Stop spending main effort on shallow selector wrappers; ridge and nonlinear selectors both fail to transfer. Train utility/dominance from richer outcome-vector and object-centric chart features.
+1. Stop treating deterministic object-layout as a likely win: it passed proxy but failed measured support relative to RGB-stat.
+2. Replace the deterministic RGB/object descriptors with learned visual-language or task/object tokens that can distinguish target, distractor, gripper/contact region, and task stage.
+3. Train utility/dominance on richer outcome-vector features; the best clean train-calibrated selector still reaches only `0.3125` selected success.
 4. Add unsafe execution rate and calibrated fallback rate to measured rollout artifacts.
-5. Rerun validation/test measured rollouts after the representation or utility model changes, not only proxy support sweeps.
+5. Rerun validation/test measured rollouts only after a representation or utility model change, not another shallow selector/proxy-only change.

workspace/reports/draft_audit_ctt.md

@@ -104,6 +104,24 @@ models, then evaluates once on held-out test rows. Its best row reaches only
 negative evidence: the current failure is not merely linear separability in the
 selector; the chart representation and outcome-vector features are still the
 paper-critical bottleneck.
+The current continuation adds a leakage-clean deterministic object-layout chart
+token path, but it is not yet promoted as a method result. The exporter
+`scripts/export_chart_object_embeddings.py` writes 64D RGB foreground
+component/layout embeddings to `data/cil_charts_rgb_refs/{train,val,test}`;
+`runs/chart_object_embeddings_rgb_refs` and
+`runs/chart_feature_audit_rgb_refs_object` verify 100% coverage and no outcome
+reads. Full three-seed proxy jobs for `base_context_obj` and
+`base_context_obs_obj` completed: object-layout-only slightly improves mean
+positive distance over the RGB-stat row (`0.4340` vs `0.4347`) while keeping
+NegativeNear@0.20 within the safety slack, but it does not beat RGB-stat on
+PPTC. Therefore measured validation rollout jobs `15116890`--`15116892` were
+submitted for `base_context_obj`; the main paper should add OutcomePTR rows
+only after those jobs finish. Those rollout jobs now finish as
+`runs/ctt_base_context_obj_val_rollout_comparison`: selected success = 0.2029,
+proposal-oracle success = 0.3816, and OutcomePTR@8 = 0.5024. This is negative
+relative to the RGB-stat validation row (selected 0.2415, proposal oracle
+0.4058), so deterministic foreground object-layout should remain a negative
+representation diagnostic, not a promoted method improvement.
 
 Current artifact evidence:
 
@@ -141,6 +159,14 @@ Current artifact evidence:
   `runs/ctt_base_context_obs_nonlinear_dominance_*_train_to_test` contain
   nonlinear train-calibrated selector diagnostics; the best row is 0.3056
   selected test success and is negative relative to the clean ridge selector.
+- `scripts/export_chart_object_embeddings.py`,
+  `runs/chart_object_embeddings_rgb_refs`, and
+  `runs/chart_feature_audit_rgb_refs_object` contain the new deterministic
+  RGB object-layout chart-token export and audit; Slurm arrays `15116833` and
+  `15116834` contain the completed full proxy jobs for the corresponding CTT
+  feature modes; `runs/ctt_base_context_obj_val_rollout_comparison` contains
+  the completed measured validation rollout for the best proxy row, which is
+  negative relative to RGB-stat support.
 - `runs/ctt_learned_dominance_val_to_test` contains the first learned
   validation-calibrated dominance selector diagnostic.
 - `runs/ctt_learned_dominance_{success,success_weighted,margin_ext,tangent}_val_to_test`