perdim trainmax CTT 2026-07-03 workspace/reports/2026-07-03.md

workspace/reports/2026-07-03.md

@@ -22,6 +22,10 @@
 - Added `--execution-action-scale` to `scripts/eval_ctt_generated_rollout.py` and `EXECUTION_ACTION_SCALE` to the Slurm wrapper. The scale is applied before action-bound diagnostics, optional clipping, and simulator execution, and is logged in the decoder plus each measured row.
 - Added `--execution-action-transform {identity,tanh}` and optional `--execution-action-scale-vector` to `scripts/eval_ctt_generated_rollout.py`, plus `EXECUTION_ACTION_TRANSFORM` and `EXECUTION_ACTION_SCALE_VECTOR` in the Slurm wrapper. The new `tanh` convention is a bounded decoder diagnostic: it maps decoded controls smoothly into finite env action bounds before action-bound diagnostics and rollout, and logs the transform/vector in the decoder and each measured row.
 - Added `scripts/slurm/eval_tanh_train_dominance.sbatch` to aggregate train-split bounded-tanh calibration rollouts and run train-calibrated dominance/learned-dominance selectors on held-out test tanh rows after the rollout jobs finish.
+- Added `scripts/build_action_scale_vector.py` to turn the train-only action-bound audit into a reproducible per-dimension execution scale artifact. The current artifact is `runs/action_scale_vector_train_base_branch_max`, fitted only on train `base_branch` max-abs bounds, with env string `0.381401261606,0.371117178171,0.354308917074,0.237850081835,0.248271283428,0.232648812617,0.215420039658`.
+- Updated `scripts/eval_ctt_generated_rollout.py` so a 7D execution scale vector can be used after ManiSkill action adaptation to a wider env action dimension; shorter vectors are padded with scale `1.0` on newly padded trailing action dimensions, and longer vectors are truncated to the adapted dimension.
+- Updated `scripts/slurm/eval_ctt_generated_rollout.sbatch` to support `CHECKPOINT_TEMPLATE` and `OUT_DIR_TEMPLATE` with `{seed}` replacement for clean Slurm array rollout submissions.
+- Added `scripts/slurm/eval_perdim_trainmax_dominance.sbatch` to aggregate per-dimension train-max train/validation/test rollout rows and evaluate train-clean dominance variants on held-out test once the rollout arrays complete.
 - Extended `scripts/eval_learned_dominance_selector.py` with logged `--feature-set` and `--target` variants, including success-weighted and tangent-code diagnostics, while keeping the original utility-margin default unchanged.
 - Extended `scripts/eval_learned_dominance_selector.py` again with deployment-visible `context` and `context_tangent` feature sets using target task id, instruction hash, source task id, and source-target task match. These features do not read evaluator-only val/test outcomes.
 - Extended `scripts/eval_dominance_selector.py` with `--score-source checkpoint`, cached chart embeddings/base scores, and single-thread torch scoring so a standalone train-only utility-energy checkpoint can be used for dominance evaluation.
@@ -126,6 +130,10 @@
 - Submitted dependent train-calibrated tanh dominance job:
   - `15120167`: `afterok:15120086:15120087:15120088`, runs `scripts/slurm/eval_tanh_train_dominance.sbatch`.
   - outputs, if the dependency succeeds: `runs/ctt_base_context_obs_train_cal_tanh_rollout_comparison`, `runs/ctt_base_context_obs_dominance_tanh_train_to_test`, and learned-dominance train-to-test tanh variants for `basic`, `context`, and `context_tangent` feature sets.
+- Submitted per-dimension train-max action-convention rollout arrays:
+  - first attempt `15130582`, `15130583`, `15130584`: train/validation/test arrays, seeds 0/1/2, `CLIP_ACTIONS=0`, identity transform, per-dimension train-only scale vector from `runs/action_scale_vector_train_base_branch_max`.
+  - first attempt exposed a 7D vector vs adapted env action-dimension mismatch; fixed in `scripts/eval_ctt_generated_rollout.py`, canceled remaining tasks, and removed incomplete run dirs.
+  - clean retry `15130732`, `15130733`, `15130734`: train/validation/test arrays, seeds 0/1/2, same convention. Current state at report update: pending due scheduler priority.
 
 ## Jobs Completed
 
@@ -436,6 +444,8 @@
 - First object-layout local smoke attempt failed with `libgomp: Thread creation failed: Resource temporarily unavailable` under default thread settings. Reran successfully with `OMP_NUM_THREADS=1`, `OPENBLAS_NUM_THREADS=1`, `MKL_NUM_THREADS=1`, and `NUMEXPR_NUM_THREADS=1`; the Slurm wrapper already sets single-thread CPU env vars.
 - Dependent Slurm job `15120167` for train-calibrated tanh dominance failed after successfully aggregating train-tanh rows because Bash parameter expansion mangled the literal `{seed}` placeholder in the default `CHECKPOINT_TEMPLATE`, producing Python `KeyError: 'seed/model'`.
 - Fix: rewrote `scripts/slurm/eval_tanh_train_dominance.sbatch` to assign the `{seed}` checkpoint template with a quoted conditional block, then reran the selector sweep locally with the same script after canceling queued retry `15120416`.
+- Per-dimension train-max first rollout attempt `15130582_0` failed with `ValueError: execution action scale vector length must match adapted action dimension`; sibling train/val/test array tasks were canceled before wasting cluster time.
+- Fix: updated `_scale_actions_4d` to pad shorter scale vectors with `1.0` after action adaptation, added regression test `test_execution_action_scale_vector_pads_after_adapted_dims`, and reran `tests/test_ctt.py` successfully before submitting clean retry arrays `15130732`--`15130734`.
 
 ## Best Metric Table
 
@@ -553,6 +563,11 @@ Action-bound audit status: new rollout artifacts now log action-bound safety lab
   - `.venv/bin/python -m pytest tests/test_ctt.py tests/test_metrics.py tests/test_cil_metrics.py tests/test_dominance_selector.py tests/test_chart_features.py -q`: 36 passed.
   - `latexmk -pdf -interaction=nonstopmode main.tex`: PDF rebuilt successfully; only existing overfull/underfull layout warnings remain.
   - Overclaim scan for submission-tier, leaderboard-style, and non-evidence roadmap phrasing in the main paper/report files returned no matches.
+- Verification after the per-dimension train-max action-convention patch:
+  - `.venv/bin/python -m py_compile scripts/eval_ctt_generated_rollout.py scripts/build_action_scale_vector.py`
+  - `.venv/bin/python -m pytest tests/test_ctt.py -q`: 11 passed.
+  - `bash -n scripts/slurm/eval_ctt_generated_rollout.sbatch`
+  - `bash -n scripts/slurm/eval_perdim_trainmax_dominance.sbatch`
 
 ## Hugging Face Sync Status
 
@@ -595,7 +610,7 @@ Action-bound audit status: new rollout artifacts now log action-bound safety lab
 1. Fix the action representation/decoder, not with a global scalar: scaled0.215 fixes base bounds but collapses generated support.
 2. Stop treating deterministic object-layout as a likely win: it passed proxy but failed measured support relative to RGB-stat.
 3. Replace deterministic RGB/object descriptors with learned visual-language or task/object tokens that distinguish target, distractor, gripper/contact region, and task stage.
-4. Improve bounded-action support/transport: train-clean tanh dominance reaches `0.3819`, matching the tanh proposal oracle, so the next bottleneck is proposal support under a valid action convention.
+4. Improve bounded-action support/transport: train-clean tanh dominance reaches `0.3819`, matching the tanh proposal oracle in micro success, so the next bottleneck is proposal support under a valid action convention. Per-dimension train-max scaled identity rollouts are now queued as the next support-preservation diagnostic.
 5. Train utility/dominance on richer outcome-vector features only after support improves; under bounded tanh, current train-clean context-success dominance matches proposal-oracle micro success but leaves a nonzero row-level selector gap.
 6. Rerun measured support/selection once under exactly one declared deployment action convention after the decoder/action representation fix; tanh is action-bound-clean but currently sacrifices too much proposal support.
 7. Add actual collision/contact labels after action-bound validity is resolved.