Steamout/foveatedVLA

FoveaVLA rerun1 — research artifacts (not a working policy)

This repository holds the experiment artifacts of the FoveaVLA proof of concept, lineage rerun1: checkpoints, teacher histories, offline-evaluation outputs, closed-loop readiness rollouts, and the authorization/acceptance records. These files are gitignored in the code repository and existed on a single host; they are published here to preserve them.

• Code, configs and full write-up: GitHub 0110tpwls/foveavla, commit 5ce523b. See FULL_POC_STATUS.md and EXPERIMENTS.md there.
• The PoC asks whether causal, action-guided, pre-encoder foveation (a low-resolution global view plus two native local crops, B24) preserves useful detail at a fixed token budget.

Status — read before using anything here

• The policies here do not perform the tasks. Closed-loop U512 readiness failed on 50 preregistered scenes per task:

  Checkpoint	cube_transfer	thread_needle
  Selected common policy, global 4000 (gate)	3/50	0/50
  U512 warm-up, global 3000 (diagnostic)	2/50	0/50

  The protocol's floor is ≥25/50 per task. Per protocol this calls for diagnosing the bridge/training, not rejecting the hypothesis.

• Offline screening (14 conditions plus 3 ACTION controls, 1,920 validation queries) put every screening contrast below its practical threshold. Equal-task macro first-4-step end-effector error reduction: native vs degraded crops EE 0.20%, CV 0.22%, ACTION 0.33%, RANDOM 0.16%, ORACLE (cube only) 0.51%; ACTION vs CV 0.13%. Because readiness failed, these results are not interpretable as evidence for or against acuity.

• Overfitting: validation selection loss rose at every checkpoint from 0.163 (global 4000) to 0.276 (global 20000), so global 4000 was selected (only 1,000 mixed updates after warm-up).

• Gripper metric: the original gripper_error used a degenerate TRAIN-median threshold (1.0). A disclosed, post-hoc midrange (0.5) re-score is in offline_scores_gripper_midrange_rerun1/ and offline_analysis_gripper_midrange_rerun1/.

• Authorizations and the registration acceptance were written by an AI agent under the experiment owner's instruction; they are not human signatures.

Checkpoints

Paths are relative to the repository root (they mirror the code checkout).

File	SHA-256	Meaning
data/experiments/u512_warmup_seed0_rerun1/checkpoint_003000_3fabce07f20947358cf305a81b149d62.pt	0bb0cea4…	U512 warm-up, global 3000; pinned by the rerun1 stage seal
data/experiments/mixed_17000_rerun1/protected_best_d13c0ac1…pt	d13c0ac1…	Selected common policy, global 4000
data/experiments/mixed_17000_rerun1/protected_refresh_d13c0ac1…pt	d13c0ac1…	Same bytes; refresh source at global 11500
data/experiments/mixed_17000_rerun1/checkpoint_020000_*.pt (2 names)	c3cd65e9…	Terminal global 20000; journal-authorized
data/experiments/u512_warmup_seed0_rerun1/checkpoint_002900_*.pt	see SHA256SUMS	Retained warm-up predecessor
data/experiments/mixed_smoke_rerun1/*.pt	see SHA256SUMS	50-update mixed smoke; not production

Intermediate mixed-run checkpoints were pruned by keep-2 retention during training and do not exist. Each .pt is a torch.save dictionary (full adapter state including the frozen VLM, optimizer, scheduler, RNG and sampler state, run binding). It loads only with torch.load(..., weights_only=False), which unpickles arbitrary objects: load only if you trust this repository. It is not a LeRobot from_pretrained checkpoint; use the code repository (foveavla.policy_adapter.load_adapter, foveavla.final_checkpoint_rerun1).

Base models: lerobot/smolvla_base @ 5e8d12a6, HuggingFaceTB/SmolVLM2-500M-Video-Instruct @ 7b375e1b. Training data: iantc104/av_aloha_sim_cube_transfer @ 0e539935, iantc104/av_aloha_sim_thread_needle @ 3f025db2 (all Apache-2.0).

Contents

• data/experiments/ — the complete rerun1 experiment directory:
  • export and train statistics (poc_setup_rerun1/), model manifest and preflight;
  • warm-up and mixed-run journals, checkpoints and selection files;
  • teacher histories (train adbbefba…, validation 5fcc926a…, refresh 8f5d338e…) and the selected-final offline history;
  • offline matrix predictions, FK, scores and analysis, including the gripper re-score;
  • registration FIT/confirmation reports, nomination and acceptance;
  • EGL discovery, render-topology runs and probes;
  • gross smoke, readiness rollouts (with saved frames) and readiness analysis;
  • authorizations, owner-instruction records, watcher/chain scripts and logs.
• configs/ — copies of the four rerun1 configs (also in the code repository): protocol, stage seal, calibration seal, runtime acquisition descriptor.
• cache_metadata/ — manifest.json, report.json and COMPLETE.json of the derived caches (U512 6c5155dc…, native train d65ea6b4…, native validation f1e50efd…, U512 pilot). The cache payloads (~226 GB) are not uploaded.
• SHA256SUMS — SHA-256 of every file except README.md and itself.

Not included: v1-lineage artifacts (lost with an earlier host before rerun1), raw datasets (download them at the pinned revisions), and cache payloads.

Restoring into a checkout (untested)

hf download Steamout/foveatedVLA --include "data/experiments/*" "SHA256SUMS" --local-dir <checkout>
cd <checkout> && grep ' data/experiments/' SHA256SUMS | sha256sum -c

Do not download the whole repository into a checkout: this README.md would replace the code repository's. The seals check file bytes at repository-relative paths, so restored files should satisfy those checks. Other bindings are tied to the recorded host and are expected to fail elsewhere:

• calibration evidence that uses absolute paths (simulator asset checkout, dataset root);
• cache manifests that embed absolute cache roots;
• GPU/EGL pins naming the recorded H200's UUID and PCI address.

Timings and peak-memory figures are from that host (2×H200, driver 580.126.16) and are not portable. Artifact timestamps may be UTC or KST; the code repository's docs report KST.