# For the Claude on the target workstation — updating your inference code

`written: 2026-08-11` · applies to the bundle in
`hf.co/AllenBi21/tactile-act-screw-baselines`

This repo contains the **reference inference stack** the `screw_t3_50k` /
`screw_sitr5_50k` checkpoints were validated with: `inference.py`,
`act_policy/`, `data/dataset.py`, `tools/live_features.py`. Your local
inference code is known to have drifted slightly. **Update yours against this
reference — do not wholesale-replace it** (your local robot-IO, rate control,
logging, and device plumbing should survive), with one exception noted in §3.

## 1. Procedure

1. Download: `hf download AllenBi21/tactile-act-screw-baselines --local-dir ref`
2. Diff `ref/inference.py` and `ref/act_policy/` against your local versions.
3. Port every difference that touches the load-bearing contract (§2) from the
   reference into your code. Keep your local plumbing everywhere else.
4. Run the verification in §4 before any robot use. Do not skip it.

## 2. Load-bearing contract (must match the reference exactly)

1. **Checkpoint loading**: the .pt is a dict
   `{"step", "model", "config", "optimizer", ...}`; build `ACTConfig` from
   `config`, `load_state_dict(ckpt["model"])` must report
   **missing=[] unexpected=[]**. qpos/tactile/action normalization lives in
   buffers inside the state_dict — never re-normalize inputs externally.
2. **Observation contract**: `{"state": (22,) float, "tactile": (3840,) float}`.
   The tactile vector is 5 fingers × 768, **concatenated in finger order
   0..4**, produced by `tools/live_features.py` (its preprocessing constants
   are training-fixed — do not "improve" or recompute them).
3. **Chunking**: `chunk_size=30, n_action_steps=30` come from the checkpoint
   config; `step()` pops from the action queue and re-queries every 30 frames;
   `reset()` must be called at every episode start.
4. **Output semantics**: `step()`/`predict_chunk()` return **absolute target
   joint positions** (22,), even for delta-trained checkpoints (integration
   happens inside `policy.predict_action_chunk`).
5. `state_history = tactile_history = 1` for these checkpoints.

## 3. When to wholesale-replace instead

If your local `act_policy/` predates the May refactor (no `forward_context`
split, no `FlowMatchingHead`, no generalized tactile-key routing), take the
bundled `act_policy/` as-is — partial merges of the model code are how silent
weight-mismatch bugs happen. The gate is §2.1: if `load_state_dict` is not
clean, replace the package.

## 4. Verification (in order, all must pass)

1. **Feature parity** (needs any episode HDF5 that contains stored `/t3/<f>`
   and `/sitr/<f>` datasets):
   ```bash
   python -m tools.live_features --kind t3   --episode <episode.hdf5>
   python -m tools.live_features --kind sitr --episode <episode.hdf5>
   # must print PASS (worst cosine > 0.999)
   ```
2. **Clean checkpoint load**: load both act_best.pt; assert no missing /
   unexpected keys, and `config.tactile_dim == 3840`.
3. **Offline eval sanity** (if the episode data is present):
   ```bash
   python inference.py --checkpoint <...>/screw_t3_50k/act_best.pt \
     --mode offline --data episodes/screwing \
     --tactile-keys t3_0 t3_1 t3_2 t3_3 t3_4 --out-csv /tmp/t3.csv
   ```
   Expect overall L1 ≈ 0.023 / val-episode L1 ≈ 0.032 (T3); ≈ 0.020 / 0.031
   (SITR). A large deviation means a contract violation — stop and compare
   against the reference again.
4. Only after 1–3 pass: dry-run the robot loop with `step()` on live data.

## 5. Environment notes

- `tools/live_features.py` needs the third-party encoder stacks:
  tactile_fusion repo (`TACTILE_FUSION_ROOT` env; loads T3 via its
  `load_baselines`) + t3 repo + t3_medium weights; gsrl repo (`GSRL_ROOT`) +
  `checkpoints/SITR_B18.pth`. These are NOT in this bundle.
- Known import trap: tactile_act's `data/` package shadows tactile_fusion's;
  `live_features.py` handles it (`_strip_shadowing_paths`) — keep that if you
  port the file.
- torch ≥ 2.x; GPU strongly recommended for the SITR encoder (86M) at 30 Hz.