Update README.md

f03d0f1 verified 18 days ago

9.05 kB

---
library_name: lerobot
tags: [robotics, act, robosynchallenge, items_handover]
pipeline_tag: robotics
---
# ACT_sim_items_handover
ACT policy for the RoboSynChallenge simulation task `items_handover`.
This staging root is a complete LeRobot `pretrained_model` directory. Proposed team HF repository: `mair-robosyn/ACT_sim_items_handover_official`; it has not been created/uploaded by this delivery. The exact saved optimizer update count is not independently recorded; inherited train_config specifies 80000 steps.
## Files
| File | Size (bytes) | sha256 |
|---|---:|---|
| `config.json` | 2037 | `8d255efa3335bb6b0c964ff1d0cbc7ff02183fdb9708320cfa38653557dc9258` |
| `model.safetensors` | 206563032 | `ed95576e59662af5c9a7a49af09822795b44b14251566f8cdb4955a0107d8071` |
| `train_config.json` | 5476 | `ba0c6f941cd72e20207196273fd1384b38fa27e4f39e11b550adc5179750e8d4` |
| SmolVLA preprocessor / postprocessor | N/A (ACT) | N/A |

`README.md` is the model card. No dataset, training logs, download locks, credentials, videos or training state are included. Weights and train_config are byte-identical to the official source. Per team SOP, the staging config differs only in `pretrained_backbone_weights`: `ResNet18_Weights.IMAGENET1K_V1` → null. This avoids an unnecessary ImageNet download; all backbone tensors are already in the weights.
## Model
- Architecture: ACTPolicy; ResNet18 vision backbone; transformer dim_model=512, 8 heads, feedforward=3200, 4 encoder layers, 1 decoder layer; VAE latent_dim=32, 4 VAE encoder layers, KL weight=10.
- Initialization/source: `RoboSynChallenge/ACT_sim_items_handover` @ `86f6ca7f5decd17a92c3218f295de53497383d14`. No additional training was performed on this selected fallback by the team.
- Inherited official training configuration: AdamW lr=1e-5, lr_backbone=1e-5, weight_decay=1e-4, batch_size=32, configured steps=80000, seed=1000, AMP=false. Effective accumulated/distributed batch and original completed update count: not recorded. The job name containing “bs64” is not evidence of effective batch 64.
- Data: `RoboSynChallenge/cobotmagic_Sim_items_handover`; verified local dataset revision `20a88bf0bc3fe3a1ae58ad97ed60a06c3431684a`, LeRobot v2.1, 1000 episodes / 327000 frames / 25 Hz. Original model's train_config leaves dataset revision null; the inspected dataset revision cannot be retroactively asserted as the official model's training revision. Data not included.
- W&B: inherited project `robosynchallenge`, run_id `em6zd573`, entity null; exact URL not recorded.
- Code: https://github.com/ruali-dev/RoboSynChallenge-Team ; inspected checkout commit `7122266b88321e3342a3df416c43c9be1c0bdcdc`; EmbodiChain `9ebee30011f378f94a7cbe78b01d8c2eacba231a`. These are checkout observations, not a reconstructed original training commit. Actual ACT deploy helper SHA256 `7aeae7b399e5de3cc57496bace803a1b618197ae818dc6ee0ac7a971b4b054ca`; original training code commit not recorded. No official source was modified for packaging.
- Environment: original official training environment not recorded. Today's evaluation: Python 3.10.12, PyTorch 2.7.0+cu128, CUDA runtime 12.8, NVIDIA GeForce RTX 5090. Staging CPU check: LeRobot 0.3.3, PyTorch 2.7.0+cu128 in the existing A100-host training Python environment, with CUDA_VISIBLE_DEVICES empty; no GPU/simulator launched.

## Input and output

- Runtime state input: `observation.state`, float32 `[B,14]`. Left arm 6 joints + gripper, then right arm 6 joints + gripper; gripper indices 6 and 13. Joint positions use the dataset/robot convention; gripper values must retain that convention rather than being treated as revolute angles.
- Runtime RGB inputs: `observation.images.cam_high`, `observation.images.cam_right_wrist`, `observation.images.cam_left_wrist`, each float32 `[B,3,480,640]`. Raw frames are uint8 HWC `[480,640,3]`, converted to CHW and divided by 255 before the policy's stored normalization. Cameras are mapped by key, not positional numbering.
- `config.json` also declares `observation.qvel` and `observation.qf` `[14]`; these are not passed by the verified ACT deployment path. They are not additional required policy inputs for that path. Privileged object poses, button state, success flags and future observations are excluded.
- Action: float32 `[B,50,14]` from `predict_action_chunk`, or `[B,14]` per `select_action`; bimanual joint/gripper targets, not Cartesian deltas. Preserve the left/right and gripper ordering.
- Normalization: VISUAL, STATE and ACTION use MEAN_STD; normalization tensors are embedded in `model.safetensors`. No external normalization file is required. Do not rebuild or reset statistics during deployment.
- n_obs_steps=1, chunk_size=50, n_action_steps=50, temporal_ensemble_coeff=null; training data 25 Hz. The internal tests used H50. Competition execution_chunk requirements must be checked separately; no deployment configuration was changed for this card.
## Procedure: download and evaluate
Confirmed upstream download (the new team repository has no revision yet):
```bash
hf download RoboSynChallenge/ACT_sim_items_handover --repo-type model --revision 86f6ca7f5decd17a92c3218f295de53497383d14 --local-dir checkpoints/ACT_sim_items_handover_official/
```
For offline ACT loading, apply the SOP's sole config-field change to that downloaded copy before evaluation; the provided staging already includes it. Load using `PreTrainedConfig.from_pretrained`, then `ACTPolicy`, rather than parsing config with `ACTConfig.from_pretrained`.
Exact command used for today's local-source 30-episode evaluation, inside `ssh robosyn` → `robosyn-workspace`:
```bash
cd /root/workspace/RoboSynChallenge
export PYTHON_BIN=/root/workspace/shorttrain-20260924-1417/envs/act/bin/python
bash policy/act/eval.sh items_handover random /root/workspace/official-finetune-20260924-2150/models/ACT_sim_items_handover 0 --max_episodes 30 --headless True --eval_video_log False --seed 240926 --max_steps 500 --model_name handover-act-official-30ep
```
The actual recorded timeout is 350 actions, because the task gym configuration caps this run despite the CLI request for 500. After manual team upload, pin its actual HF revision and run download/load plus official `policy/act/eval.sh` verification on that fresh download before calling the delivery EVAL VERIFIED. That post-upload verification has not been performed.
## Results (team-internal, not organizer results)
- Protocol: `items_handover`, random, official task configuration and success predicate; 30 accepted episodes, seed generator 240926, actual timeout 350 actions, host `ssh robosyn`, RTX 5090. Expert feasibility filtering accepted 30 of 52 candidate seeds and skipped 22; it did not supply corrective policy actions. This filtered protocol is not an unfiltered random-seed population estimate.
- Success: **1/30 = 3.33%**, mean action steps 349.2333. The ACT 2000-step fine-tune scored **0/30** on the same accepted seeds and protocol; that is insufficient evidence of meaningful superiority, but offers no reason to replace the original fallback.
- Accepted seeds: 1437328931, 512975484, 728356423, 290298465, 297345191, 1808607130, 1364265482, 2092946110, 1713388067, 917421052, 1416049871, 282520247, 783217757, 162260320, 938907990, 790779347, 450785757, 1624254241, 436607482, 1797006081, 601867212, 188790587, 449922006, 283940036, 521500024, 1584445822, 2072979752, 225898799, 1433950075, 461806314.
- Evidence: `/root/workspace/RoboSynChallenge/eval_result/items_handover/act/random/None/handover-act-official-30ep/2026-10-11_10-13-26/evaluation_metrics.json`, SHA256 `f8534231f4952354bc80d27bc0603192419829617a33ddf3d09de4071092ad98`.
- SmolVLA 500-step artifact exists, but only pipeline/dummy validation is recorded; task success rate is **未评测**. Issue #19 was closed as deferred (`not_planned`), not as delivery completion. No confirmed teammate SmolVLA handover delivery was found in the reviewed artifacts.
## Status
- UPLOADED: **no** for this proposed team delivery; HF revision N/A. The upstream official source exists at the revision above.
- LOAD VERIFIED: **no** in the SOP's fresh-HF-download sense. Local staging CPU strict loading and finite dummy outputs `[1,50,14]` / `[1,14]` passed on 2026-10-11; all loaded weight/normalization tensors exactly matched the source.
- EVAL VERIFIED (team-internal): **no** for this new team HF delivery. Today's source-checkpoint official-script evaluation is recorded separately as 1/30; it is not a post-upload pinned-revision verification.
- Package is ready for manual upload as a fallback candidate; the SOP's final handoff requirement remains incomplete until post-upload verification. No competition website submission or receipt is implied.
## Warning
These results come from our local runs. The organizers did not verify them.
The candidate has low measured task success. It is a fallback recommendation among the audited candidates, not proof of robust handover performance. No official/reference SR is substituted for the measured result.

	---
	library_name: lerobot
	tags: [robotics, act, robosynchallenge, items_handover]
	pipeline_tag: robotics
	---

	# ACT_sim_items_handover

	ACT policy for the RoboSynChallenge simulation task `items_handover`.
	This staging root is a complete LeRobot `pretrained_model` directory. Proposed team HF repository: `mair-robosyn/ACT_sim_items_handover_official`; it has not been created/uploaded by this delivery. The exact saved optimizer update count is not independently recorded; inherited train_config specifies 80000 steps.

	## Files

	\| File \| Size (bytes) \| sha256 \|
	\|---\|---:\|---\|
	\| `config.json` \| 2037 \| `8d255efa3335bb6b0c964ff1d0cbc7ff02183fdb9708320cfa38653557dc9258` \|
	\| `model.safetensors` \| 206563032 \| `ed95576e59662af5c9a7a49af09822795b44b14251566f8cdb4955a0107d8071` \|
	\| `train_config.json` \| 5476 \| `ba0c6f941cd72e20207196273fd1384b38fa27e4f39e11b550adc5179750e8d4` \|
	\| SmolVLA preprocessor / postprocessor \| N/A (ACT) \| N/A \|

	`README.md` is the model card. No dataset, training logs, download locks, credentials, videos or training state are included. Weights and train_config are byte-identical to the official source. Per team SOP, the staging config differs only in `pretrained_backbone_weights`: `ResNet18_Weights.IMAGENET1K_V1` → null. This avoids an unnecessary ImageNet download; all backbone tensors are already in the weights.

	## Model

	- Architecture: ACTPolicy; ResNet18 vision backbone; transformer dim_model=512, 8 heads, feedforward=3200, 4 encoder layers, 1 decoder layer; VAE latent_dim=32, 4 VAE encoder layers, KL weight=10.
	- Initialization/source: `RoboSynChallenge/ACT_sim_items_handover` @ `86f6ca7f5decd17a92c3218f295de53497383d14`. No additional training was performed on this selected fallback by the team.
	- Inherited official training configuration: AdamW lr=1e-5, lr_backbone=1e-5, weight_decay=1e-4, batch_size=32, configured steps=80000, seed=1000, AMP=false. Effective accumulated/distributed batch and original completed update count: not recorded. The job name containing “bs64” is not evidence of effective batch 64.
	- Data: `RoboSynChallenge/cobotmagic_Sim_items_handover`; verified local dataset revision `20a88bf0bc3fe3a1ae58ad97ed60a06c3431684a`, LeRobot v2.1, 1000 episodes / 327000 frames / 25 Hz. Original model's train_config leaves dataset revision null; the inspected dataset revision cannot be retroactively asserted as the official model's training revision. Data not included.
	- W&B: inherited project `robosynchallenge`, run_id `em6zd573`, entity null; exact URL not recorded.
	- Code: https://github.com/ruali-dev/RoboSynChallenge-Team ; inspected checkout commit `7122266b88321e3342a3df416c43c9be1c0bdcdc`; EmbodiChain `9ebee30011f378f94a7cbe78b01d8c2eacba231a`. These are checkout observations, not a reconstructed original training commit. Actual ACT deploy helper SHA256 `7aeae7b399e5de3cc57496bace803a1b618197ae818dc6ee0ac7a971b4b054ca`; original training code commit not recorded. No official source was modified for packaging.
	- Environment: original official training environment not recorded. Today's evaluation: Python 3.10.12, PyTorch 2.7.0+cu128, CUDA runtime 12.8, NVIDIA GeForce RTX 5090. Staging CPU check: LeRobot 0.3.3, PyTorch 2.7.0+cu128 in the existing A100-host training Python environment, with CUDA_VISIBLE_DEVICES empty; no GPU/simulator launched.

	## Input and output

	- Runtime state input: `observation.state`, float32 `[B,14]`. Left arm 6 joints + gripper, then right arm 6 joints + gripper; gripper indices 6 and 13. Joint positions use the dataset/robot convention; gripper values must retain that convention rather than being treated as revolute angles.
	- Runtime RGB inputs: `observation.images.cam_high`, `observation.images.cam_right_wrist`, `observation.images.cam_left_wrist`, each float32 `[B,3,480,640]`. Raw frames are uint8 HWC `[480,640,3]`, converted to CHW and divided by 255 before the policy's stored normalization. Cameras are mapped by key, not positional numbering.
	- `config.json` also declares `observation.qvel` and `observation.qf` `[14]`; these are not passed by the verified ACT deployment path. They are not additional required policy inputs for that path. Privileged object poses, button state, success flags and future observations are excluded.
	- Action: float32 `[B,50,14]` from `predict_action_chunk`, or `[B,14]` per `select_action`; bimanual joint/gripper targets, not Cartesian deltas. Preserve the left/right and gripper ordering.
	- Normalization: VISUAL, STATE and ACTION use MEAN_STD; normalization tensors are embedded in `model.safetensors`. No external normalization file is required. Do not rebuild or reset statistics during deployment.
	- n_obs_steps=1, chunk_size=50, n_action_steps=50, temporal_ensemble_coeff=null; training data 25 Hz. The internal tests used H50. Competition execution_chunk requirements must be checked separately; no deployment configuration was changed for this card.

	## Procedure: download and evaluate

	Confirmed upstream download (the new team repository has no revision yet):

	```bash
	hf download RoboSynChallenge/ACT_sim_items_handover --repo-type model --revision 86f6ca7f5decd17a92c3218f295de53497383d14 --local-dir checkpoints/ACT_sim_items_handover_official/
	```

	For offline ACT loading, apply the SOP's sole config-field change to that downloaded copy before evaluation; the provided staging already includes it. Load using `PreTrainedConfig.from_pretrained`, then `ACTPolicy`, rather than parsing config with `ACTConfig.from_pretrained`.

	Exact command used for today's local-source 30-episode evaluation, inside `ssh robosyn` → `robosyn-workspace`:

	```bash
	cd /root/workspace/RoboSynChallenge
	export PYTHON_BIN=/root/workspace/shorttrain-20260924-1417/envs/act/bin/python
	bash policy/act/eval.sh items_handover random /root/workspace/official-finetune-20260924-2150/models/ACT_sim_items_handover 0 --max_episodes 30 --headless True --eval_video_log False --seed 240926 --max_steps 500 --model_name handover-act-official-30ep
	```

	The actual recorded timeout is 350 actions, because the task gym configuration caps this run despite the CLI request for 500. After manual team upload, pin its actual HF revision and run download/load plus official `policy/act/eval.sh` verification on that fresh download before calling the delivery EVAL VERIFIED. That post-upload verification has not been performed.

	## Results (team-internal, not organizer results)

	- Protocol: `items_handover`, random, official task configuration and success predicate; 30 accepted episodes, seed generator 240926, actual timeout 350 actions, host `ssh robosyn`, RTX 5090. Expert feasibility filtering accepted 30 of 52 candidate seeds and skipped 22; it did not supply corrective policy actions. This filtered protocol is not an unfiltered random-seed population estimate.
	- Success: 1/30 = 3.33%, mean action steps 349.2333. The ACT 2000-step fine-tune scored 0/30 on the same accepted seeds and protocol; that is insufficient evidence of meaningful superiority, but offers no reason to replace the original fallback.
	- Accepted seeds: 1437328931, 512975484, 728356423, 290298465, 297345191, 1808607130, 1364265482, 2092946110, 1713388067, 917421052, 1416049871, 282520247, 783217757, 162260320, 938907990, 790779347, 450785757, 1624254241, 436607482, 1797006081, 601867212, 188790587, 449922006, 283940036, 521500024, 1584445822, 2072979752, 225898799, 1433950075, 461806314.
	- Evidence: `/root/workspace/RoboSynChallenge/eval_result/items_handover/act/random/None/handover-act-official-30ep/2026-10-11_10-13-26/evaluation_metrics.json`, SHA256 `f8534231f4952354bc80d27bc0603192419829617a33ddf3d09de4071092ad98`.
	- SmolVLA 500-step artifact exists, but only pipeline/dummy validation is recorded; task success rate is 未评测. Issue #19 was closed as deferred (`not_planned`), not as delivery completion. No confirmed teammate SmolVLA handover delivery was found in the reviewed artifacts.

	## Status

	- UPLOADED: no for this proposed team delivery; HF revision N/A. The upstream official source exists at the revision above.
	- LOAD VERIFIED: no in the SOP's fresh-HF-download sense. Local staging CPU strict loading and finite dummy outputs `[1,50,14]` / `[1,14]` passed on 2026-10-11; all loaded weight/normalization tensors exactly matched the source.
	- EVAL VERIFIED (team-internal): no for this new team HF delivery. Today's source-checkpoint official-script evaluation is recorded separately as 1/30; it is not a post-upload pinned-revision verification.
	- Package is ready for manual upload as a fallback candidate; the SOP's final handoff requirement remains incomplete until post-upload verification. No competition website submission or receipt is implied.

	## Warning

	These results come from our local runs. The organizers did not verify them.
	The candidate has low measured task success. It is a fallback recommendation among the audited candidates, not proof of robust handover performance. No official/reference SR is substituted for the measured result.