OpenWAM-Stanford commited on
Commit
8d28e78
·
verified ·
1 Parent(s): cf4c68f

Describe the selected release candidate, usage, and validation limits

Browse files
Files changed (1) hide show
  1. README.md +111 -0
README.md ADDED
@@ -0,0 +1,111 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ tags:
4
+ - robotics
5
+ - libero
6
+ - video-to-action
7
+ - openwam
8
+ pipeline_tag: robotics
9
+ ---
10
+
11
+ # OpenWAM VTA LIBERO-10
12
+
13
+ A dual-expert video-then-action policy for LIBERO-10. This release contains
14
+ the complete video and action experts from the selected step-15,000 checkpoint.
15
+ It is not an inverse-dynamics-only model or a video-backbone-only export.
16
+
17
+ The policy first predicts future video latents from the task prompt and observed
18
+ scene, then predicts robot actions conditioned on those latents. During
19
+ closed-loop execution, new real observations update its recurrent history.
20
+ This is a single trained VTA policy, not a composition with an external IDM.
21
+
22
+ **Status:** private release candidate at
23
+ [`OpenWAM-Stanford/OpenWAM-VTA-LIBERO10`](https://huggingface.co/OpenWAM-Stanford/OpenWAM-VTA-LIBERO10).
24
+ The selected checkpoint is step **15,000**, replacing the earlier proposed
25
+ VTA release. The inference weights contain approximately 7.67 billion FP32
26
+ parameter elements (about 30.7 GB); this is not an inference-memory estimate.
27
+
28
+ Project: https://openwam.github.io/
29
+
30
+ Code: https://github.com/OpenWAM/OpenWAM
31
+
32
+ ## Inference Contract
33
+
34
+ - Program: `video_then_action`, with `video_only` recurrent history visibility.
35
+ - Two 128x128 RGB camera views, concatenated into a 128x256 frontend input.
36
+ - Four predicted latent frames per chunk and sixteen raw 7D action commands.
37
+ - Attention window: 30; 20 video and 20 action denoising steps.
38
+ - Video guidance scale: 5; conditioned action generation with scale 1.
39
+ - Per-chunk proprioception, single-frame condition latents, and the
40
+ `legacy_prefix_single_frame_perchunk_proprio` sequence contract.
41
+
42
+ Use `resolved_config.yaml` with `model_state.pt` through the maintained
43
+ OpenWAM checkpoint loading path. Configure the external LingBot-VA VAE,
44
+ text encoder, and tokenizer assets using `paths.models.lingbot_va_base`.
45
+ Their source revision is recorded in `release_metadata.json`. The full
46
+ checkpoint supplies the policy weights; no private initialization checkpoint
47
+ is needed for inference.
48
+
49
+ ## Download And Run
50
+
51
+ Authenticate with an account that can read this private repository, then download
52
+ the inference bundle:
53
+
54
+ ```bash
55
+ hf download OpenWAM-Stanford/OpenWAM-VTA-LIBERO10 --local-dir vta-libero10
56
+ cd vta-libero10
57
+ sha256sum --check SHA256SUMS
58
+ ```
59
+
60
+ Install OpenWAM and its LIBERO simulator dependencies following the code
61
+ repository instructions. Use the code revision recorded in
62
+ `release_metadata.json`, and set `paths.models.lingbot_va_base` in
63
+ `configs/local_paths.yaml` to the matching frontend assets. Run from that
64
+ checkout with `CKPT` set to the downloaded bundle's absolute path:
65
+
66
+ ```bash
67
+ MUJOCO_GL=osmesa python scripts/run_libero_policy.py \
68
+ --cfg "$CKPT/resolved_config.yaml" --checkpoint "$CKPT/model_state.pt" \
69
+ --benchmark libero_10 --task-id 0 --episode-idx 0 --seed 0 \
70
+ --frontend-encode-mode lingbot_streaming_vae \
71
+ --inference-window-size 30 --execute-frame-chunk-size 4 \
72
+ --startup-model-obs-frames 1 --startup-env-init-steps 5 \
73
+ --max-timestep 800 --max-chunks 50 \
74
+ --runtime-device cuda:0 --action-device cuda:0 \
75
+ --frontend-device cuda:0 --decode-device cuda:0 \
76
+ --save-rollout-video --output-dir outputs/vta-libero10
77
+ ```
78
+
79
+ This example uses the maintained CLI and preserves the checkpoint's inference
80
+ contract. A fresh GPU rollout of this packaged checkpoint remains to be run;
81
+ the example is not a new benchmark result. Do not use partial-checkpoint loading
82
+ or substitute another checkpoint's inference configuration.
83
+
84
+ ## Contents And Evidence
85
+
86
+ | File | Contents |
87
+ | --- | --- |
88
+ | `model_state.pt` | Tensor-only policy checkpoint, including both experts |
89
+ | `resolved_config.yaml` | Portable architecture and inference settings |
90
+ | `release_metadata.json` | Code/assets revisions, tensor counts, source/export hashes |
91
+ | `SHA256SUMS` | Integrity checks for all release files |
92
+ | `README.md` | This model card |
93
+
94
+ The bundle contains tensor-only policy weights, a portable inference config,
95
+ checksums, and release metadata. It does not contain optimizer state, training
96
+ logs, or private dataset paths. Every active policy tensor retains its exact
97
+ original values and dtype. The export omits only the two tensors belonging to
98
+ the retired post-decoded feature projection (`visual_tower.decoder.proj`),
99
+ which is absent from the maintained VTA inference architecture. Neither the
100
+ VAE nor either expert is removed. Source and exported file hashes are recorded
101
+ separately; the exported file is not byte-identical to the training archive.
102
+
103
+ No benchmark success rate is assigned to this checkpoint by this release.
104
+ Results measured with other VTA checkpoints or other inference overrides must
105
+ not be attributed to these weights. Architecture/configuration checks establish
106
+ loadability, not closed-loop policy performance.
107
+
108
+ Weight-license confirmation and a rollout smoke test remain release gates.
109
+ The policy was developed for simulated LIBERO manipulation; no real-robot
110
+ safety or generalization guarantee is made. Private availability does not
111
+ constitute a redistribution license or approval for deployment.