RainyBot's picture
Publish validated final step 2895
e03488e verified
|
Raw History Blame Contribute Delete
7.42 kB
---
library_name: pytorch
license: mit
language:
- en
tags:
- robotics
- robot-learning
- world-model
- vision-language-action
- semwam
- real-robot
model_name: SemWAM Real Robot — Transfer Dropper v1
---
# SemWAM Real Robot — Transfer Dropper v1
A task-specific SemWAM checkpoint for **Transfer Dropper**, released by RainyBot. SemWAM jointly predicts robot actions and future RGB, depth and semantic representations from current camera observations, a language instruction and robot state.
This is the **final checkpoint of the completed training run**, at optimizer step **2895**. All eight original node launchers exited successfully, and the saved optimizer step equals the archived training budget. Release verification was performed on 2026-11-10 (UTC).
## Intended use and evidence
The release supports research on multimodal world–action modeling and integration with a compatible real-robot interface. Checkpoint structure, tensor finiteness, saved inference identity, numerical normalization statistics and save-completion inventory were verified. Training completion does not establish a real-robot success rate: this release makes no claim of closed-loop performance or state-of-the-art results.
## Files
| File | Purpose |
| --- | --- |
| `checkpoints/weights/step_002895.pt` | Final SemWAM v2 checkpoint; includes the saved model contract and frozen semantic compressor |
| `inference_config.json` | Machine-independent model and processor contract |
| `config.yaml` | Public inference configuration; restore operational asset paths on your machine |
| `dataset_stats.json` | Numerical action/state normalization statistics |
| `checkpoints/weights/dataset_stats.json` | Identical statistics companion for automatic checkpoint discovery |
| `checkpoint_manifest.json` | Checkpoint SHA256, byte size, step, public identity and required asset checksums |
| `model_card_summary.json` | Readable architecture and interface summary |
| `trainer_state.reference.json` | Minimal public step/contract reference; no optimizer or RNG state |
| `SHA256SUMS` | SHA256 checksums for release files |
Weights retain the canonical `step_XXXXXX.pt` filename expected by SemWAM tooling. Task identity and release version are expressed by the repository name. Intermediate checkpoints, optimizer partitions, RNG state and training data are excluded.
## Download
```python
from pathlib import Path
from huggingface_hub import snapshot_download
# For reproducible experiments, set revision to a verified 40-character commit
# from this model repository's commit history.
release_dir = Path(snapshot_download(
repo_id="RainyBot/SemWAM-RealRobot-TransferDropper-v1",
revision="main",
))
checkpoint = release_dir / "checkpoints/weights/step_002895.pt"
statistics = release_dir / "dataset_stats.json"
```
```bash
cd /path/to/downloaded/release
sha256sum --check SHA256SUMS
```
## Source and loading
Use the SemWAM code at `https://github.com/RainyRobo/SemWAM`. Packaging checkout: `19d1247e24e1990a7aac99fbd5cddd674771df9a`. The packaging checkout contained local changes; this commit alone is not a claim that it exactly reproduces the training source. Consult the source repository's setup and evaluation documentation.
From a configured SemWAM checkout, inspect the public v2 identity before constructing the model:
```python
import torch
from omegaconf import OmegaConf
from wam.eval.checkpoint_schema import validate_checkpoint
from wam.eval.checkpoint_config import apply_checkpoint_to_cfg
payload = torch.load(checkpoint, map_location="cpu", mmap=True, weights_only=True)
validate_checkpoint(payload)
cfg = OmegaConf.load(release_dir / "config.yaml")
apply_checkpoint_to_cfg(
cfg, str(checkpoint), payload=payload,
restore_inference=True, prepare_native=False,
)
# Bind local paths for the exact frozen assets, then construct the model
# through the SemWAM runtime and load the validated checkpoint modules.
```
Check frozen dependencies in the configured source environment:
```bash
PYTHONPATH=src python -m wam_setup check --role eval \
--checkpoint /path/to/downloaded/release/checkpoint_manifest.json
```
The model is a SemWAM v2 checkpoint, not a Transformers `AutoModel` package. This release does not provide a turnkey robot driver. The RoboTwin joint-action adapter is not interchangeable with this bimanual end-effector pose interface.
## Observation, action and normalization interface
- Three RGB views: primary camera, left wrist and right wrist.
- Raw robot state and returned actions each have 16 values: `[left xyz, left quaternion xyzw, left gripper, right xyz, right quaternion xyzw, right gripper]`.
- The model uses 20-dimensional position/rotation-6D/gripper representations. State is absolute; actions are relative SE(3) transforms from the current state, with absolute gripper values.
- The saved action horizon is 32. Postprocessing must preserve the full horizon, undo the packaged affine normalization, then restore absolute poses with the saved processor.
- Fresh normalization is unclipped z-score. Reusing another task's statistics, applying ±5 clipping, swapping quaternion conventions or adding an action-index shift changes the released interface.
- Depth conditioning is estimated from **current RGB only**. Future teacher targets are not inputs at inference. Preserve the saved depth preprocessing and camera geometry in `inference_config.json`.
## Saved architecture
```json
{
"model.arch": "rgb_dep_sem",
"model.attention_layout": "rgb_auxiliary_causal",
"model.action_conditioning": "predicted_depth_head_semantic",
"model.world_action_visibility": "none",
"model.input_paradigm": "t_concat",
"model.view_layout": "robotwin_t",
"model.depth.repr": "dvae",
"model.depth.condition_source": "estimated",
"model.depth.target_source": "dataset",
"model.depth.condition_temporal_context": "current_frame",
"model.depth_projection": "independent",
"model.semantic_projection": "independent",
"model.semantic.parametrization": "v_pred",
"model.semantic.compression": "svae",
"model.semantic.channels": 48,
"model.semantic.compression.output_normalization": "layernorm"
}
```
The complete saved contract is authoritative. Restore it before model construction; do not reinterpret this checkpoint with the current default recipe. The semantic compressor is embedded. Other frozen encoders/codecs are external dependencies with the exact expected hashes recorded in the manifest and inference configuration.
## Integrity
- Weight SHA256: `e0ed5e94cdc6e469e79151b28d472c295b4b73e332502e156dee9543232f9019`
- Weight bytes: `14649561065`
- Statistics file SHA256: `e491398d0dc7cfa8d83950ef792afb2d1318cc5162df15ee62edc41a817bc75d`
- Numerical statistics fingerprint: `93a1d2c296037547209132c79d5966c44e80d147db2359ca952574bd5795bef1`
- Checkpoint version: 2
- Complete optimizer/RNG state included: **no**
## License and attribution
SemWAM-authored material is released under the MIT license; see `LICENSE`. Third-party pretrained components retain their respective licenses and attribution requirements; see `THIRD_PARTY_NOTICES.md`. External frozen assets are not redistributed in this package.
No paper identifier or BibTeX citation is supplied because a verified publication identifier has not been recorded for this release. Cite the model repository and the exact revision used in your experiments.