mickeykang commited on
Commit
5a2a8ff
·
verified ·
1 Parent(s): 578537f

FINAL: 10-epoch checkpoint (step 735270)

Browse files
Files changed (3) hide show
  1. README.md +13 -8
  2. config.json +1 -1
  3. model.safetensors +1 -1
README.md CHANGED
@@ -9,18 +9,21 @@ tags:
9
  - dom
10
  ---
11
 
12
- # SmolVLA-MultiFrame on DOM — step 529,689 (~epoch 7.2)
13
 
14
  Multi-frame **SmolVLA** fine-tuned on the **DOM** (Dynamic Object Manipulation) dataset.
15
- Root holds the **latest** checkpoint: global step / batch **529,689** (~epoch 7.2 of DOM, loss ≈ 0.002).
 
16
 
17
  ## What this is
18
  - **Backbone:** `lerobot/smolvla_base` — SmolVLM2-500M-Video-Instruct (SigLIP vision + SmolLM2)
19
- + flow-matching action expert. VL-aligned + robot-pretrained.
 
20
  - **Training:** **full fine-tune** (403M / 450M trainable, vision encoder unfrozen) on `hzxie/DOM`
21
- (Franka, cameras `opst_cam` + `wrist_cam`, state 6-d, action 7-d, chunk 50).
22
  - **Multi-frame:** temporal window **{t-2, t}** (`DELTA_TIMESTAMPS observation: [-2, 0]`) — each frame is
23
- fed to SmolVLM2 as a **separate image** so the model perceives object motion (DOM is dynamic).
 
24
  - **Setup:** 8×H200, global batch 640 (40 × grad_accum 2 × 8), AdamW lr 1e-4, cosine + 1000 warmup, bf16.
25
 
26
  ## ⚠️ Important — load with MultiFrameSmolVLAPolicy
@@ -45,7 +48,9 @@ github.com/mickeykang16/DynamicVLA — branch **`smolvla-multiframe-dom`**
45
  (`policies/smolvla_multiframe.py`, `configs/smolvla.yaml`, `utils/helpers.py`).
46
 
47
  ## Notes
48
- - Mid-training checkpoint (training was still ongoing at ~epoch 8/500). Loss is deeply converged
49
- (~0.002) but **loss does not guarantee sim success** — judge by DOM sim success-rate.
50
- - Earlier checkpoints (steps 40,000 / 275,581 / 427,635) previously occupied root; recoverable from git history.
 
51
  - Built to test whether a VL-aligned backbone + multi-frame closes the DOM sim gap seen with DynamicVLA.
 
 
9
  - dom
10
  ---
11
 
12
+ # SmolVLA-MultiFrame on DOM — FINAL (10 epochs, step 735,270)
13
 
14
  Multi-frame **SmolVLA** fine-tuned on the **DOM** (Dynamic Object Manipulation) dataset.
15
+ This is the **final checkpoint** — training stopped at exactly **10 epochs** (global step / batch
16
+ **735,270** = 10 × 73,527), loss ≈ 0.0015.
17
 
18
  ## What this is
19
  - **Backbone:** `lerobot/smolvla_base` — SmolVLM2-500M-Video-Instruct (SigLIP vision + SmolLM2)
20
+ + flow-matching action expert. VL-aligned + robot-pretrained. **All weights loaded from
21
+ smolvla_base (no random init)**, then full fine-tuned.
22
  - **Training:** **full fine-tune** (403M / 450M trainable, vision encoder unfrozen) on `hzxie/DOM`
23
+ (Franka, cameras `opst_cam` + `wrist_cam`, state 6-d, action 7-d, chunk 50). 10 epochs, ~12 days on 8×H200.
24
  - **Multi-frame:** temporal window **{t-2, t}** (`DELTA_TIMESTAMPS observation: [-2, 0]`) — each frame is
25
+ fed to SmolVLM2 as a **separate image** so the model perceives object motion (DOM is dynamic; some DOM
26
+ instructions literally reference object velocity, e.g. "the moving object with the higher initial velocity").
27
  - **Setup:** 8×H200, global batch 640 (40 × grad_accum 2 × 8), AdamW lr 1e-4, cosine + 1000 warmup, bf16.
28
 
29
  ## ⚠️ Important — load with MultiFrameSmolVLAPolicy
 
48
  (`policies/smolvla_multiframe.py`, `configs/smolvla.yaml`, `utils/helpers.py`).
49
 
50
  ## Notes
51
+ - Final 10-epoch checkpoint. Loss is deeply converged (~0.0015) but **loss does not guarantee sim
52
+ success** — judge by DOM sim success-rate.
53
+ - Intermediate checkpoints (steps 40,000 / 275,581 / 427,635 / 529,689) previously occupied root;
54
+ recoverable from git history.
55
  - Built to test whether a VL-aligned backbone + multi-frame closes the DOM sim gap seen with DynamicVLA.
56
+ Evaluate vs DynamicVLA (our mid-train + the released checkpoint) on the same DOM sim harness.
config.json CHANGED
@@ -91,5 +91,5 @@
91
  0
92
  ]
93
  },
94
- "epoch": 8
95
  }
 
91
  0
92
  ]
93
  },
94
+ "epoch": 11
95
  }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3f9c29c7510ec79955324356747163dcb67404499b804a3d004c69f650dbe524
3
  size 1197790032
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:48a3d64aba3b8020b2e343cb520025aa89b369f564eeef570e27ec523cc6731c
3
  size 1197790032