js2025 commited on
Commit
8a3cd77
·
verified ·
1 Parent(s): 5e08971

Update model card (trimmed to match the Chinese version)

Browse files
Files changed (1) hide show
  1. README.md +6 -141
README.md CHANGED
@@ -9,16 +9,11 @@ tags:
9
  - pose-estimation
10
  ---
11
 
12
- # **AssemLM 2.0 Checkpoint**
13
-
14
- **Language:** English | [中文](README_zh.md)
15
 
16
  **Related links:** [AssemLM 1.0 paper (arXiv)](https://arxiv.org/abs/2604.08983) · [AssemLM official website](https://assemlmhome.github.io/) · [Model repository](https://huggingface.co/TeleEmbodied/AssemLM2.0) · [Dataset repository](https://huggingface.co/datasets/TeleEmbodied/AssemLM2.0_Datasets)
17
 
18
- This directory contains the AssemLM 2.0 continuous-pose checkpoint
19
- (`epoch_0215_steps_2111515`) and the matching training/evaluation configuration.
20
- The model predicts a 9D pose for the moving part of a two-part assembly from two
21
- point clouds, two rendered manual images, and an object-category instruction.
22
 
23
  ## Files
24
 
@@ -27,10 +22,6 @@ point clouds, two rendered manual images, and an object-category instruction.
27
  | `config.yaml` | Active AssemLM 2.0 model and training configuration |
28
  | `epoch_0215_steps_2111515/pytorch_model/mp_rank_00_model_states.pt` | Full model checkpoint of the release |
29
 
30
- Local working copies may additionally contain `README_zh.md` (Chinese version of
31
- this card) and backup files such as `config copy.yaml`; these are not part of the
32
- repository.
33
-
34
  Checkpoint details:
35
 
36
  | Item | Value |
@@ -41,23 +32,12 @@ Checkpoint details:
41
  | Stored format | PyTorch state dictionary |
42
  | SHA-256 | `ae2030284436e500ff090ccebb9c6135493c7ddc6f522d3ce03125d3f72aa9fc` |
43
 
44
- The Hugging Face repository is private and currently contains:
45
-
46
- ```text
47
- .gitattributes
48
- config.yaml
49
- epoch_0215_steps_2111515/pytorch_model/mp_rank_00_model_states.pt
50
- ```
51
-
52
  ## Model architecture
53
 
54
- AssemLM 2.0 combines a Qwen3-VL vision-language backbone with a
55
- rotation-equivariant point-cloud encoder and a continuous pose-regression head.
56
-
57
  | Component | Configuration |
58
  |---|---|
59
  | Framework | `AssemLM2` |
60
- | Base VLM | Qwen3-VL-2B-Instruct, supplied through `ASSEMLM_VLM_PATH` |
61
  | Attention implementation | `eager` |
62
  | Point encoder | `vn_dgcnn_patch` |
63
  | Point feature dimension | 512 |
@@ -66,129 +46,14 @@ rotation-equivariant point-cloud encoder and a continuous pose-regression head.
66
  | Patch bridge | 4 transformer layers, hidden size 1024, 8 heads |
67
  | Bridge layout | `four_branch_equiv_invariant` |
68
  | Point projector | hidden sizes `[1024, 2048]`, output size 2048 |
69
- | Point M-RoPE | enabled, grid size 64, `joint_isotropic` |
70
- | Point attention | bidirectional over all point tokens |
71
  | Pose output | 9D: 3D translation + 6D rotation |
72
  | Pose head | hidden size 4096, 2 residual blocks |
73
 
74
- ## Data contract
75
-
76
- Training and evaluation discover every `assemlm_*.hdf5` file below
77
- `ASSEMLM_DATA_ROOT`.
78
-
79
- ```yaml
80
- data_features: src_pc,tgt_pc,imgs,lang,asset,category,pre_pose
81
- import_rotation: true
82
- centering_mode: mean
83
- ```
84
-
85
- The loader applies rotation augmentation and mean centering. Translation
86
- augmentation is not used by this release.
87
-
88
- Each HDF5 sample provides:
89
-
90
- | Field | Description |
91
- |---|---|
92
- | `partA-pc` | Moving-part point cloud |
93
- | `base_partB-pc` | Fixed/base-part point cloud |
94
- | `image_base_<manual>` | Base-part manual image |
95
- | `image_assemble_<manual>` | Assembled-state manual image |
96
- | `category` | Object category |
97
-
98
- The `<manual>` suffix is `lineart` for Two by Two and `freestyle` for the other
99
- release datasets.
100
-
101
- ## Evaluation metrics
102
-
103
- The AssemLM 2.0 evaluator reports:
104
-
105
- | Metric | Meaning |
106
- |---|---|
107
- | `Avg GD` | Mean geodesic rotation error |
108
- | `RMSE(T)` | Translation RMSE |
109
- | `CD` | One half of the symmetric Chamfer distance |
110
- | `PA(0.01)` | Part accuracy for `CD < 0.01` |
111
- | `PA(0.008)` | Part accuracy for `CD < 0.008` |
112
- | `PA(0.005)` | Part accuracy for `CD < 0.005` |
113
- | `CD(R)` | Translation-corrected Chamfer distance, also divided by two |
114
-
115
- ## Loading and evaluation
116
-
117
- Set the required paths:
118
-
119
- ```bash
120
- export ASSEMLM_VLM_PATH=/path/to/Qwen3-VL-2B-Instruct
121
- export CONFIG_PATH=/path/to/AssemLM2/config.yaml
122
- export CHECKPOINT_PATH=/path/to/AssemLM2/epoch_0215_steps_2111515/pytorch_model/mp_rank_00_model_states.pt
123
- ```
124
-
125
- Run the release evaluation launcher:
126
-
127
- ```bash
128
- cd /path/to/AssemLM
129
- bash scripts/eval_assemlm_v2_seed0_9.sh
130
- ```
131
-
132
- The configuration uses:
133
-
134
- ```yaml
135
- base_vlm: ${oc.env:ASSEMLM_VLM_PATH}
136
- ```
137
-
138
- Therefore `ASSEMLM_VLM_PATH` must be available before loading `config.yaml`.
139
-
140
- To start the GUI with automatic dataset and model loading:
141
-
142
- ```bash
143
- assemlm-gui \
144
- --host 0.0.0.0 \
145
- --port 7860 \
146
- --dataset /path/to/assemlm_partnet.hdf5 \
147
- --checkpoint "${CHECKPOINT_PATH}" \
148
- --config "${CONFIG_PATH}"
149
- ```
150
-
151
- To construct the model directly:
152
-
153
- ```python
154
- from omegaconf import OmegaConf
155
- import torch
156
-
157
- from assemlm.model.framework import build_framework
158
-
159
- cfg = OmegaConf.load("config.yaml")
160
- model = build_framework(cfg)
161
- state_dict = torch.load(
162
- "epoch_0215_steps_2111515/pytorch_model/mp_rank_00_model_states.pt",
163
- map_location="cpu",
164
- )
165
- if isinstance(state_dict, dict) and "state_dict" in state_dict:
166
- state_dict = state_dict["state_dict"]
167
- model.load_state_dict(state_dict, strict=True)
168
- model.eval()
169
- ```
170
-
171
- ## Intended use
172
-
173
- This checkpoint is intended for:
174
-
175
- - AssemLM 2.0 evaluation and reproducibility studies;
176
- - continuous 6D pose prediction for two-part robotic assembly;
177
- - multimodal spatial reasoning from point clouds, rendered manuals, and text.
178
-
179
- The matching training/evaluation data is published in the companion dataset
180
- repository [`TeleEmbodied/AssemLM2.0_Datasets`](https://huggingface.co/datasets/TeleEmbodied/AssemLM2.0_Datasets)
181
- (five full HDF5 files plus the `preview/` samples and the lightweight `sub500/`
182
- subsets used by the GUI).
183
-
184
  ## Licensing
185
 
186
- This checkpoint is released for research use. It was trained on the AssemLM 2.0
187
- data collection, which aggregates several upstream 3D assembly datasets with
188
- different terms (IKEA-Manual, PartNet, PartNeXt, BiAssemble, Two by Two); see the
189
- **Licensing** section of the dataset card for the per-subset details. This
190
- release does not supersede any upstream license, and users remain responsible
191
- for complying with the terms of the original datasets.
192
 
193
  ## Citation
194
 
 
9
  - pose-estimation
10
  ---
11
 
12
+ # **AssemLM 2.0 Model**
 
 
13
 
14
  **Related links:** [AssemLM 1.0 paper (arXiv)](https://arxiv.org/abs/2604.08983) · [AssemLM official website](https://assemlmhome.github.io/) · [Model repository](https://huggingface.co/TeleEmbodied/AssemLM2.0) · [Dataset repository](https://huggingface.co/datasets/TeleEmbodied/AssemLM2.0_Datasets)
15
 
16
+ This directory contains the AssemLM 2.0 checkpoint (`epoch_0215_steps_2111515`) and the matching training/evaluation configuration. The model predicts a 9D pose for the moving part of a two-part assembly from two point clouds, two rendered manual images, and an object-category instruction.
 
 
 
17
 
18
  ## Files
19
 
 
22
  | `config.yaml` | Active AssemLM 2.0 model and training configuration |
23
  | `epoch_0215_steps_2111515/pytorch_model/mp_rank_00_model_states.pt` | Full model checkpoint of the release |
24
 
 
 
 
 
25
  Checkpoint details:
26
 
27
  | Item | Value |
 
32
  | Stored format | PyTorch state dictionary |
33
  | SHA-256 | `ae2030284436e500ff090ccebb9c6135493c7ddc6f522d3ce03125d3f72aa9fc` |
34
 
 
 
 
 
 
 
 
 
35
  ## Model architecture
36
 
 
 
 
37
  | Component | Configuration |
38
  |---|---|
39
  | Framework | `AssemLM2` |
40
+ | Base VLM | Qwen3-VL-2B-Instruct |
41
  | Attention implementation | `eager` |
42
  | Point encoder | `vn_dgcnn_patch` |
43
  | Point feature dimension | 512 |
 
46
  | Patch bridge | 4 transformer layers, hidden size 1024, 8 heads |
47
  | Bridge layout | `four_branch_equiv_invariant` |
48
  | Point projector | hidden sizes `[1024, 2048]`, output size 2048 |
49
+ | Point M-RoPE | grid size 64, `joint_isotropic` |
50
+ | Point attention | custom causal attention |
51
  | Pose output | 9D: 3D translation + 6D rotation |
52
  | Pose head | hidden size 4096, 2 residual blocks |
53
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
54
  ## Licensing
55
 
56
+ This checkpoint is released for research use. It was trained on the AssemLM 2.0 data collection, which aggregates several upstream 3D assembly datasets with different terms (IKEA-Manual, PartNet, PartNeXt, BiAssemble, Two by Two). This release does not supersede any upstream license, and users remain responsible for complying with the terms of the original datasets.
 
 
 
 
 
57
 
58
  ## Citation
59