YutianChen commited on
Commit
aa2cbcf
·
verified ·
1 Parent(s): b141167

Release Unified Inertial Encoder checkpoint

Browse files
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ assets/overview.png filter=lfs diff=lfs merge=lfs -text
37
+ assets/teaser.png filter=lfs diff=lfs merge=lfs -text
.hydra/config.yaml ADDED
@@ -0,0 +1,486 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ model:
2
+ tokenizer:
3
+ _target_: src.model.tokenizer.AddRegister
4
+ output_dim: ${model.token_dim}
5
+ num_register: 4
6
+ base_tokenizer:
7
+ _target_: src.model.tokenizer.ContinuousKernel_Conv1DTokenizer
8
+ output_dim: ${model.token_dim}
9
+ window_span_ms: ${model.token_size}
10
+ activation: silu
11
+ num_taps: 4
12
+ micro_split: 4
13
+ num_trunk_features: ${len:${model.feature_from_trunk}}
14
+ trunk:
15
+ _target_: src.model.transformer.Transformer
16
+ feature_dim: ${model.token_dim}
17
+ layer: 6
18
+ num_heads: 8
19
+ layer_type:
20
+ - device
21
+ - full
22
+ - device
23
+ - full
24
+ - device
25
+ - full
26
+ is_causal: ${model.is_causal}
27
+ use_rope: true
28
+ mlp_active: silu
29
+ moe_config: null
30
+ adaptor:
31
+ _target_: src.model.adaptor.EmptyAdaptor
32
+ _target_: src.model.model.FoundationIMU
33
+ token_dim: 768
34
+ token_size: 125.0
35
+ token_unit: ms
36
+ is_causal: false
37
+ trainable: all
38
+ feature_from_trunk:
39
+ - 1
40
+ - 3
41
+ - 5
42
+ data:
43
+ nymeria_t:
44
+ train:
45
+ - _target_: src.data.imu.transform.AddGravity
46
+ - _target_: src.data.imu.transform.RandomEmbodimentScale
47
+ scale_range:
48
+ - 0.75
49
+ - 1.0
50
+ - _target_: src.data.imu.transform.RandomTimeWarp
51
+ speed_range:
52
+ - 0.7
53
+ - 1.3
54
+ - _target_: src.data.imu.transform.AddNoise
55
+ acc_noise_density: 0.01
56
+ gyro_noise_density: 0.0003
57
+ - _target_: src.data.imu.transform.AddBias
58
+ acc_bias_randwalk: 0.0005
59
+ gyro_bias_randwalk: 5.0e-05
60
+ - _target_: src.data.imu.transform.AddVibration
61
+ - _target_: src.data.imu.transform.AddImpact
62
+ - _target_: src.data.imu.transform.RandomFrequencyReduction
63
+ freq_range:
64
+ - 120.0
65
+ - 240.0
66
+ - _target_: src.data.imu.transform.AddTimestampJitter
67
+ cv_range:
68
+ - 0.0
69
+ - 0.15
70
+ - _target_: src.data.imu.transform.DeterministicDropDevice
71
+ valid:
72
+ - _target_: src.data.imu.transform.AddGravity
73
+ - _target_: src.data.imu.transform.AddNoise
74
+ acc_noise_density: 0.01
75
+ gyro_noise_density: 0.0003
76
+ - _target_: src.data.imu.transform.AddBias
77
+ acc_bias_randwalk: 0.0005
78
+ gyro_bias_randwalk: 5.0e-05
79
+ - _target_: src.data.imu.transform.AddVibration
80
+ - _target_: src.data.imu.transform.AddImpact
81
+ - _target_: src.data.imu.transform.DeterministicDropDevice
82
+ naturalmotion_t:
83
+ train:
84
+ - _target_: src.data.imu.transform.AddGravity
85
+ - _target_: src.data.imu.transform.RandomEmbodimentScale
86
+ scale_range:
87
+ - 0.75
88
+ - 1.0
89
+ - _target_: src.data.imu.transform.RandomTimeWarp
90
+ speed_range:
91
+ - 0.7
92
+ - 1.3
93
+ - _target_: src.data.imu.transform.AddNoise
94
+ acc_noise_density: 0.01
95
+ gyro_noise_density: 0.0003
96
+ - _target_: src.data.imu.transform.AddBias
97
+ acc_bias_randwalk: 0.0005
98
+ gyro_bias_randwalk: 5.0e-05
99
+ - _target_: src.data.imu.transform.AddVibration
100
+ - _target_: src.data.imu.transform.AddImpact
101
+ - _target_: src.data.imu.transform.RandomFrequencyReduction
102
+ freq_range:
103
+ - 120.0
104
+ - 240.0
105
+ - _target_: src.data.imu.transform.AddTimestampJitter
106
+ cv_range:
107
+ - 0.0
108
+ - 0.15
109
+ - _target_: src.data.imu.transform.DeterministicDropDevice
110
+ valid:
111
+ - _target_: src.data.imu.transform.AddGravity
112
+ - _target_: src.data.imu.transform.AddNoise
113
+ acc_noise_density: 0.01
114
+ gyro_noise_density: 0.0003
115
+ - _target_: src.data.imu.transform.AddBias
116
+ acc_bias_randwalk: 0.0005
117
+ gyro_bias_randwalk: 5.0e-05
118
+ - _target_: src.data.imu.transform.AddVibration
119
+ - _target_: src.data.imu.transform.AddImpact
120
+ - _target_: src.data.imu.transform.DeterministicDropDevice
121
+ bones_soma_t:
122
+ train:
123
+ - _target_: src.data.imu.transform.AddGravity
124
+ - _target_: src.data.imu.transform.RandomEmbodimentScale
125
+ scale_range:
126
+ - 0.75
127
+ - 1.0
128
+ - _target_: src.data.imu.transform.RandomTimeWarp
129
+ speed_range:
130
+ - 0.7
131
+ - 1.3
132
+ - _target_: src.data.imu.transform.AddNoise
133
+ acc_noise_density: 0.01
134
+ gyro_noise_density: 0.0003
135
+ - _target_: src.data.imu.transform.AddBias
136
+ acc_bias_randwalk: 0.0005
137
+ gyro_bias_randwalk: 5.0e-05
138
+ - _target_: src.data.imu.transform.AddVibration
139
+ - _target_: src.data.imu.transform.AddImpact
140
+ - _target_: src.data.imu.transform.RandomFrequencyReduction
141
+ freq_range:
142
+ - 120.0
143
+ - 240.0
144
+ - _target_: src.data.imu.transform.AddTimestampJitter
145
+ cv_range:
146
+ - 0.0
147
+ - 0.15
148
+ - _target_: src.data.imu.transform.DeterministicDropDevice
149
+ valid:
150
+ - _target_: src.data.imu.transform.AddGravity
151
+ - _target_: src.data.imu.transform.AddNoise
152
+ acc_noise_density: 0.01
153
+ gyro_noise_density: 0.0003
154
+ - _target_: src.data.imu.transform.AddBias
155
+ acc_bias_randwalk: 0.0005
156
+ gyro_bias_randwalk: 5.0e-05
157
+ - _target_: src.data.imu.transform.AddVibration
158
+ - _target_: src.data.imu.transform.AddImpact
159
+ - _target_: src.data.imu.transform.DeterministicDropDevice
160
+ hiw_500_t:
161
+ train:
162
+ - _target_: src.data.imu.transform.RandomEmbodimentScale
163
+ scale_range:
164
+ - 1.0
165
+ - 1.33
166
+ - _target_: src.data.imu.transform.RandomTimeWarp
167
+ speed_range:
168
+ - 0.9
169
+ - 1.6
170
+ - _target_: src.data.imu.transform.AddNoise
171
+ acc_noise_density: 0.01
172
+ gyro_noise_density: 0.0003
173
+ exclude_devices:
174
+ - pelvis
175
+ - _target_: src.data.imu.transform.AddBias
176
+ acc_bias_randwalk: 0.0005
177
+ gyro_bias_randwalk: 5.0e-05
178
+ exclude_devices:
179
+ - pelvis
180
+ - _target_: src.data.imu.transform.DeterministicDropDevice
181
+ valid:
182
+ - _target_: src.data.imu.transform.AddNoise
183
+ acc_noise_density: 0.01
184
+ gyro_noise_density: 0.0003
185
+ exclude_devices:
186
+ - pelvis
187
+ - _target_: src.data.imu.transform.AddBias
188
+ acc_bias_randwalk: 0.0005
189
+ gyro_bias_randwalk: 5.0e-05
190
+ exclude_devices:
191
+ - pelvis
192
+ - _target_: src.data.imu.transform.DeterministicDropDevice
193
+ bones_g1_t:
194
+ train:
195
+ - _target_: src.data.imu.transform.AddGravity
196
+ - _target_: src.data.imu.transform.RandomEmbodimentScale
197
+ scale_range:
198
+ - 1.0
199
+ - 1.33
200
+ - _target_: src.data.imu.transform.RandomTimeWarp
201
+ speed_range:
202
+ - 0.9
203
+ - 1.6
204
+ - _target_: src.data.imu.transform.AddNoise
205
+ acc_noise_density: 0.01
206
+ gyro_noise_density: 0.0003
207
+ - _target_: src.data.imu.transform.AddBias
208
+ acc_bias_randwalk: 0.0005
209
+ gyro_bias_randwalk: 5.0e-05
210
+ - _target_: src.data.imu.transform.AddVibration
211
+ - _target_: src.data.imu.transform.AddImpact
212
+ - _target_: src.data.imu.transform.AddTimestampJitter
213
+ cv_range:
214
+ - 0.0
215
+ - 0.15
216
+ - _target_: src.data.imu.transform.DeterministicDropDevice
217
+ valid:
218
+ - _target_: src.data.imu.transform.AddGravity
219
+ - _target_: src.data.imu.transform.AddNoise
220
+ acc_noise_density: 0.01
221
+ gyro_noise_density: 0.0003
222
+ - _target_: src.data.imu.transform.AddBias
223
+ acc_bias_randwalk: 0.0005
224
+ gyro_bias_randwalk: 5.0e-05
225
+ - _target_: src.data.imu.transform.AddVibration
226
+ - _target_: src.data.imu.transform.AddImpact
227
+ - _target_: src.data.imu.transform.DeterministicDropDevice
228
+ window:
229
+ train:
230
+ _target_: src.data.imu.window.RandomWindow_IMU
231
+ window_size: ${data_conf.window_size}
232
+ window_unit: ${data_conf.window_unit}
233
+ valid:
234
+ _target_: src.data.imu.window.SlidingWindow_IMU
235
+ window_size: ${data_conf.window_size}
236
+ window_unit: ${data_conf.window_unit}
237
+ step_size: ${data_conf.window_size}
238
+ train:
239
+ - _target_: src.data.util.build_corpus
240
+ _recursive_: false
241
+ reader:
242
+ _target_: src.data.imu.nymeria_processed_lang.Nymeria_XSens_Processed_IMU_Language
243
+ target_sensors:
244
+ - pelvis
245
+ - head
246
+ - torso_t8
247
+ - left_shoulder
248
+ - left_upper_arm
249
+ - left_forearm
250
+ - left_hand
251
+ - right_shoulder
252
+ - right_upper_arm
253
+ - right_forearm
254
+ - right_hand
255
+ - left_upper_leg
256
+ - left_lower_leg
257
+ - left_foot
258
+ - right_upper_leg
259
+ - right_lower_leg
260
+ - right_foot
261
+ root: ${machine-conf:cache_root}/nymeria_imu_processed_v3
262
+ stems: src/config/data/stems/nymeria_train.txt
263
+ layers:
264
+ - ${data.nymeria_t.train}
265
+ - ${data.window.train}
266
+ - _target_: src.data.util.build_corpus
267
+ _recursive_: false
268
+ reader:
269
+ _target_: src.data.imu.naturalmotion_processed_lang.NaturalMotion_XSens_Processed_IMU_Language
270
+ target_sensors:
271
+ - pelvis
272
+ - head
273
+ - torso_t8
274
+ - left_shoulder
275
+ - left_upper_arm
276
+ - left_forearm
277
+ - left_hand
278
+ - right_shoulder
279
+ - right_upper_arm
280
+ - right_forearm
281
+ - right_hand
282
+ - left_upper_leg
283
+ - left_lower_leg
284
+ - left_foot
285
+ - right_upper_leg
286
+ - right_lower_leg
287
+ - right_foot
288
+ root: ${machine-conf:cache_root}/vt_natural_motion_processed_v3
289
+ stems: src/config/data/stems/naturalmotion_train.txt
290
+ layers:
291
+ - ${data.naturalmotion_t.train}
292
+ - ${data.window.train}
293
+ - _target_: src.data.util.build_corpus
294
+ _recursive_: false
295
+ reader:
296
+ _target_: src.data.imu.bones_seed_processed.Bones_Seed_SOMA_Processed_IMU_w_Lang
297
+ root: ${machine-conf:cache_root}/bones_seed_soma_processed_v4
298
+ stems: src/config/data/stems/bones_segments/bones_soma_train.txt
299
+ layers:
300
+ - ${data.bones_soma_t.train}
301
+ - ${data.window.train}
302
+ - _target_: src.data.util.build_corpus
303
+ _recursive_: false
304
+ reader:
305
+ _target_: src.data.imu.bones_seed_processed.Bones_Seed_G1_Processed_IMU_w_Lang
306
+ root: ${machine-conf:cache_root}/bones_seed_processed_v4
307
+ stems: src/config/data/stems/bones_segments/bones_g1_train.txt
308
+ layers:
309
+ - ${data.bones_g1_t.train}
310
+ - ${data.window.train}
311
+ valid:
312
+ - _target_: src.data.util.build_corpus
313
+ _recursive_: false
314
+ reader:
315
+ _target_: src.data.imu.nymeria_processed_lang.Nymeria_XSens_Processed_IMU_Language
316
+ target_sensors:
317
+ - pelvis
318
+ - head
319
+ - torso_t8
320
+ - left_shoulder
321
+ - left_upper_arm
322
+ - left_forearm
323
+ - left_hand
324
+ - right_shoulder
325
+ - right_upper_arm
326
+ - right_forearm
327
+ - right_hand
328
+ - left_upper_leg
329
+ - left_lower_leg
330
+ - left_foot
331
+ - right_upper_leg
332
+ - right_lower_leg
333
+ - right_foot
334
+ root: ${machine-conf:cache_root}/nymeria_imu_processed_v3
335
+ stems: src/config/data/stems/nymeria_valid.txt
336
+ layers:
337
+ - ${data.nymeria_t.valid}
338
+ - ${data.window.valid}
339
+ - _target_: src.data.util.build_corpus
340
+ _recursive_: false
341
+ reader:
342
+ _target_: src.data.imu.naturalmotion_processed_lang.NaturalMotion_XSens_Processed_IMU_Language
343
+ target_sensors:
344
+ - pelvis
345
+ - head
346
+ - torso_t8
347
+ - left_shoulder
348
+ - left_upper_arm
349
+ - left_forearm
350
+ - left_hand
351
+ - right_shoulder
352
+ - right_upper_arm
353
+ - right_forearm
354
+ - right_hand
355
+ - left_upper_leg
356
+ - left_lower_leg
357
+ - left_foot
358
+ - right_upper_leg
359
+ - right_lower_leg
360
+ - right_foot
361
+ root: ${machine-conf:cache_root}/vt_natural_motion_processed_v3
362
+ stems: src/config/data/stems/naturalmotion_valid.txt
363
+ layers:
364
+ - ${data.naturalmotion_t.valid}
365
+ - ${data.window.valid}
366
+ - _target_: src.data.util.build_corpus
367
+ _recursive_: false
368
+ reader:
369
+ _target_: src.data.imu.bones_seed_processed.Bones_Seed_SOMA_Processed_IMU_w_Lang
370
+ root: ${machine-conf:cache_root}/bones_seed_soma_processed_v4
371
+ stems: src/config/data/stems/bones_segments/bones_soma_valid.txt
372
+ layers:
373
+ - ${data.bones_soma_t.valid}
374
+ - ${data.window.valid}
375
+ - _target_: src.data.util.build_corpus
376
+ _recursive_: false
377
+ reader:
378
+ _target_: src.data.imu.bones_seed_processed.Bones_Seed_G1_Processed_IMU_w_Lang
379
+ root: ${machine-conf:cache_root}/bones_seed_processed_v4
380
+ stems:
381
+ - src/config/data/stems/bones_segments/bones_g1_valid.txt
382
+ - src/config/data/stems/bones_segments/bones_g1_human2robot.txt
383
+ layers:
384
+ - ${data.bones_g1_t.valid}
385
+ - ${data.window.valid}
386
+ - _target_: src.data.util.build_corpus
387
+ _recursive_: false
388
+ reader:
389
+ _target_: src.data.imu.humanoid_superodom_npz.Humanoid_SuperOdom_NPZ_Processed_Language
390
+ root: ${machine-conf:cache_root}/humanoid_superodom_clips_v2
391
+ stems: src/config/data/stems/superodom_clips_eval.txt
392
+ layers:
393
+ - ${data.superodom_t.valid}
394
+ - ${data.window.valid}
395
+ superodom_t:
396
+ train:
397
+ - _target_: src.data.imu.transform.DeterministicDropDevice
398
+ valid:
399
+ - _target_: src.data.imu.transform.DeterministicDropDevice
400
+ optim:
401
+ _target_: src.model.optimizer_util.AdamW_Muon_Hybrid_Optimizer
402
+ muon_lr: 0.002
403
+ muon_weight_decay: 0.001
404
+ muon_momentum: 0.95
405
+ muon_nesterov: true
406
+ muon_ns_steps: 5
407
+ muon_adjust_lr_fn: match_rms_adamw
408
+ adamw_lr: 0.005
409
+ adamw_weight_decay: 0.001
410
+ adamw_betas:
411
+ - 0.9
412
+ - 0.999
413
+ adamw_eps: 1.0e-08
414
+ loss:
415
+ _target_: src.model.loss.LossCompose_AutoBalance
416
+ anchor: L_denoise
417
+ min_lambda: 0.02
418
+ max_lambda: 50.0
419
+ ema_decay: 0.99
420
+ funcs:
421
+ - name: L_denoise
422
+ grad: true
423
+ scale: auto
424
+ func:
425
+ _target_: src.model.loss.Denoise_Loss
426
+ masked_weight: 0.0
427
+ unmasked_weight: 1.0
428
+ - name: L_vel
429
+ grad: true
430
+ scale: auto
431
+ func:
432
+ _target_: src.model.loss.Velocity_Loss
433
+ masked_weight: 0.0
434
+ unmasked_weight: 1.0
435
+ huber_delta: null
436
+ - name: L_Lang
437
+ grad: true
438
+ scale: 1.0
439
+ func:
440
+ _target_: src.model.loss.Lang_CLIP_Loss
441
+ - name: L_traj_vel
442
+ grad: true
443
+ scale: auto
444
+ func:
445
+ _target_: src.model.loss.Trajectory_Velocity_Loss
446
+ horizons:
447
+ - 250.0
448
+ - 1000.0
449
+ - 2000.0
450
+ strides: null
451
+ unit: ms
452
+ integrator: rk4
453
+ precision: float32
454
+ huber_delta: null
455
+ detach_rotation: true
456
+ debug: false
457
+ data_conf:
458
+ window_size: 4096.0
459
+ window_step: 4096.0
460
+ window_unit: ms
461
+ wandb:
462
+ enable: true
463
+ entity: null
464
+ project: Unified-Inertial-Encoder
465
+ runid: Exp15_FullTrain_C
466
+ meta:
467
+ seed: 0
468
+ ema_decay:
469
+ - 0.8
470
+ - 0.9
471
+ output: ${hydra:runtime.output_dir}
472
+ allow_ddp_unused_param: false
473
+ resume_from: null
474
+ resume_full_state: true
475
+ loader:
476
+ batch_size: 128
477
+ num_workers: 8
478
+ prefetch_factor: 1
479
+ trainer:
480
+ accelerator: auto
481
+ devices: auto
482
+ precision: bf16-true
483
+ max_epochs: 64
484
+ gradient_clip_val: 1.0
485
+ log_every_n_steps: 1
486
+ accumulate_grad_batches: 1
.hydra/hydra.yaml ADDED
@@ -0,0 +1,170 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ hydra:
2
+ run:
3
+ dir: ${machine-conf:exp_root}/${now:%Y-%m-%d}/${now:%H-%M-%S}
4
+ sweep:
5
+ dir: multirun/${now:%Y-%m-%d}/${now:%H-%M-%S}
6
+ subdir: ${hydra.job.num}
7
+ launcher:
8
+ _target_: hydra._internal.core_plugins.basic_launcher.BasicLauncher
9
+ sweeper:
10
+ _target_: hydra._internal.core_plugins.basic_sweeper.BasicSweeper
11
+ max_batch_size: null
12
+ params: null
13
+ help:
14
+ app_name: ${hydra.job.name}
15
+ header: '${hydra.help.app_name} is powered by Hydra.
16
+
17
+ '
18
+ footer: 'Powered by Hydra (https://hydra.cc)
19
+
20
+ Use --hydra-help to view Hydra specific help
21
+
22
+ '
23
+ template: '${hydra.help.header}
24
+
25
+ == Configuration groups ==
26
+
27
+ Compose your configuration from those groups (group=option)
28
+
29
+
30
+ $APP_CONFIG_GROUPS
31
+
32
+
33
+ == Config ==
34
+
35
+ Override anything in the config (foo.bar=value)
36
+
37
+
38
+ $CONFIG
39
+
40
+
41
+ ${hydra.help.footer}
42
+
43
+ '
44
+ hydra_help:
45
+ template: 'Hydra (${hydra.runtime.version})
46
+
47
+ See https://hydra.cc for more info.
48
+
49
+
50
+ == Flags ==
51
+
52
+ $FLAGS_HELP
53
+
54
+
55
+ == Configuration groups ==
56
+
57
+ Compose your configuration from those groups (For example, append hydra/job_logging=disabled
58
+ to command line)
59
+
60
+
61
+ $HYDRA_CONFIG_GROUPS
62
+
63
+
64
+ Use ''--cfg hydra'' to Show the Hydra config.
65
+
66
+ '
67
+ hydra_help: ???
68
+ hydra_logging:
69
+ version: 1
70
+ formatters:
71
+ simple:
72
+ format: '[%(asctime)s][HYDRA] %(message)s'
73
+ handlers:
74
+ console:
75
+ class: logging.StreamHandler
76
+ formatter: simple
77
+ stream: ext://sys.stdout
78
+ root:
79
+ level: INFO
80
+ handlers:
81
+ - console
82
+ loggers:
83
+ logging_example:
84
+ level: DEBUG
85
+ disable_existing_loggers: false
86
+ job_logging:
87
+ version: 1
88
+ formatters:
89
+ simple:
90
+ format: '[%(asctime)s][%(name)s][%(levelname)s] - %(message)s'
91
+ handlers:
92
+ console:
93
+ class: logging.StreamHandler
94
+ formatter: simple
95
+ stream: ext://sys.stdout
96
+ file:
97
+ class: logging.FileHandler
98
+ formatter: simple
99
+ filename: ${hydra.runtime.output_dir}/${hydra.job.name}.log
100
+ root:
101
+ level: INFO
102
+ handlers:
103
+ - console
104
+ - file
105
+ disable_existing_loggers: false
106
+ env: {}
107
+ mode: RUN
108
+ searchpath: []
109
+ callbacks: {}
110
+ output_subdir: .hydra
111
+ overrides:
112
+ hydra:
113
+ - hydra.mode=RUN
114
+ task:
115
+ - experiment=exp15_full_train_C
116
+ job:
117
+ name: train
118
+ chdir: null
119
+ override_dirname: experiment=exp15_full_train_C
120
+ id: ???
121
+ num: ???
122
+ config_name: train
123
+ env_set: {}
124
+ env_copy: []
125
+ config:
126
+ override_dirname:
127
+ kv_sep: '='
128
+ item_sep: ','
129
+ exclude_keys: []
130
+ runtime:
131
+ version: 1.3.4
132
+ version_base: '1.3'
133
+ cwd: .
134
+ config_sources:
135
+ - path: hydra.conf
136
+ schema: pkg
137
+ provider: hydra
138
+ - path: src/config
139
+ schema: file
140
+ provider: main
141
+ - path: ''
142
+ schema: structured
143
+ provider: schema
144
+ output_dir: .
145
+ choices:
146
+ experiment: exp15_full_train_C
147
+ data/transform@data.superodom_t: prune_device
148
+ loss: velocity+lang+denoise+traj
149
+ optim: muon_adamw
150
+ data: all_data
151
+ data/window: random
152
+ data/transform@data.bones_g1_t: humanoid_bones_real_disturb
153
+ data/transform@data.hiw_500_t: humanoid_hiw500
154
+ data/transform@data.bones_soma_t: human_xsens_real_disturb
155
+ data/transform@data.naturalmotion_t: human_xsens_real_disturb
156
+ data/transform@data.nymeria_t: human_xsens_real_disturb
157
+ model: continuous_foundation_imu_full
158
+ model/adaptor: empty
159
+ model/trunk: full_aa_transformer
160
+ model/tokenizer: ck_conv_4reg_tokenizer
161
+ hydra/env: default
162
+ hydra/callbacks: null
163
+ hydra/job_logging: default
164
+ hydra/hydra_logging: default
165
+ hydra/hydra_help: default
166
+ hydra/help: default
167
+ hydra/sweeper: basic
168
+ hydra/launcher: basic
169
+ hydra/output: default
170
+ verbose: false
.hydra/overrides.yaml ADDED
@@ -0,0 +1 @@
 
 
1
+ - experiment=exp15_full_train_C
README.md ADDED
@@ -0,0 +1,84 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: pytorch
3
+ license: other
4
+ pipeline_tag: robotics
5
+ tags:
6
+ - robotics
7
+ - imu
8
+ - inertial-odometry
9
+ - humanoid
10
+ - human-motion
11
+ datasets:
12
+ - bones-studio/seed
13
+ - Open-IO/humanoid-g1-odometry
14
+ ---
15
+
16
+ # A Unified Inertial Encoder for Human and Humanoid Motion: Checkpoint
17
+
18
+ [Project Page](https://unified-inertial-encoder.github.io/) | [Code](https://github.com/unified-inertial-encoder/Unified-Inertial-Encoder)
19
+
20
+ ![One inertial encoder shared by human and humanoid motion](assets/teaser.png)
21
+
22
+ One encoder, trained on human motion and kinematically retargeted G1 motion, transfers zero-shot to
23
+ a physical Unitree G1. This repository holds the checkpoint behind the paper's "Ours" results.
24
+
25
+ ![Method overview](assets/overview.png)
26
+
27
+ ## Files
28
+
29
+ ```
30
+ checkpoints/unified_encoder.ckpt model weights (EMA, bf16, 55.6M parameters)
31
+ .hydra/ Hydra config of the training run (experiment=exp15_full_train_C)
32
+ ```
33
+
34
+ Keep `.hydra/` next to `checkpoints/`, because the code rebuilds the model from it. The checkpoint
35
+ holds weights only, without optimizer state, so it supports evaluation and fine-tuning from these
36
+ weights, not exact resumption of the training run.
37
+
38
+ ## Quick Usage
39
+
40
+ With the [code](https://github.com/unified-inertial-encoder/Unified-Inertial-Encoder) installed:
41
+
42
+ ```bash
43
+ hf download Open-IO/Unified-Inertial-Encoder --local-dir ckpts/unified-inertial-encoder
44
+ hf download Open-IO/humanoid-g1-odometry --repo-type dataset \
45
+ --local-dir "$FOUNDATIONIMU_CACHE_ROOT/humanoid_superodom_clips_v2"
46
+
47
+ RUN=ckpts/unified-inertial-encoder
48
+ python -m src.eval_foundation \
49
+ --ckpt "$RUN/checkpoints/unified_encoder.ckpt" \
50
+ --data_from "$RUN" \
51
+ --dataset Humanoid_SuperOdom_NPZ_Processed_Language \
52
+ --json real_g1.json
53
+ ```
54
+
55
+ `src.eval_foundation` needs a CUDA GPU. See the code repository for the other datasets.
56
+
57
+ ## Results
58
+
59
+ Scores reported in the paper. AVE is the average velocity error (m/s). ATE<sub>GT</sub> and
60
+ ATE<sub>Pred</sub> are trajectory errors (m) using ground-truth rotation and rotation preintegrated
61
+ from the gyroscope.
62
+
63
+ | Dataset | AVE | ATE<sub>GT</sub> | ATE<sub>Pred</sub> |
64
+ |---|---|---|---|
65
+ | Nymeria | 0.026 | 2.468 | 18.20 |
66
+ | Natural Motion | 0.049 | 7.456 | 50.54 |
67
+ | BONES-Seed (Human) | 0.031 | 0.113 | 0.148 |
68
+ | BONES-Seed (G1) | 0.024 | 0.101 | 0.127 |
69
+ | Real-Trajectory (G1) | 0.099 | 3.153 | 11.50 |
70
+
71
+ Real-G1 clips are never used for gradient training, but their scores informed the selection of this
72
+ checkpoint.
73
+
74
+ ## Citation
75
+
76
+ ```bibtex
77
+ @misc{chen2026unifiedinertial,
78
+ title={A Unified Inertial Encoder for Human and Humanoid Motion},
79
+ author={Chen, Yutian and Zhao, Shibo and Duan, Rocky and Liu, Karen and
80
+ Sferrazza, Carmelo and Shi, Guanya and Qiu, Yuheng and Sreenath, Koushil},
81
+ year={2026},
82
+ url={https://unified-inertial-encoder.github.io/}
83
+ }
84
+ ```
assets/overview.png ADDED

Git LFS Details

  • SHA256: 56590dacf16a6c477769266a71e2a4f32027bd22c968d7f0fd61b25f17ad630f
  • Pointer size: 131 Bytes
  • Size of remote file: 413 kB
assets/teaser.png ADDED

Git LFS Details

  • SHA256: 729e4b90b32d08fc32875084239cac3b32ee27a79a44d02e388a53a36053235b
  • Pointer size: 131 Bytes
  • Size of remote file: 750 kB
checkpoints/unified_encoder.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b352f6125a0eb4ae077d0cf0e168da560205b1bd3306c31ed676651cbf5be954
3
+ size 111153503