cskrren commited on
Commit
db393c6
·
verified ·
1 Parent(s): 2e30b4c

Release joint Pi3 checkpoint: MANO 110k, Mask 40k, pi32 matching

Browse files
README.md ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ tags:
5
+ - stereohand
6
+ - stereo
7
+ - hand-reconstruction
8
+ - slam
9
+ - pytorch
10
+ ---
11
+ # StereoHand
12
+
13
+ [Code](https://github.com/cskrren/StereoHand)
14
+
15
+ Joint Pi3 inference checkpoint combining temporal MANO **110,000 steps**, temporal Mask **40,000 steps**, and the **Pi3 matching `pi32`** decoder/head. Temporal step counts are counted from the start of temporal fine-tuning. The hand Pi3 register token is used for shared inference.
16
+
17
+ ## Checkpoint
18
+
19
+ `stereohand_pi3_mano110k_mask40k_matching_pi32.pt` contains full reconstructed model states, including the frozen Pi3 and stereo parameters. Each hand branch is restored from its initial model plus its latest complete training delta. Independently trained Mask and MANO fusion layers are preserved. Identical tensors share storage; values and dtypes are unchanged. Optimizer state is omitted.
20
+
21
+ The file uses the `stereohand-joint-v1` format:
22
+
23
+ ```python
24
+ import torch
25
+ checkpoint = torch.load(
26
+ "stereohand_pi3_mano110k_mask40k_matching_pi32.pt",
27
+ map_location="cpu", mmap=True, weights_only=True,
28
+ )
29
+ mask_state = checkpoint["states"]["mask"]
30
+ mano_state = checkpoint["states"]["mano"]
31
+ matching_state = checkpoint["states"]["matching"]
32
+ ```
33
+
34
+ ## Use with StereoHand
35
+
36
+ Set up the code repository and its runtime assets using the repository instructions. Download the checkpoint and the compatibility utility:
37
+
38
+ ```python
39
+ from huggingface_hub import hf_hub_download
40
+ for name in ["stereohand_pi3_mano110k_mask40k_matching_pi32.pt", "unpack_checkpoint.py"]:
41
+ hf_hub_download("cskrren/StereoHand", name, local_dir="weights")
42
+ ```
43
+
44
+ The current `SharedHeads` loader expects separate Mask, MANO, and matching paths. Export them from this single-file distribution and update an existing reconstruction config:
45
+
46
+ ```bash
47
+ python weights/unpack_checkpoint.py \
48
+ weights/stereohand_pi3_mano110k_mask40k_matching_pi32.pt \
49
+ --output weights/unpacked \
50
+ --config /path/to/your/reconstruct.json
51
+ ```
52
+
53
+ Use the resulting `weights/unpacked/reconstruct.json` with the repository's reconstruction command. Its runtime/source/asset paths come from your supplied config. The complete branch file is used for both initial and checkpoint inputs, which is compatible with the existing strict loader. The output directory must be new.
54
+
55
+ ## Validation
56
+
57
+ The merge checks shared Pi3/stereo equality, matching-backbone compatibility (retaining the hand register token), finite tensor values, and exact tensor equality after saving and reloading. `VALIDATION.json` records the result. This validates checkpoint assembly; it is not a new accuracy benchmark for the 110k snapshot.
VALIDATION.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "complete": true,
3
+ "format": "stereohand-joint-v1",
4
+ "steps": {
5
+ "mask": {
6
+ "step": 40000,
7
+ "global_step": 40000,
8
+ "source_run": "stereohand_temporal_16gpu_20261105/runs/mask",
9
+ "delta_keys": 123
10
+ },
11
+ "mano": {
12
+ "step": 110000,
13
+ "global_step": 110000,
14
+ "source_run": "stereohand_temporal_16gpu_20261105/runs/mano",
15
+ "delta_keys": 605
16
+ }
17
+ },
18
+ "matching": "pi32",
19
+ "register_token_policy": "hand",
20
+ "deduplicated_tensors": 2038,
21
+ "bytes": 12035397828,
22
+ "tensor_equality_after_reload": true,
23
+ "finite_tensors": true,
24
+ "shared_backbone_verified": true,
25
+ "optimizer_included": false,
26
+ "created_at": 1794316894.7070687
27
+ }
stereohand_pi3_mano110k_mask40k_matching_pi32.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:88ee0fd5c77870b33ffc398f567566e619f869b38378322a23a875c74c1b90b4
3
+ size 12035397828
unpack_checkpoint.py ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Export a joint checkpoint to files accepted by StereoHand SharedHeads."""
2
+ import argparse,json
3
+ from pathlib import Path
4
+ import torch
5
+
6
+ def unpack(checkpoint,output,model_config):
7
+ bundle=torch.load(checkpoint,map_location='cpu',mmap=True,weights_only=True)
8
+ if bundle.get('format')!='stereohand-joint-v1':raise ValueError('Unsupported checkpoint format')
9
+ root=Path(output).resolve();root.mkdir(parents=True,exist_ok=False)
10
+ for branch in ('mask','mano','matching'):
11
+ torch.save({'model':bundle['states'][branch]},root/(branch+'.pt'))
12
+ config=json.loads(Path(model_config).read_text())
13
+ model=config.get('model',config)
14
+ model.pop('inference_cache',None)
15
+ for branch in ('mask','mano'):
16
+ model[branch+'_initial']=model[branch+'_checkpoint']=str(root/(branch+'.pt'))
17
+ model[branch+'_step']=bundle['steps'][branch]['step']
18
+ model['matching_checkpoint']=str(root/'matching.pt')
19
+ target=root/'reconstruct.json';target.write_text(json.dumps(config,indent=2))
20
+ print(target)
21
+ if __name__=='__main__':
22
+ p=argparse.ArgumentParser();p.add_argument('checkpoint');p.add_argument('--output',required=True);p.add_argument('--config',required=True)
23
+ a=p.parse_args();unpack(a.checkpoint,a.output,a.config)