File size: 2,596 Bytes
db393c6
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
---
language:
- en
tags:
- stereohand
- stereo
- hand-reconstruction
- slam
- pytorch
---
# StereoHand

[Code](https://github.com/cskrren/StereoHand)

Joint Pi3 inference checkpoint combining temporal MANO **110,000 steps**, temporal Mask **40,000 steps**, and the **Pi3 matching `pi32`** decoder/head. Temporal step counts are counted from the start of temporal fine-tuning. The hand Pi3 register token is used for shared inference.

## Checkpoint

`stereohand_pi3_mano110k_mask40k_matching_pi32.pt` contains full reconstructed model states, including the frozen Pi3 and stereo parameters. Each hand branch is restored from its initial model plus its latest complete training delta. Independently trained Mask and MANO fusion layers are preserved. Identical tensors share storage; values and dtypes are unchanged. Optimizer state is omitted.

The file uses the `stereohand-joint-v1` format:

```python
import torch
checkpoint = torch.load(
    "stereohand_pi3_mano110k_mask40k_matching_pi32.pt",
    map_location="cpu", mmap=True, weights_only=True,
)
mask_state = checkpoint["states"]["mask"]
mano_state = checkpoint["states"]["mano"]
matching_state = checkpoint["states"]["matching"]
```

## Use with StereoHand

Set up the code repository and its runtime assets using the repository instructions. Download the checkpoint and the compatibility utility:

```python
from huggingface_hub import hf_hub_download
for name in ["stereohand_pi3_mano110k_mask40k_matching_pi32.pt", "unpack_checkpoint.py"]:
    hf_hub_download("cskrren/StereoHand", name, local_dir="weights")
```

The current `SharedHeads` loader expects separate Mask, MANO, and matching paths. Export them from this single-file distribution and update an existing reconstruction config:

```bash
python weights/unpack_checkpoint.py \
  weights/stereohand_pi3_mano110k_mask40k_matching_pi32.pt \
  --output weights/unpacked \
  --config /path/to/your/reconstruct.json
```

Use the resulting `weights/unpacked/reconstruct.json` with the repository's reconstruction command. Its runtime/source/asset paths come from your supplied config. The complete branch file is used for both initial and checkpoint inputs, which is compatible with the existing strict loader. The output directory must be new.

## Validation

The merge checks shared Pi3/stereo equality, matching-backbone compatibility (retaining the hand register token), finite tensor values, and exact tensor equality after saving and reloading. `VALIDATION.json` records the result. This validates checkpoint assembly; it is not a new accuracy benchmark for the 110k snapshot.