hamer-p150 on Tenstorrent p150
Derived from HaMeR (geopavlakos/hamer) by Georgios Pavlakos and co-authors, "Reconstructing Hands in 3D with Transformers" (CVPR 2024, arXiv:2312.05251). The weights at changh95/hamer-weights are the authors' released checkpoint, unmodified. Port: changh95/tt-hamer. See the upstream repository for training and evaluation details.
HaMeR (Hand Mesh Recovery), a 3D hand pose and shape regressor with about 670M parameters: a ViT-H/16 backbone (631M) on a 256×192 hand crop and a transformer head (39M) that regresses the parameters of the MANO hand model. One request takes a full image (PNG or JPEG), one hand box and whether the hand is a left or right hand; the box is cropped to a square of twice its larger side (by default) and resized, and left hands are mirrored through the right-hand model and mirrored back. It returns the MANO pose (global orientation and 15 joint rotations as 3×3 matrices), the 10 MANO shape coefficients, the crop camera and the camera translation of the hand in the full image. With your own MANO model file it also returns the 778-vertex hand mesh, 21 3D joints and their 2D positions in the image. Hand detection is not included: the caller supplies the boxes.
Runs on Tenstorrent p150 (mesh P150).
Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).
At a glance
| Hardware | p150 |
| License | MIT (HaMeR code and weights) with the MANO non-commercial research licence |
Intended use
Non-commercial research and prototyping of 3D hand pose and shape estimation from single images, one hand box at a time, with boxes from your own hand detector or annotations.
Prerequisites
- The Tenstorrent CLI,
tt:uv tool install tenstorrent(or use tt-model alone, see the Quickstart) - Docker
- Tenstorrent hardware: p150
- Optional, for the hand mesh, the 21 joints and their 2D positions: your own MANO hand model file
MANO_RIGHT.pkl. It needs a registration at mano.is.tue.mpg.de, is licensed for non-commercial research only and may not be redistributed, so this package does not include it. Copy it to~/.cache/tt-model/hamer-p150/weights/MANO_RIGHT.pklbefore you start the server. Without it the server returns the MANO parameters and cameras only ("mano_available": false). - About 6 GB of free host RAM: the server process peaks at about 5.7 GB while it starts and holds about 4 GB while serving
- About 8 GB of free disk: the image (about 4.2 GB, downloaded as about 0.9 GB), the weights (2.7 GB, in your Hugging Face cache) and the compile cache that the first start writes (about 150 MB)
Quickstart
tt serve changh95/hamer-p150
tt serve downloads the Docker image and the changh95/hamer-weights at e8217b46d54b4e8afaaf0dd6efb6b62341c97d32 weights (into your HF cache; they are not in the image), then starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.
Without tt-cli: tt-model serve changh95/hamer-p150.
Send the first sample frame and its right-hand box to the port tt serve reported (the boxes of all four
sample frames are in media/bboxes.json):
curl -sLO https://huggingface.co/changh95/hamer-p150/resolve/main/media/sample1.jpg
printf '{"image":"%s","bbox":[20,170,295,260],"is_right":true}' "$(base64 -w0 sample1.jpg)" > req.json
curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json
Capabilities
This package exposes the model's own HTTP server on port 20000, not an OpenAI-compatible chat API — its request and response shapes are the model's own. See the author's notes for the payload it expects.
POST /predict estimates one hand; GET /health and GET /info report readiness, the weights and whether the
MANO model file was found; GET /v1/models is a stub.
| request field | meaning |
|---|---|
image |
base64 PNG or JPEG of the full image (up to 50 megapixels) |
bbox |
hand box [x1, y1, x2, y2] in pixels of the image; without it the centre square is used and the camera translation is not meaningful |
is_right |
true (default) for a right hand, false for a left hand |
rescale_factor |
side of the square crop as a multiple of the box's larger side, more than 0.5 and at most 5.0 (default 2.0) |
return_vertices |
include the mesh vertices when the MANO model file is loaded (default true) |
return_faces |
include the 1,538 mesh triangles when the MANO model file is loaded (default false) |
Response for the quickstart request (rotmats shortened to its first matrix):
{"image_size": [334, 512], "bbox": [20.0, 170.0, 295.0, 260.0], "bbox_source": "request", "is_right": true,
"rescale_factor": 2.0, "bbox_center": [157.5, 215.0], "bbox_size": 550.0, "focal_length": 10000.0,
"rotmats": [[[0.957625, 0.102389, 0.269204], [0.226289, -0.845696, -0.483313], [0.178179, 0.52375, -0.833029]], "..."],
"betas": [-0.225999, -0.1103, -0.441802, 0.039725, -0.029646, -0.017788, 0.034932, 0.083898, -0.007344, 0.085609],
"cam": [4.675733, -0.052668, 0.006226], "cam_t_full": [-0.060056, -0.02566, 7.777099],
"mano_available": false, "trace_active": true,
"timing_ms": {"preprocess": 3.21, "device": 3.92, "mano": 0.0, "total": 7.3}}
rotmats[16][3][3]: index 0 is the global (wrist) orientation in the camera frame, 1 to 15 the MANO joint rotations relative to their parent joints, all for MANO's right-hand model; outputs of a left hand are mirrored back.betas[10]: MANO shape coefficients.cam[s, tx, ty]: weak-perspective camera of the crop.cam_t_full[tx, ty, tz]: translation of the hand in metres in the camera frame of the full image, for a pinhole camera withfocal_length= 5000 / 256 × max(width, height) pixels and the principal point at the image centre.image_sizeis[width, height].- With the MANO model file:
joints[21][3]andvertices[778][3]in metres along the camera axes, without the translation (addcam_t_fullfor camera coordinates);keypoints_2d[21][2]in pixels of the image;faces[1538][3]on request. timing_ms: the server's own time for the image decode and crop (preprocess), the model call (device), the MANO layer and the whole request.- The image also contains a Python API,
hamer.HamerModel(code/hamer/api.py), which runs the same steps in-process and returns the same values;examples/quickstart.pyshows how to run it in the image.
Example output
media/sample1.jpg with the meshes of its two boxes from media/bboxes.json, drawn with the MANO hand model
from the outputs of the fp32 PyTorch reference (middle) and of the tt-nn port on a Blackhole chip (right),
same weights. Non-commercial use only: a frame of the InterHand2.6M dataset, CC BY-NC 4.0 (see Licensing).
Expected performance
| eval | this package | reference | data |
|---|---|---|---|
| Wrist-orientation error vs FreiHAND ground truth, mean / median | 8.84° / 7.59° | 8.89° / 7.58° (fp32 CPU reference, same images) | first 500 of the 3,960 FreiHAND v2 evaluation images |
| Hands within 10° / 15° / 30° of the ground-truth wrist orientation | 70.0 % / 91.4 % / 99.4 % | 70.4 % / 91.0 % / 99.4 % (same) | same 500 |
| FreiHAND PA-MPJPE / PA-MPVPE (mm), F@5 / F@15 | not measured | 6.0 / 5.7, 0.785 / 0.990 (published, HaMeR paper Table 1) | all 3,960 evaluation images |
| PCC of the 157 regression outputs vs the fp32 CPU reference, mean / min | 0.999974 / 0.998851 | – | 532 hands |
| Rotation difference to the reference, all 16 rotations: mean / p99 / max | 0.37° / 1.60° / 19.9° | – | 532 hands |
| Wrist-orientation difference to the reference, mean / max | 0.26° / 19.9° (one hand; all others ≤ 1.8°) | – | 532 hands |
Camera depth (cam_t_full z) difference to the reference, mean / max |
0.17 % / 1.50 % | – | 532 hands |
The ground truth is FreiHAND's MANO global
orientation in the camera frame (hand box: the tight box of the projected ground-truth joints), the reference is
this package's fp32 PyTorch implementation of HaMeR on the CPU with the same weights and the same pre- and
post-processing, and the published joint and mesh figures are shown for reference only. The 532 hands are
those 500 images plus the 8 hands of media/ with 4 box variants each (as annotated, shifted by 10 % in x or
y, enlarged by 15 %).
| speed (batch 1, warm, one hand) | input | median (p99) | throughput |
|---|---|---|---|
| Device time per hand, back-to-back trace replays | 256×192 crop | 3.30 ms (3.32) | 303 hands/s |
| Model call on a crop: upload, device, readback | 256×192 crop | 3.92 ms (3.99) | 255 hands/s |
Python model("sample1.jpg", bbox=...): decode, crop and resize, model call, camera |
media/sample1.jpg (334×512), right hand |
7.03 ms (7.83) | 142 hands/s |
Served POST /predict, server timing_ms.total |
same | 7.59 ms (7.86) | – |
Served POST /predict, client round trip on loopback |
same, 30.9 KB request | 8.58 ms (9.30) | – |
Medians (p99), measured with this package's image on one p150 in an 8-vCPU AMD EPYC-Rome host, without the
MANO model file:
code/scripts/bench_p150.py (device: 20 rounds of 200 trace replays; calls: 500 each) and
code/scripts/bench_http.py (1,000 sequential requests on one connection); the image decode, crop and resize
run on the host CPU (timing_ms.preprocess 3.5 ms of the served total), so the Python and served rows depend
on the host CPU and its load.
Limitations
- Not for commercial use: the MANO hand model, whose parameters the network predicts and which the mesh output
needs, is licensed for non-commercial research only, and the sample frames in
media/for non-commercial use only (CC BY-NC 4.0). - Not for medical, ergonomic, biometric or safety decisions, or for controlling robots or machines: the outputs come with no accuracy guarantee for a given image.
- Not a hand detector: it needs a box per hand and estimates one hand per request. A wrong box or a wrong
is_rightgives a wrong result without an error. - The mesh, joint and 2D outputs with a MANO model file, and
return_faces, were not tested with this package, and its joint and mesh accuracy (PA-MPJPE, PA-MPVPE) was not measured. - Batch 1, one hand per device call; requests run one at a time on the chip.
- Runs on one p150 only; multi-chip meshes and other boards are not supported or tested.
- The device computes in reduced precision (bf16, and bfp8 for some weights), so outputs differ slightly from the fp32 reference (eval table): on 1 of 532 hands, an ambiguous FreiHAND image, the wrist orientation differed by 19.9°, and both results were far from the ground truth. The server does not start if its start-up check against the CPU reference falls below PCC 0.99.
- Occluded, blurred or ambiguous hands get wrong poses on the CPU and the p150 alike: on 30.0 % of the 500 FreiHAND hands the wrist orientation was more than 10° from the ground truth (29.6 % for the fp32 reference).
Licensing
- Weights: changh95/hamer-weights at commit
e8217b46d54b, the HaMeR authors' released checkpoint (hamer.ckpt,mano_mean_params.npz), unmodified, MIT; downloaded bytt serveinto your Hugging Face cache, not redistributed here. - MANO hand model (
MANO_RIGHT.pkl, optional, supplied by you): the MANO licence, non-commercial research only; not included. - Port, serving, benchmark and example code (
code/hamer/,code/scripts/,code/tests/,examples/): Apache-2.0; the fp32 referencecode/hamer/reference.pyis adapted from geopavlakos/hamer (MIT). - tt-metal: the image is built from the public commit in Provenance plus the local changes in
patches/tt-metal-8b98410.local.patch, Apache-2.0. - Sample frames
media/sample1.jpgtomedia/sample4.jpgand the rendermedia/sample1_result.png: frames of the InterHand2.6M dataset, CC BY-NC 4.0, non-commercial use only; the render draws model outputs onsample1.jpg.
Feedback
Questions or problems with this package: open a discussion at https://huggingface.co/changh95/hamer-p150/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: support@tenstorrent.com.
Provenance
The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:
| component | built from |
|---|---|
| tt-metal | 8b98410e730bb504fea43a88609756e34821d91d (dirty tree — the image includes uncommitted changes) |
code/ digest |
b50ec424a1dbe3e0 (sha256, first 16 hex digits) |
| built | 2026-11-12T10:34:31+00:00 by tt-model 0.1.0 |
Model tree for changh95/hamer-p150
Base model
changh95/hamer-weights