myned-ai
/

facepoints

@@ -1,134 +1,134 @@
----
-license: cc-by-nc-4.0
-library_name: pytorch
-pipeline_tag: keypoint-detection
-tags:
-  - face-landmarks
-  - gnm
-  - 3d-face
-  - dense-landmarks
----
-# Facepoints: 620 face points on the GNM Head mesh
-![The 620 points and the 68- and 98-point sets on a studio photo](images/intro.jpg)
-Finds **620 points on a face in a photo**. Every point is a fixed location on Google's open
-[GNM Head](https://github.com/google/GNM) mesh, so the output fits that model to a person directly —
-for building or animating a 3D avatar. The same forward pass also gives the standard 68-point (iBUG)
-and 98-point (WFLW) landmark sets and an error estimate for every point. To our knowledge (2026-10-23)
-it is the first publicly released detector trained on points defined on the GNM mesh.
-| | |
-|---|---|
-| Input | one face, cropped from a photo or video frame; a face detector supplies the box |
-| Output | 620 points, each (x, y) plus its own error estimate, in pixels; the 68- and 98-point sets |
-| Points | 416 landmark and 150 skin points, 36 eyelid-edge, 16 front teeth (7 upper, 9 lower), 2 iris centres (`points_gnm.json` maps each to the GNM Head mesh, v3.0) |
-| Model | HRNet-W48, 256-px input, 102.7 M parameters (+1.2 M for the 68/98 heads) |
-| Weights | `model/model.safetensors` + `model/config.json`; SHA-256 of the weights `b37075caa3eaf0d00f23f32925b16ec3a8e2e45c42aa9c00956f94f2574e6526` |
-For face tracking and avatar research. Not for identifying or judging people, and not for commercial
-use (the weights are CC BY-NC 4.0; see *Licence*).
-## Use
-```bash
-git clone https://huggingface.co/myned-ai/facepoints && cd facepoints   # git-lfs for the weights
-pip install -e .    # torch, timm, safetensors, opencv-python-headless, numpy
-```
-```python
-import cv2
-from facepoints import Detector, square_box, gnm_positions
-det = Detector("./model")                       # the folder with model.safetensors + config.json
-img = cv2.imread("face.jpg")                    # BGR
-x, y, w, h = my_face_detector(img)              # any face box, e.g. OpenCV's YuNet
-out = det.detect(img, square_box(x, y, w, h))   # the box, then two passes on the model's own extent
-out["points"], out["std"]   # (620, 2) image px, and one predicted std per point (px)
-out["300w"], out["wflw"]    # (68, 2) and (98, 2)
-gnm_positions(mesh_xyz)     # (620, 3): the same points on any posed GNM Head mesh (weights x vertices)
-```
-Fitting the GNM Head to a face = solving for the mesh whose projected `gnm_positions` match
-`out["points"]`, weighted by `1 / std`. One face at a time; the output moves with the box, which is
-why `detect` re-frames on its own points. A box with no face in it yields points with a very large
-`std` — threshold it. A complete example with OpenCV's face finder:
-`python run.py photo.jpg out.jpg` (the model from `./model`, or `--model DIR`); tests: `pytest tests`.
-## Results
-**Against registered 3D meshes** of 9 studio people none of the models trained on, 14 cameras, their
-widest mouth shapes. The score is how well the points follow the face: median distance to the paired
-truth vertex, in mm, after removing each point's constant offset at rest (protocol in the report).
-![Median error against the registered mesh: ours, Pixel3DMM, 3DDFA-V3](images/scan_mm.png)
-| | lips | cheeks | chin | rest of face |
-|---|---|---|---|---|
-| **ours** | **1.78** | **2.00** | 2.12 | 0.81 |
-| Pixel3DMM | 1.81 | 2.19 | **1.67** | **0.78** |
-| 3DDFA-V3 | 2.40 | 2.61 | 2.84 | 1.41 |
-Level with Pixel3DMM except on the chin, ahead of 3DDFA-V3 on every region.
-![Three Multiface people, frontal and side view, per-point error of each model](images/examples_scan.jpg)
-**On the 2D benchmarks** (the 68/98-point outputs; full test sets; NME = mean point error as % of the
-eye-corner distance):
-![NME on 300W, 300VW and WFLW: ours and five other models](images/bench_2d.png)
-| | 300W | 300VW | WFLW |
-|---|---|---|---|
-| **ours, 68/98-point heads** | 3.33 | 3.52 | 4.66 |
-| **ours, 620 points through a linear map** | 3.31 | 3.56 | 4.59 |
-| STAR (official checkpoints) | 3.06 | 3.45 | 4.47 |
-| SPIGA | 3.02 | 3.37 | 4.06 |
-| RwSA (ICCV 2025) | 2.96 | 3.38 | 4.23 |
-| FAN | 3.69 | 4.11 | 7.27 |
-| MediaPipe (478 points, mapped) | 3.87 | 4.00 | 7.72 |
-Behind the three models built for these benchmarks by 0.2–0.6, level with STAR on video, ahead of FAN
-and MediaPipe everywhere. Profile faces are the weak spot.
-![WFLW test faces, one per condition: ours, STAR, SPIGA, RwSA, MediaPipe](images/examples_wflw.jpg)
-**The error estimate** (`std`) comes from a Student-t likelihood (ν = 10). Error rises with it, but
-the intervals are too narrow: multiply `std` by 1.34 to calibrate on average. Measured on studio
-faces only.
-## Limitations
-- Hidden points (teeth behind closed lips, the far side of a turned face) are estimated, not seen;
-  sunglasses give guessed eyelids and irises; closed eyes come out about 1 mm open.
-- The 620 points have no hand-labelled truth on wild photos; their wild accuracy is known only through
-  the 68/98 outputs.
-- On 24 held-out studio people, error correlated with darker skin (Spearman −0.44, p = 0.03),
-  confounded with exposure; Ava-256 is biased to lighter-skinned people under 35.
-## Training
-The 620 points: Ava-256 (Meta), 230 people, 16 cameras; labels are projections of each person's
-registered mesh, whose vertices were mapped onto the GNM template. The 68/98 heads: the 300W and WFLW
-training photos; on those photos the 620 points carry pseudo-labels. HRNet-W48, three stages, one
-run. To replicate the model, follow [train/README.md](train/README.md): data preparation, the three
-commands and the export. The scripts and the shipped sample lists, split and pseudo-label map:
-[prep/](prep/); every key: [train/config.py](train/config.py).
-Protocol, data and analysis in full: technical report (in preparation).
-## Licence
-Copyright 2026 Myned AI (opensource@myned.ai).
-| | licence | source |
-|---|---|---|
-| code | Apache-2.0 | `LICENSE` |
-| `model/model.safetensors`, `points_gnm.json`, `prep/*.npz` | CC BY-NC 4.0 | `LICENSE-WEIGHTS` |
-| Ava-256 · Multiface (images) | CC BY-NC 4.0 | LICENSE files |
-| 300W · WFLW (WIDER FACE images) · 300VW | research only · CC BY-NC-ND · none stated | dataset pages |
-| ImageNet start (timm `hrnet_w48.ms_in1k`) | weights MIT; ImageNet terms non-commercial | model card |
-| GNM Head | Apache-2.0, attribution required | LICENSE file |
-Figures show Multiface people (consented) and WFLW test faces as the benchmark publishes them.

+---
+license: cc-by-nc-4.0
+library_name: pytorch
+pipeline_tag: keypoint-detection
+tags:
+  - face-landmarks
+  - gnm
+  - 3d-face
+  - dense-landmarks
+---
+# Facepoints: 620 dense face landmarks on the GNM Head mesh
+![The 620 points and the 68- and 98-point sets on a studio photo](images/intro.jpg)
+Finds **620 points on a face in a photo**. Every point is a fixed location on Google's open
+[GNM Head](https://github.com/google/GNM) mesh, so the output fits that model to a person directly —
+for building or animating a 3D avatar. The same forward pass also gives the standard 68-point (iBUG)
+and 98-point (WFLW) landmark sets and an error estimate for every point. To our knowledge (2026-10-23)
+it is the first publicly released detector trained on points defined on the GNM mesh.
+| | |
+|---|---|
+| Input | one face, cropped from a photo or video frame; a face detector supplies the box |
+| Output | 620 points, each (x, y) plus its own error estimate, in pixels; the 68- and 98-point sets |
+| Points | 416 landmark and 150 skin points, 36 eyelid-edge, 16 front teeth (7 upper, 9 lower), 2 iris centres (`points_gnm.json` maps each to the GNM Head mesh, v3.0) |
+| Model | HRNet-W48, 256-px input, 102.7 M parameters (+1.2 M for the 68/98 heads) |
+| Weights | `model/model.safetensors` + `model/config.json`; SHA-256 of the weights `b37075caa3eaf0d00f23f32925b16ec3a8e2e45c42aa9c00956f94f2574e6526` |
+For face tracking and avatar research. Not for identifying or judging people, and not for commercial
+use (the weights are CC BY-NC 4.0; see *Licence*).
+## Use
+```bash
+git clone https://huggingface.co/myned-ai/facepoints && cd facepoints   # git-lfs for the weights
+pip install -e .    # torch, timm, safetensors, opencv-python-headless, numpy
+```
+```python
+import cv2
+from facepoints import Detector, square_box, gnm_positions
+det = Detector("./model")                       # the folder with model.safetensors + config.json
+img = cv2.imread("face.jpg")                    # BGR
+x, y, w, h = my_face_detector(img)              # any face box, e.g. OpenCV's YuNet
+out = det.detect(img, square_box(x, y, w, h))   # the box, then two passes on the model's own extent
+out["points"], out["std"]   # (620, 2) image px, and one predicted std per point (px)
+out["300w"], out["wflw"]    # (68, 2) and (98, 2)
+gnm_positions(mesh_xyz)     # (620, 3): the same points on any posed GNM Head mesh (weights x vertices)
+```
+Fitting the GNM Head to a face = solving for the mesh whose projected `gnm_positions` match
+`out["points"]`, weighted by `1 / std`. One face at a time; the output moves with the box, which is
+why `detect` re-frames on its own points. A box with no face in it yields points with a very large
+`std` — threshold it. A complete example with OpenCV's face finder:
+`python run.py photo.jpg out.jpg` (the model from `./model`, or `--model DIR`); tests: `pytest tests`.
+## Results
+**Against registered 3D meshes** of 9 studio people none of the models trained on, 14 cameras, their
+widest mouth shapes. The score is how well the points follow the face: median distance to the paired
+truth vertex, in mm, after removing each point's constant offset at rest (protocol in the report).
+![Median error against the registered mesh: ours, Pixel3DMM, 3DDFA-V3](images/scan_mm.png)
+| | lips | cheeks | chin | rest of face |
+|---|---|---|---|---|
+| **ours** | **1.78** | **2.00** | 2.12 | 0.81 |
+| Pixel3DMM | 1.81 | 2.19 | **1.67** | **0.78** |
+| 3DDFA-V3 | 2.40 | 2.61 | 2.84 | 1.41 |
+Level with Pixel3DMM except on the chin, ahead of 3DDFA-V3 on every region.
+![Three Multiface people, frontal and side view, per-point error of each model](images/examples_scan.jpg)
+**On the 2D benchmarks** (the 68/98-point outputs; full test sets; NME = mean point error as % of the
+eye-corner distance):
+![NME on 300W, 300VW and WFLW: ours and five other models](images/bench_2d.png)
+| | 300W | 300VW | WFLW |
+|---|---|---|---|
+| **ours, 68/98-point heads** | 3.33 | 3.52 | 4.66 |
+| **ours, 620 points through a linear map** | 3.31 | 3.56 | 4.59 |
+| STAR (official checkpoints) | 3.06 | 3.45 | 4.47 |
+| SPIGA | 3.02 | 3.37 | 4.06 |
+| RwSA (ICCV 2025) | 2.96 | 3.38 | 4.23 |
+| FAN | 3.69 | 4.11 | 7.27 |
+| MediaPipe (478 points, mapped) | 3.87 | 4.00 | 7.72 |
+Behind the three models built for these benchmarks by 0.2–0.6, level with STAR on video, ahead of FAN
+and MediaPipe everywhere. Profile faces are the weak spot.
+![WFLW test faces, one per condition: ours, STAR, SPIGA, RwSA, MediaPipe](images/examples_wflw.jpg)
+**The error estimate** (`std`) comes from a Student-t likelihood (ν = 10). Error rises with it, but
+the intervals are too narrow: multiply `std` by 1.34 to calibrate on average. Measured on studio
+faces only.
+## Limitations
+- Hidden points (teeth behind closed lips, the far side of a turned face) are estimated, not seen;
+  sunglasses give guessed eyelids and irises; closed eyes come out about 1 mm open.
+- The 620 points have no hand-labelled truth on wild photos; their wild accuracy is known only through
+  the 68/98 outputs.
+- On 24 held-out studio people, error correlated with darker skin (Spearman −0.44, p = 0.03),
+  confounded with exposure; Ava-256 is biased to lighter-skinned people under 35.
+## Training
+The 620 points: Ava-256 (Meta), 230 people, 16 cameras; labels are projections of each person's
+registered mesh, whose vertices were mapped onto the GNM template. The 68/98 heads: the 300W and WFLW
+training photos; on those photos the 620 points carry pseudo-labels. HRNet-W48, three stages, one
+run. To replicate the model, follow [train/README.md](train/README.md): data preparation, the three
+commands and the export. The scripts and the shipped sample lists, split and pseudo-label map:
+[prep/](prep/); every key: [train/config.py](train/config.py).
+Protocol, data and analysis in full: technical report (in preparation).
+## Licence
+Copyright 2026 Myned AI (opensource@myned.ai).
+| | licence | source |
+|---|---|---|
+| code | Apache-2.0 | `LICENSE` |
+| `model/model.safetensors`, `points_gnm.json`, `prep/*.npz` | CC BY-NC 4.0 | `LICENSE-WEIGHTS` |
+| Ava-256 · Multiface (images) | CC BY-NC 4.0 | LICENSE files |
+| 300W · WFLW (WIDER FACE images) · 300VW | research only · CC BY-NC-ND · none stated | dataset pages |
+| ImageNet start (timm `hrnet_w48.ms_in1k`) | weights MIT; ImageNet terms non-commercial | model card |
+| GNM Head | Apache-2.0, attribution required | LICENSE file |
+Figures show Multiface people (consented) and WFLW test faces as the benchmark publishes them.