---
license: other
license_name: research-only
license_link: https://github.com/zenonLEE/MDFLD#terms-of-use
library_name: pytorch
pipeline_tag: keypoint-detection
tags:
  - face-landmark-detection
  - cartoon-face-landmark-detection
  - facial-landmarks
  - 68-landmarks
  - cartoon
  - caricature
  - eyelid
  - torchscript
---

# MDFLD v2: cartoon face landmark detection (68 points)

Model weights for **Towards Multi-Domain Face Landmark Detection with Synthetic Data from Diffusion Model** (Yuanming Li, Gwantae Kim, Jeong-gi Kwak, Bon-hwa Ku, Hanseok Ko; ICASSP 2024).

[Paper (arXiv:2401.13191)](https://arxiv.org/abs/2401.13191) · [Code](https://github.com/zenonLEE/MDFLD) · [Project page](https://zenonlee.github.io/MDFLD/)

![68 landmarks and eyelid curves on a photo, paintings, a caricature and 3D animation](teaser.jpg)

This is the cartoon model v2: the paper model fine-tuned with self-training on cartoon and caricature faces. On the CariFace caricature test set (1,560 caricatures) it lowers the 68-point NME from 6.70% to 5.03% (eyes 3.49% → 2.78%, jaw 10.42% → 7.75%) with the package's three-pass crop; on held-out real-face video frames it reaches 1.74% (v1: 3.03%). See the [GitHub README](https://github.com/zenonLEE/MDFLD#cartoon-model-v2) for the full table, the training data and the limitations.

## Use

The easiest way is the `mdfld` package, which downloads this model, finds the face and adds eyelid curves:

```bash
pip install "mdfld[sam] @ git+https://github.com/zenonLEE/MDFLD"
mdfld image face.png --out face.json --vis face_mdfld.png
```

The file `mdfld_cartoon_v2.torchscript.pt` can also be used directly with PyTorch, without any code from the repository:

```python
import torch
net = torch.jit.load("mdfld_cartoon_v2.torchscript.pt", map_location="cuda")
# x: (N, 3, 256, 256) float tensor of BGR face crops, values pixel / 255 * 2 - 1
out = net(x)                        # (N, 68, 2), x and y in [-1, 1]
pts = ((out + 1) * 256 - 1) / 2     # pixels of the 256 x 256 crop
```

The crop is a square centred on the face box with side equal to the larger side of the box, resized to 256 × 256. For the best accuracy, run two more passes, each on a square of 1.1 × the larger side of the box of the previous points, as the package does (`mdfld/landmarks.py`). The 68 points follow the iBUG 300-W order (0-based).

## Files

| File | Use |
|---|---|
| `mdfld_cartoon_v2.torchscript.pt` | TorchScript model, used by the `mdfld` package |

The same weights in the original checkpoint format (`mdfld_cartoon_v2.pkl`, for the STAR-based code in the repository) are in the [v2.0 release](https://github.com/zenonLEE/MDFLD/releases/tag/v2.0).

## Terms of use

Research use only, without warranty. The model was trained on third-party data with their own terms (CariFace is for research only; several cartoon sets contain third-party artwork). See the [terms of use](https://github.com/zenonLEE/MDFLD#terms-of-use).

## Citation

```bibtex
@inproceedings{li2024towards,
  title={Towards Multi-Domain Face Landmark Detection with Synthetic Data from Diffusion Model},
  author={Li, Yuanming and Kim, Gwantae and Kwak, Jeong-gi and Ku, Bon-hwa and Ko, Hanseok},
  booktitle={ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages={6730--6734},
  year={2024},
  organization={IEEE}
}
```

Teaser images: public-domain works from Wikimedia Commons and frames of *Sintel* (© copyright Blender Foundation | durian.blender.org, CC BY 3.0).