File size: 4,445 Bytes
58d3873 2c29cd1 58d3873 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 | ---
license: other
license_name: sapiens2-license
license_link: https://github.com/facebookresearch/sapiens2/blob/main/LICENSE.md
pipeline_tag: image-feature-extraction
library_name: timm
tags:
- sapiens
- sapiens2
- vision-transformer
- human-centric
- pretrained-backbone
- feature-extraction
- transformers
- timm
base_model: facebook/sapiens2-pretrain-0.1b
---
> **NOTE:** This is a native **timm (EVA) remap** of [facebook/sapiens2-pretrain-0.1b](https://huggingface.co/facebook/sapiens2-pretrain-0.1b/tree/b16fc30feddc2cf9330b2980a26a8cc91f51e697). Checkpoint keys have been converted to timm naming; the weights have not been fine-tuned. The original [Sapiens2 License](LICENSE.md) applies. The upstream model card is reproduced below with timm usage instructions.
# Sapiens2-0.1B
Sapiens2 is a family of high-resolution vision transformers pretrained on **1 billion human images** — designed for human-centric tasks such as pose estimation, body-part segmentation, surface normals, and pointmaps.
This repository contains the **0.1B parameter pretrained backbone**. It produces dense per-patch features suitable for fine-tuning downstream task heads.
- 📄 **Paper:** [arXiv:2604.21681](https://arxiv.org/pdf/2604.21681)
- 🌐 **Project Page:** [rawalkhirodkar.github.io/sapiens2](https://rawalkhirodkar.github.io/sapiens2)
- 💻 **Code:** [github.com/facebookresearch/sapiens2](https://github.com/facebookresearch/sapiens2)
## Model Details
- **Developed by:** Meta
- **Model type:** Vision Transformer
- **License:** [Sapiens2 License](https://github.com/facebookresearch/sapiens2/blob/main/LICENSE.md)
- **Task:** pretrain
- **Format:** safetensors
- **File:** `model.safetensors`
## Quick Start
Use a timm version that includes Sapiens2 support.
```python
import torch
import timm
from PIL import Image
device = "cuda" if torch.cuda.is_available() else "cpu"
model = timm.create_model(
"hf-hub:timm/vit_base_patch16_sapiens2.fb", pretrained=True, use_naflex=False,
).eval().to(device)
data_config = timm.data.resolve_model_data_config(model)
transform = timm.data.create_transform(**data_config, is_training=False)
image = Image.open("image.jpg").convert("RGB")
x = transform(image).unsqueeze(0).to(device)
with torch.inference_mode():
tokens = model.forward_features(x)
cls_features = tokens[:, 0]
patch_features = tokens[:, model.num_prefix_tokens:] # exclude CLS and register tokens
```
`model(x)` uses CLS-token pooling by default, matching the original Sapiens2 convention.
Pass `global_pool="avg"` to `create_model` for average pooling over patch tokens.
## Model Card
| Field | Value |
|-------|-------|
| Architecture | Sapiens2 ViT (RoPE, GQA, SwiGLU, RMSNorm, QK-norm) |
| Parameters | 0.114 B |
| FLOPs | 0.342 T |
| Embedding dim | 768 |
| Layers | 12 |
| Attention heads | 12 |
| Pretraining resolution | 1024 × 768 (H × W) |
| Patch size | 16 |
| Pretraining data | 1B human images |
### Sapiens2 Family
| Model | Params | FLOPs | Embed dim | Layers | Heads |
|-------|--------|-------|-----------|--------|-------|
| **Sapiens2-0.1B** *(this)* | 0.114 B | 0.342 T | 768 | 12 | 12 |
| [Sapiens2-0.4B](https://huggingface.co/facebook/sapiens2-pretrain-0.4b) | 0.398 B | 1.260 T | 1024 | 24 | 16 |
| [Sapiens2-0.8B](https://huggingface.co/facebook/sapiens2-pretrain-0.8b) | 0.818 B | 2.592 T | 1280 | 32 | 16 |
| [Sapiens2-1B](https://huggingface.co/facebook/sapiens2-pretrain-1b) | 1.462 B | 4.715 T | 1536 | 40 | 24 |
| [Sapiens2-1B-4K](https://huggingface.co/facebook/sapiens2-pretrain-1b-4k) | 1.607 B | — | 1536 | 40 | 24 |
| [Sapiens2-5B](https://huggingface.co/facebook/sapiens2-pretrain-5b) | 5.071 B | 15.722 T | 2432 | 56 | 32 |
See the [Sapiens2 Collection](https://huggingface.co/collections/facebook/sapiens2) for all variants and downstream task checkpoints (pose, segmentation, normals, pointmaps).
## Intended Use
- Feature extraction for human-centric downstream tasks
- Initialization for fine-tuning task heads (pose, segmentation, normals, pointmap)
- Research on human-centric vision
## License
Released under the [Sapiens2 License](https://github.com/facebookresearch/sapiens2/blob/main/LICENSE.md).
## Citation
```bibtex
@article{khirodkarsapiens2,
title={Sapiens2},
author={Khirodkar, Rawal and Wen, He and Martinez, Julieta and Dong, Yuan and Su, Zhaoen and Saito, Shunsuke},
journal={arXiv preprint arXiv:2604.21681},
year={2026}
}
```
|