Download README.md from Digital-Dermatology/SkinMap-Factory: direct link, hf CLI and curl.
- Browser
- Download file 3.2 kB
-
https://huggingface.co/Digital-Dermatology/SkinMap-Factory/resolve/main/README.md
- Command line
-
hf download hf://Digital-Dermatology/SkinMap-Factory/README.md
-
curl -L -H "Authorization: Bearer $HF_TOKEN" -o README.md https://huggingface.co/Digital-Dermatology/SkinMap-Factory/resolve/main/README.md
license: cc-by-nc-4.0
library_name: pytorch
tags:
- dermatology
- medical-imaging
- image-generation
- diffusion
- rectified-flow
pipeline_tag: image-to-image
extra_gated_prompt: >-
This model generates synthetic skin images. It is for research only and must
not be used for diagnosis or patient care. Access is approved by the Digital
Dermatology maintainers.
extra_gated_fields:
Name: text
Affiliation: text
Intended use: text
I will use this model for non-commercial research only: checkbox
SkinMap Factory
SkinMap Factory renders skin images from SkinMap embeddings and clinical metadata. Code, training and editing methods: Digital-Dermatology/skinmap-factory.
| File | skinmap_factory_step2195000.pt (EMA weights, codec and config) |
| SHA-256 | 3559b8f401159f02ddf021e5dad43ce411d116d8fef81529c34667360a4f86a3 |
| Model | PixelDiT, 729M parameters, pixel space, 16 x 16 patches |
| Objective | Rectified flow, 2,195,000 steps, REPA alignment to DINOv2 |
| Output | 256 x 256 RGB |
| Conditions | 1024-d SkinMap embedding, plus modality, body_region, fitzpatrick, gender, age, continent, icd_category |
| Sampling defaults | 100 steps, guidance 1.5 |
Access
The repo is gated. Request access above, wait for approval, then log in with hf auth login (or set HF_TOKEN) on each machine that downloads the model.
Usage
pip install git+https://github.com/Digital-Dermatology/skinmap-factory.git
from transformers import AutoModel
from torchvision.utils import save_image
from skinmap_factory import generate, load_decoder
encoder = AutoModel.from_pretrained("Digital-Dermatology/SkinMap", trust_remote_code=True, device="cuda")
z = encoder.encode_image("lesion.jpg") # (1024,) SkinMap embedding
decoder, codec, config = load_decoder(device="cuda") # downloads this repo once
images = generate(decoder, codec, config, [z] * 4,
metadata={"modality": "dermoscopy", "body_region": "trunk_back"},
seed=0) # (4, 3, 256, 256) in [0, 1]
save_image(images, "samples.png") # four samples of one embedding
The embedding sets most of the image, and metadata is optional: fields left out count as unknown. codec.vocab lists the allowed values of each field. A batch of six images peaks at about 6 GB of GPU memory. From the command line:
skinmap-factory-sample --z-npy embeddings.npy --output samples/
The repo README documents the metadata values and options.
Limits
The model learned from public dermatology datasets, and it inherits their gaps: few images of darker skin (Fitzpatrick V and VI), few children, and few images of rare diseases. Generated images are synthetic. Do not use them as clinical evidence or present them as real patients.
License
Weights: CC BY-NC 4.0, because several training datasets allow only non-commercial use. Code: Apache-2.0.