|
Download README.md from Digital-Dermatology/SkinMap-Factory: direct link, hf CLI and curl.
- Browser
- Download file 3.2 kB
-
https://huggingface.co/Digital-Dermatology/SkinMap-Factory/resolve/main/README.md
- Command line
-
hf download hf://Digital-Dermatology/SkinMap-Factory/README.md
-
curl -L -H "Authorization: Bearer $HF_TOKEN" -o README.md https://huggingface.co/Digital-Dermatology/SkinMap-Factory/resolve/main/README.md
3.2 kB
| license: cc-by-nc-4.0 | |
| library_name: pytorch | |
| tags: | |
| - dermatology | |
| - medical-imaging | |
| - image-generation | |
| - diffusion | |
| - rectified-flow | |
| pipeline_tag: image-to-image | |
| extra_gated_prompt: >- | |
| This model generates synthetic skin images. It is for research only and must not be | |
| used for diagnosis or patient care. Access is approved by the Digital Dermatology | |
| maintainers. | |
| extra_gated_fields: | |
| Name: text | |
| Affiliation: text | |
| Intended use: text | |
| I will use this model for non-commercial research only: checkbox | |
| # SkinMap Factory | |
| SkinMap Factory renders skin images from [SkinMap](https://huggingface.co/Digital-Dermatology/SkinMap) embeddings and clinical metadata. Code, training and editing methods: [Digital-Dermatology/skinmap-factory](https://github.com/Digital-Dermatology/skinmap-factory). | |
| | | | | |
| |---|---| | |
| | File | `skinmap_factory_step2195000.pt` (EMA weights, codec and config) | | |
| | SHA-256 | `3559b8f401159f02ddf021e5dad43ce411d116d8fef81529c34667360a4f86a3` | | |
| | Model | PixelDiT, 729M parameters, pixel space, 16 x 16 patches | | |
| | Objective | Rectified flow, 2,195,000 steps, REPA alignment to DINOv2 | | |
| | Output | 256 x 256 RGB | | |
| | Conditions | 1024-d SkinMap embedding, plus `modality`, `body_region`, `fitzpatrick`, `gender`, `age`, `continent`, `icd_category` | | |
| | Sampling defaults | 100 steps, guidance 1.5 | | |
| ## Access | |
| The repo is gated. Request access above, wait for approval, then log in with `hf auth login` (or set `HF_TOKEN`) on each machine that downloads the model. | |
| ## Usage | |
| ```sh | |
| pip install git+https://github.com/Digital-Dermatology/skinmap-factory.git | |
| ``` | |
| ```python | |
| from transformers import AutoModel | |
| from torchvision.utils import save_image | |
| from skinmap_factory import generate, load_decoder | |
| encoder = AutoModel.from_pretrained("Digital-Dermatology/SkinMap", trust_remote_code=True, device="cuda") | |
| z = encoder.encode_image("lesion.jpg") # (1024,) SkinMap embedding | |
| decoder, codec, config = load_decoder(device="cuda") # downloads this repo once | |
| images = generate(decoder, codec, config, [z] * 4, | |
| metadata={"modality": "dermoscopy", "body_region": "trunk_back"}, | |
| seed=0) # (4, 3, 256, 256) in [0, 1] | |
| save_image(images, "samples.png") # four samples of one embedding | |
| ``` | |
| The embedding sets most of the image, and metadata is optional: fields left out count as unknown. `codec.vocab` lists the allowed values of each field. A batch of six images peaks at about 6 GB of GPU memory. From the command line: | |
| ```sh | |
| skinmap-factory-sample --z-npy embeddings.npy --output samples/ | |
| ``` | |
| The [repo README](https://github.com/Digital-Dermatology/skinmap-factory#released-model) documents the metadata values and options. | |
| ## Limits | |
| The model learned from public dermatology datasets, and it inherits their gaps: few images of darker skin (Fitzpatrick V and VI), few children, and few images of rare diseases. Generated images are synthetic. Do not use them as clinical evidence or present them as real patients. | |
| ## License | |
| Weights: CC BY-NC 4.0, because several training datasets allow only non-commercial use. Code: Apache-2.0. | |