File size: 4,649 Bytes
132d60d 1cc0afc 132d60d 5e737f2 1cc0afc 132d60d 5e737f2 1cc0afc 5e737f2 1cc0afc 5e737f2 132d60d 5e737f2 132d60d 5e737f2 132d60d 1cc0afc 132d60d 1cc0afc 132d60d 5c06799 132d60d 1cc0afc 132d60d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 | ---
license: mit
pipeline_tag: image-to-3d
tags:
- garment-particles
- 3d-garments
- sewing-patterns
- diffusion
- flow-matching
- fsdp2
---
# Garment Particles (Realistic Image Fine-Tuned Checkpoints)
Official fine-tuned checkpoints for **Garment Particles**, adapted for realistic and rendered image conditioning.
- **Paper**: [Garment Particles: A 2D--3D Symmetric Garment Representation for Generation and Editing](https://huggingface.co/papers/2605.26391)
- **Project Page**: [https://garment-particles.github.io](https://garment-particles.github.io)
- **Code**: [https://github.com/garment-particles/GarmentParticles](https://github.com/garment-particles/GarmentParticles)
- **Base Models**: [georgeNakayama/GarmentParticles](https://huggingface.co/georgeNakayama/GarmentParticles)
---
## Overview
This repository hosts fine-tuned **Stage 1 (PGF)** checkpoints across multiple training paradigms:
1. **`from_text_baseline/`**: Trained directly from the `pgf_text` checkpoint on realistic images without prior synthetic image cross-attention.
2. **`pgf_image_realistic_step*/`**: Fine-tuned from the pretrained `pgf_image` baseline.
3. **`edge/`**: Pretrained Stage 2 Edge Model for 2D sewing pattern reconstruction.
4. **`test_images/`**: 200 evaluation garment test images (`eval_set_40_prompt5`).
---
## Checkpoints Summary
### 1. From-Text Baseline (`from_text_baseline/`)
*Trained from `pgf_text` using 46,119 realistic GPT Image 2 complete outfit renders across 16 × H100 GPUs.*
| Checkpoint Directory | Steps | Training Description | Size |
|---|---|---|---|
| `from_text_baseline/pgf_image_realistic_step1000/` | 1,000 | Early vision-text cross-attention alignment from text base | 15 GB |
| `from_text_baseline/pgf_image_realistic_step2000/` | 2,000 | Intermediate alignment & geometry adaptation from text base | 15 GB |
### 2. Fine-Tuned Checkpoints (From `pgf_image`)
| Checkpoint Directory | Steps | Val Loss | Training Phase & Recommended Usage | Size |
|---|---|---|---|---|
| `pgf_image_realistic_step5000/` | 5,000 | 0.8781 | **Early Stage:** Initial domain adaptation from synthetic renders | 15 GB |
| `pgf_image_realistic_step10000/` | 10,000 | 0.7472 | **Early-Mid:** Rapid feature alignment & general silhouette formation | 15 GB |
| `pgf_image_realistic_step15000/` | 15,000 | 0.6867 | **Mid Stage:** Balanced generation before fine pattern specialization | 15 GB |
| `pgf_image_realistic_step20000/` | 20,000 | 0.6466 | **High Diversity:** Strong realistic feature capture, diverse variations | 15 GB |
| `pgf_image_realistic_step25000/` | 25,000 | 0.6255 | **Late Stage:** High geometric consistency and detailed seams | 15 GB |
| `pgf_image_realistic_step30000/` | 30,000 | 0.6150 | **Near-Convergence:** Crisp geometric shapes and panel alignments | 15 GB |
| `pgf_image_realistic_step35000/` | 35,000 | 0.6131 | **Fully Converged:** Final plateaued checkpoint (-30.2% loss reduction) | 15 GB |
| `edge/` | - | - | **Stage 2:** Pretrained Edge Model for 2D pattern reconstruction | 8.8 GB |
---
## Evaluation Test Images
- **`test_images/`**: 200 real-world & diverse evaluation garment images (`eval_set_40_prompt5`) spanning multiple fabric textures, silhouettes, and draping behaviors.
---
## Quickstart & Inference
Download a specific checkpoint:
```bash
# Example: Download step 1000 from from_text_baseline along with edge model and test images
hf download image2garment/GarmentParticles-Realistic --include "from_text_baseline/pgf_image_realistic_step1000/*" "edge/*" "test_images/*" --local-dir checkpoints_hub/realistic
```
Run two-stage image-conditioned inference:
```bash
torchrun --standalone --nproc_per_node=1 inference/infer_twostage.py \
eval.sample_per_batch=1 eval.n_samples=0 eval.evaluate=False \
train.exp_name=realistic_img_samples sample.num_sampling_steps=100 \
gpf_ckpt=null \
dataset.front_only=True dataset.use_all_captions=True \
dataset.img_drop_prob=0 dataset.text_drop_prob=1 \
model.use_qknorm=True model.use_rope=False model.in_channels=6 model.freeze_everything=False \
edge_model.use_qknorm=True \
edge_model_ckpt=checkpoints_hub/realistic/edge \
model=sparse_lightningdit_v3_xl1_w_img_text_v2 \
pgf_weight_init=checkpoints_hub/realistic/from_text_baseline/pgf_image_realistic_step1000 \
--config-name sparselightningdit_xl_garment_particle_inference
```
---
## Citation
```bibtex
@inproceedings{garmentparticles2026,
title={Garment Particles: A 2D--3D Symmetric Garment Representation for Generation and Editing},
author={George Nakayama and others},
booktitle={SIGGRAPH Conference Papers},
year={2026}
}
```
|