README.md

---
license: mit
pipeline_tag: image-to-3d
tags:
  - garment-particles
  - 3d-garments
  - sewing-patterns
  - diffusion
  - flow-matching
  - fsdp2
---

# Garment Particles (Realistic Image Fine-Tuned Checkpoints)

Official fine-tuned checkpoints for **Garment Particles**, adapted for realistic and rendered image conditioning.

- **Paper**: [Garment Particles: A 2D--3D Symmetric Garment Representation for Generation and Editing](https://huggingface.co/papers/2605.26391)
- **Project Page**: [https://garment-particles.github.io](https://garment-particles.github.io)
- **Code**: [https://github.com/garment-particles/GarmentParticles](https://github.com/garment-particles/GarmentParticles)
- **Base Models**: [georgeNakayama/GarmentParticles](https://huggingface.co/georgeNakayama/GarmentParticles)

---

## Overview

This repository hosts fine-tuned **Stage 1 (PGF)** checkpoints across multiple training paradigms:
1. **`from_text_baseline/`**: Trained directly from the `pgf_text` checkpoint on realistic images without prior synthetic image cross-attention.
2. **`pgf_image_realistic_step*/`**: Fine-tuned from the pretrained `pgf_image` baseline.
3. **`edge/`**: Pretrained Stage 2 Edge Model for 2D sewing pattern reconstruction.
4. **`test_images/`**: 200 evaluation garment test images (`eval_set_40_prompt5`).

---

## Checkpoints Summary

### 1. From-Text Baseline (`from_text_baseline/`)
*Trained from `pgf_text` using 46,119 realistic GPT Image 2 complete outfit renders across 16 × H100 GPUs.*

| Checkpoint Directory | Steps | Training Description | Size |
|---|---|---|---|
| `from_text_baseline/pgf_image_realistic_step1000/` | 1,000 | Early vision-text cross-attention alignment from text base | 15 GB |
| `from_text_baseline/pgf_image_realistic_step2000/` | 2,000 | Intermediate alignment & geometry adaptation from text base | 15 GB |

### 2. Fine-Tuned Checkpoints (From `pgf_image`)

| Checkpoint Directory | Steps | Val Loss | Training Phase & Recommended Usage | Size |
|---|---|---|---|---|
| `pgf_image_realistic_step5000/` | 5,000 | 0.8781 | **Early Stage:** Initial domain adaptation from synthetic renders | 15 GB |
| `pgf_image_realistic_step10000/` | 10,000 | 0.7472 | **Early-Mid:** Rapid feature alignment & general silhouette formation | 15 GB |
| `pgf_image_realistic_step15000/` | 15,000 | 0.6867 | **Mid Stage:** Balanced generation before fine pattern specialization | 15 GB |
| `pgf_image_realistic_step20000/` | 20,000 | 0.6466 | **High Diversity:** Strong realistic feature capture, diverse variations | 15 GB |
| `pgf_image_realistic_step25000/` | 25,000 | 0.6255 | **Late Stage:** High geometric consistency and detailed seams | 15 GB |
| `pgf_image_realistic_step30000/` | 30,000 | 0.6150 | **Near-Convergence:** Crisp geometric shapes and panel alignments | 15 GB |
| `pgf_image_realistic_step35000/` | 35,000 | 0.6131 | **Fully Converged:** Final plateaued checkpoint (-30.2% loss reduction) | 15 GB |
| `edge/` | - | - | **Stage 2:** Pretrained Edge Model for 2D pattern reconstruction | 8.8 GB |

---

## Evaluation Test Images

- **`test_images/`**: 200 real-world & diverse evaluation garment images (`eval_set_40_prompt5`) spanning multiple fabric textures, silhouettes, and draping behaviors.

---

## Quickstart & Inference

Download a specific checkpoint:

```bash
# Example: Download step 1000 from from_text_baseline along with edge model and test images
hf download image2garment/GarmentParticles-Realistic   --include "from_text_baseline/pgf_image_realistic_step1000/*" "edge/*" "test_images/*"   --local-dir checkpoints_hub/realistic
```

Run two-stage image-conditioned inference:

```bash
torchrun --standalone --nproc_per_node=1 inference/infer_twostage.py \
  eval.sample_per_batch=1 eval.n_samples=0 eval.evaluate=False \
  train.exp_name=realistic_img_samples sample.num_sampling_steps=100 \
  gpf_ckpt=null \
  dataset.front_only=True dataset.use_all_captions=True \
  dataset.img_drop_prob=0 dataset.text_drop_prob=1 \
  model.use_qknorm=True model.use_rope=False model.in_channels=6 model.freeze_everything=False \
  edge_model.use_qknorm=True \
  edge_model_ckpt=checkpoints_hub/realistic/edge \
  model=sparse_lightningdit_v3_xl1_w_img_text_v2 \
  pgf_weight_init=checkpoints_hub/realistic/from_text_baseline/pgf_image_realistic_step1000 \
  --config-name sparselightningdit_xl_garment_particle_inference
```

---

## Citation

```bibtex
@inproceedings{garmentparticles2026,
  title={Garment Particles: A 2D--3D Symmetric Garment Representation for Generation and Editing},
  author={George Nakayama and others},
  booktitle={SIGGRAPH Conference Papers},
  year={2026}
}
```