Add comprehensive model card for Realistic Image GarmentParticles

README.md

@@ -0,0 +1,110 @@
+---
+license: mit
+pipeline_tag: image-to-3d
+tags:
+  - garment-particles
+  - 3d-garments
+  - sewing-patterns
+  - diffusion
+  - flow-matching
+  - fsdp2
+---
+
+# Garment Particles (Realistic Image Fine-Tuned Checkpoints)
+
+Official fine-tuned checkpoints for **Garment Particles**, adapted for realistic and rendered image conditioning.
+
+- **Paper**: [Garment Particles: A 2D--3D Symmetric Garment Representation for Generation and Editing](https://huggingface.co/papers/2605.26391)
+- **Project Page**: [https://garment-particles.github.io](https://garment-particles.github.io)
+- **Code**: [https://github.com/garment-particles/GarmentParticles](https://github.com/garment-particles/GarmentParticles)
+- **Base Models**: [georgeNakayama/GarmentParticles](https://huggingface.co/georgeNakayama/GarmentParticles)
+
+---
+
+## Overview
+
+This repository hosts the **realistic image-conditioned Stage 1 (PGF)** fine-tuned checkpoints alongside the shared **Stage 2 Edge Model**.
+
+The Stage 1 model was fine-tuned for 35,000 steps across 16 × NVIDIA H100 GPUs using PyTorch FSDP2 on 46,119 realistic garment outfits generated with GPT Image 2 from GarmentCodeData-v2.
+
+### Validation Loss Trajectory
+
+| Checkpoint | Steps | Validation Loss | Notes |
+|---|---|---|---|
+| Step 5,000 | 5k | 0.8781 | Initial adaptation |
+| Step 10,000 | 10k | 0.7472 | Rapid feature alignment |
+| Step 15,000 | 15k | 0.6867 | Stable generation |
+| Step 20,000 | 20k | 0.6466 | High geometric detail |
+| Step 25,000 | 25k | 0.6255 | Fine-grain pattern refinement |
+| **Step 30,000** | 30k | **0.6150** | Near-convergence milestone |
+| **Step 35,000** | 35k | **0.6131** | **Final fully converged checkpoint (-30.2% loss reduction)** |
+
+---
+
+## Checkpoints Included
+
+| Checkpoint Directory | Description | Size |
+|---|---|---|
+| `pgf_image_realistic_step35000/` | **Recommended:** Fully converged Stage 1 PGF flow matching model | 15 GB (16 shards) |
+| `pgf_image_realistic_step30000/` | Step 30,000 Stage 1 checkpoint | 15 GB (16 shards) |
+| `edge/` | Pretrained Stage 2 Edge & Sewing Pattern Reconstruction Model | 8.8 GB (8 shards) |
+
+---
+
+## Quickstart & Inference
+
+Clone the [GarmentParticles repository](https://github.com/garment-particles/GarmentParticles):
+
+```bash
+git clone https://github.com/garment-particles/GarmentParticles.git
+cd GarmentParticles
+
+# Install dependencies
+pip install torch torchvision
+pip install -r src/requirements.txt
+pip install flash-attn --no-build-isolation
+export PYTHONPATH=$PWD/src:$PYTHONPATH
+```
+
+Download the checkpoints:
+
+```bash
+hf download image2garment/GarmentParticles-Realistic --local-dir src/checkpoints/realistic
+```
+
+Run two-stage image-conditioned inference:
+
+```bash
+torchrun --standalone --nproc_per_node=1 inference/infer_twostage.py \
+  eval.sample_per_batch=1 eval.n_samples=0 eval.evaluate=False \
+  train.exp_name=realistic_img_samples sample.num_sampling_steps=100 \
+  gpf_ckpt=null \
+  dataset.front_only=True dataset.use_all_captions=True \
+  dataset.img_drop_prob=0 dataset.text_drop_prob=1 \
+  model.use_qknorm=True model.use_rope=False model.in_channels=6 model.freeze_everything=False \
+  edge_model.use_qknorm=True \
+  edge_model_ckpt=checkpoints/realistic/edge \
+  model=sparse_lightningdit_v3_xl1_w_img_text_v2 \
+  pgf_weight_init=checkpoints/realistic/pgf_image_realistic_step35000 \
+  --config-name sparselightningdit_xl_garment_particle_inference
+```
+
+---
+
+## Architecture Specifications
+
+- **Stage 1 (PGF):** `SparseLightningDiTV3CrossAttnVarlenImgTextV2` (28 DiT layers, 1152 hidden dimension, 16 attention heads, QK-Norm, DINOv2-Large image encoder + CLIP text conditioning, velocity flow matching).
+- **Stage 2 (Edge Model):** `LightningCrossAttnDiTV3EdgeModelVarlen` (24 DiT layers, 1024 hidden dimension, 16 attention heads, RoPE, RMSNorm, SwiGLU, FlashAttention-2, 37 panels, 38 curves).
+
+---
+
+## Citation
+
+```bibtex
+@inproceedings{garmentparticles2026,
+  title={Garment Particles: A 2D--3D Symmetric Garment Representation for Generation and Editing},
+  author={George Nakayama and others},
+  booktitle={SIGGRAPH Conference Papers},
+  year={2026}
+}
+```