| --- |
| license: other |
| license_name: upstream-model-licenses-apply |
| license_link: https://github.com/TWWinde/ReSeed/blob/main/NOTICE.md |
| library_name: diffusers |
| tags: |
| - reseed |
| - diffusion |
| - distillation |
| - text-to-image |
| - text-to-video |
| base_model: |
| - stabilityai/stable-diffusion-3-medium-diffusers |
| - stabilityai/stable-diffusion-3.5-medium |
| - black-forest-labs/FLUX.1-dev |
| - Wan-AI/Wan2.1-T2V-1.3B-Diffusers |
| --- |
| |
| # ReSeed |
|
|
| **A First-Step Bias Restores Lost Diversity in Diffusion Model Distillation** |
|
|
| [Paper](https://arxiv.org/abs/2610.37110) · [Project page](https://twwinde.github.io/reseed-project-page/) · [Code](https://github.com/TWWinde/ReSeed) |
|
|
| ReSeed improves diversity in a frozen four-step student by fitting small |
| residual biases to the teacher's first-interval velocity. The biases are |
| enabled only in the first denoising step; the original weights and final |
| three steps remain unchanged. |
|
|
| This repository provides four ReSeed adapters and two self-trained SD3 DMD |
| students. The selected adapter runs and our two student runs use training |
| seed 42. Generation seeds can be chosen independently. |
|
|
| ## Student models and their sources |
|
|
| ### SD3-Medium and SD3.5-Medium: trained by us |
|
|
| We trained these four-step students in our CRRA `unified_distill` codebase |
| from the corresponding Stability AI models. Both released students are |
| **seed 42, checkpoint-2000**. Their full transformer weights are included: |
|
|
| | Student | Released weights | Teacher | Teacher sampling reference | |
| | --- | --- | --- | --- | |
| | SD3-Medium DMD | [checkpoint-2000](https://huggingface.co/Wenwu2000/ReSeed/tree/main/students/sd3m/seed_42/checkpoint-2000) | [SD3-Medium](https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers) | 28 steps, CFG 7 | |
| | SD3.5-Medium DMD | [checkpoint-2000](https://huggingface.co/Wenwu2000/ReSeed/tree/main/students/sd35m/seed_42/checkpoint-2000) | [SD3.5-Medium](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium) | 40 steps, CFG 4.5 | |
|
|
| These are our experimental students, **not checkpoints from the official |
| DMD2 release**. They use four-step FlowMatch Euler inference at CFG 1. |
| For the underlying methods, see [DMD](https://arxiv.org/abs/2311.18828) and |
| [DMD2](https://github.com/tianweiy/DMD2). |
|
|
| The original training manifests are included beside each student. The |
| SD3-Medium manifest retains the historical objective label `tdm` and records |
| the implementation `sd_distill_classic_v1`; the SD3.5-Medium manifest records |
| `dmd`. These records have not been relabelled. The release's exact student |
| and adapter pairings are listed in [release_manifest.json](release_manifest.json). |
|
|
| ### Wan2.1 DMD2: public checkpoint from the DFD authors |
|
|
| The video student is the Wan2.1-T2V-1.3B DMD2 checkpoint released by the |
| [Data-Forcing Distillation authors](https://github.com/csy2077/data-forcing-distillation). |
| We use **`dmd2_t2v_vipe/0001000`**, trained on the ViPE `long` set: |
|
|
| - [Checkpoint files](https://huggingface.co/csusupergear/wan_t2v_checkpoints_release/tree/main/dmd2_t2v_vipe) |
| - [Teacher: Wan2.1-T2V-1.3B](https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B-Diffusers) |
| - [DFD / fastgen code and setup](https://github.com/csy2077/data-forcing-distillation) |
|
|
| We did not train this DMD2 student. We trained the ReSeed adapter on top of |
| it and provide only that adapter here. The separately released DFD |
| post-trained model (`ours_wan_t2v_vipe/0000040`) is a different baseline, |
| not the student used by this adapter. |
|
|
| ### FLUX: public SenseFlow student |
|
|
| We use the four-step **SenseFlow-FLUX** transformer from |
| [domiso/SenseFlow](https://huggingface.co/domiso/SenseFlow/tree/main/SenseFlow-FLUX), |
| with shared components from [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev). |
| This student was released by the SenseFlow authors, not trained by us. |
| Download it separately; this repository contains its ReSeed adapter only. |
|
|
| ## Released adapters |
|
|
| | Student | Adapter directory | Adapter updates | Bias parameters | |
| | --- | --- | ---: | ---: | |
| | SD3-Medium DMD | `adapters/sd3m/seed_42/checkpoint-1900` | 1900 | 72,192 | |
| | SD3.5-Medium DMD | `adapters/sd35m/seed_42/checkpoint-2100` | 2100 | 72,192 | |
| | SenseFlow-FLUX | `adapters/flux/seed_42/checkpoint-2000` | 2000 | 233,472 | |
| | Wan2.1 DMD2 | `adapters/wan21/seed_42/checkpoint-300` | 300 | 46,080 | |
|
|
| Each directory contains `residual_biases.pt` and `manifest.json`. Adapter |
| updates count ReSeed fitting, not student distillation: both included SD3 |
| students remain at update 2000. Use each adapter with its listed student. |
| These are residual-bias adapters, not LoRA weights. |
|
|
| ## Download and install |
|
|
| Use Python 3.10+ and a CUDA/BF16-capable PyTorch installation. Our image |
| runtime used PyTorch 2.6.0, Diffusers 0.34.0 and Transformers 4.45.2. |
|
|
| ```bash |
| pip install huggingface_hub==0.36.2 |
| hf download Wenwu2000/ReSeed --revision main --local-dir reseed-weights |
| |
| git clone https://github.com/TWWinde/ReSeed.git |
| git -C ReSeed checkout eed9a03bd85a0af8a7dc36e8da5b48d2a31f58ca |
| pip install -r reseed-weights/requirements.txt |
| pip install -e ReSeed/image --no-deps |
| ``` |
|
|
| The full download is about 18.22 GB, mostly the two student transformers. |
| To download only SD3-Medium, use: |
|
|
| ```bash |
| hf download Wenwu2000/ReSeed --revision main --local-dir reseed-weights \ |
| --include README.md requirements.txt inference.py release_manifest.json \ |
| "adapters/sd3m/*" "students/sd3m/*" |
| ``` |
|
|
| Use `sd35m` instead of `sd3m` for SD3.5-Medium. For FLUX or Wan, only the |
| corresponding adapter is needed from this repository. Upstream gated models |
| require accepting their terms and logging in with `hf auth login`. |
|
|
| ## SD3 image generation |
|
|
| Download the public components for the model you want to run: |
|
|
| ```bash |
| hf download stabilityai/stable-diffusion-3-medium-diffusers \ |
| --revision ea42f8cef0f178587cf766dc8129abd379c90671 --local-dir models/sd3m |
| |
| hf download stabilityai/stable-diffusion-3.5-medium \ |
| --revision b940f670f0eda2d07fbb75229e779da1ad11eb80 --local-dir models/sd35m |
| ``` |
|
|
| Generate an image with the matching DMD student and ReSeed: |
|
|
| ```bash |
| python reseed-weights/inference.py --weights reseed-weights --model sd3m \ |
| --base models/sd3m --prompt "a photo of a cat" --seed 0 \ |
| --output samples/sd3m_reseed.png |
| |
| python reseed-weights/inference.py --weights reseed-weights --model sd35m \ |
| --base models/sd35m --prompt "a photo of a cat" --seed 0 \ |
| --output samples/sd35m_reseed.png |
| ``` |
|
|
| Add `--without-reseed` and choose a different output filename to generate |
| the plain DMD baseline with the same prompt and initial noise. |
|
|
| The script loads our student transformer and the public model's text |
| encoders, tokenizers, VAE and scheduler. It uses 1024×1024 BF16 inference, |
| four Euler steps, CFG 1 and static shift 3. The pipeline receives unshifted |
| sigmas `[1, .75, .5, .25]`; do not substitute the shifted boundaries from |
| the training manifest. Initial noise is drawn in FP32 on CPU, then cast to |
| BF16. The adapter is disabled after the first step. |
|
|
| ## FLUX image generation |
|
|
| ```bash |
| hf download black-forest-labs/FLUX.1-dev \ |
| --revision 3de623fc3c33e44ffbe2bad470d0f45bccf2eb21 --local-dir models/flux-dev |
| hf download domiso/SenseFlow --revision 1bbd81546d590acf24470b39ab28f9cd918c3927 \ |
| --include "SenseFlow-FLUX/*" --local-dir models/senseflow |
| ``` |
|
|
| Create `prompts.csv`: |
|
|
| ```csv |
| case_number,evaluation_seed,prompt,prompt_id,sample_id |
| 0,0,a photo of a cat,0,0 |
| 1,1,a photo of a cat,0,1 |
| ``` |
|
|
| Then run: |
|
|
| ```bash |
| cd ReSeed/image |
| python tools/flux_generate.py --model senseflow \ |
| --dev-model ../../models/flux-dev \ |
| --senseflow-transformer ../../models/senseflow/SenseFlow-FLUX \ |
| --first-interval-residual-bias-checkpoint ../../reseed-weights/adapters/flux/seed_42/checkpoint-2000 \ |
| --prompts ../../prompts.csv --expected-count 2 --samples-per-prompt 2 \ |
| --evaluation-protocol reseed-example \ |
| --steps 4 --resolution 1024 --max-sequence-length 512 \ |
| --output ../../samples/flux_reseed |
| cd ../.. |
| ``` |
|
|
| Omit the adapter argument for the plain SenseFlow baseline. This student |
| uses its native four-step dynamically shifted Euler schedule with the |
| guidance embedding disabled. Do not pass the teacher's guidance 3.5 to the |
| student. The transformer is in `SenseFlow-FLUX`, not a nested `transformer/` folder. |
|
|
| ## Wan video generation |
|
|
| First follow the [ReSeed video setup](https://github.com/TWWinde/ReSeed/blob/eed9a03bd85a0af8a7dc36e8da5b48d2a31f58ca/video/README.md) |
| to install the video code and apply the ReSeed inference patch to DFD. |
| The video runtime has separate dependencies from the image package. |
|
|
| ```bash |
| hf download csusupergear/wan_t2v_checkpoints_release \ |
| --include "dmd2_t2v_vipe/0001000*" --local-dir wan-checkpoints |
| ``` |
|
|
| Keep the `.pth` and `.net_model.zip` files under their original names. |
| From the configured DFD checkout, use a text file with one prompt per line: |
|
|
| ```bash |
| export DFD_CKPT_ROOT=/absolute/path/to/wan-checkpoints |
| export DFD_RESEED_ADAPTER=/absolute/path/to/reseed-weights/adapters/wan21/seed_42/checkpoint-300 |
| export DFD_PROMPT_FILE=/absolute/path/to/video-prompts.txt |
| python evaluation/generate_multiseed.py --config evaluation/config.yaml \ |
| --models dmd2 dmd2_reseed --seeds 0 1 \ |
| --output-root evaluation/reseed-example |
| ``` |
|
|
| This generates matched DMD2 and DMD2 + ReSeed videos. The student uses four |
| ODE steps with boundaries `[.999, .937, .833, .624, 0]`, no student CFG, |
| 81 frames at 832×480, 16 FPS and BF16. The teacher reference uses 50 UniPC |
| steps, CFG 5 and shift 5. ReSeed is active only in the first student step. |
|
|
| ## Evaluation prompts and noise |
|
|
| The image diversity evaluation uses **100 COCO prompts × 10 noise seeds**: |
|
|
| - [100 prompts](evaluation/diversity_100x10/prompts.txt) |
| - [1,000 prompt/seed cases](evaluation/diversity_100x10/prompt_seed_cases.csv) |
| - [Noise generator](evaluation/diversity_100x10/generate_noise.py) and [protocol](evaluation/diversity_100x10/protocol.json) |
|
|
| These inputs are shared by SD3-Medium, SD3.5-Medium and SenseFlow-FLUX at |
| 1024×1024. Case `i` uses seed `2026081900 + i`. Each noise tensor has shape |
| `[16, 128, 128]`, drawn on CPU in FP32 before the model's BF16 cast or FLUX |
| packing. We publish the seeds and generator rather than 1 GB of tensor files. |
| Use PyTorch 2.6.0 to reconstruct the reference inputs: |
|
|
| ```bash |
| hf download Wenwu2000/ReSeed --revision main --local-dir reseed-weights \ |
| --include "evaluation/diversity_100x10/*" |
| |
| # Export one case; omit --case to export all 1,000 tensors (about 1 GB). |
| python reseed-weights/evaluation/diversity_100x10/generate_noise.py \ |
| --case 0 --output evaluation-noise-case0 |
| ``` |
|
|
| The export includes a SHA-256 hash of every generated FP32 tensor. This is |
| the image diversity bank, not the COCO5K quality bank or Wan video inputs. |
| The case records match the original evaluation list; only CRLF line endings |
| have been changed to LF for this release, with both hashes in the protocol. |
|
|
| ## Files and licenses |
|
|
| The release contains inference weights, not optimizer state, fake-score |
| models, public teacher weights or generated samples. Text encoders |
| and VAEs come from the linked base models. File hashes, pinned source |
| revisions and checkpoint pairings are recorded in `release_manifest.json`; |
| its branch and visibility fields describe the original private release. |
| The repository is now public, with current instructions on `main` and the |
| original `single-seed` branch retained as an archive. |
|
|
| The ReSeed code is MIT-licensed. Model weights retain their upstream |
| licenses and access conditions; see [NOTICE](https://github.com/TWWinde/ReSeed/blob/main/NOTICE.md). |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{tang2026reseed, |
| title={A First-Step Bias Restores Lost Diversity in Diffusion Model Distillation}, |
| author={Wenwu Tang and Dong Wang and Yun Cheng and Olga Saukh}, |
| year={2026}, |
| eprint={2610.37110}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.CV}, |
| url={https://arxiv.org/abs/2610.37110} |
| } |
| ``` |
|
|
