Text-to-Image
Diffusers
Safetensors
reseed
diffusion
distillation
text-to-video
Wenwu2000 commited on
Commit
611ba9b
·
verified ·
1 Parent(s): 51133c2

Simplify README and clarify DMD checkpoint sources

Browse files
Files changed (1) hide show
  1. README.md +113 -379
README.md CHANGED
@@ -16,170 +16,89 @@ base_model:
16
  - Wan-AI/Wan2.1-T2V-1.3B-Diffusers
17
  ---
18
 
19
- # ReSeed — single-seed model weights
20
 
21
  **A First-Step Bias Restores Lost Diversity in Diffusion Model Distillation**
22
 
23
  [Paper](https://arxiv.org/abs/2610.37110) · [Project page](https://twwinde.github.io/reseed-project-page/) · [Code](https://github.com/TWWinde/ReSeed)
24
 
25
- This branch contains four ReSeed bias adapters and the two self-trained DMD
26
- students needed to use the SD3 adapters. It is a **private, single-seed
27
- release**: every selected training run uses seed **42**. Use the
28
- **`main` revision** when downloading. The original `single-seed` branch is
29
- retained as a release snapshot.
30
-
31
- The SD3-Medium and SD3.5-Medium students were trained by the ReSeed authors.
32
- The FLUX and Wan students come from public releases by their respective
33
- authors. Public teacher models and shared components are referenced below,
34
- not copied into this repository.
35
-
36
- ## Contents
37
-
38
- - [What ReSeed changes](#what-reseed-changes)
39
- - [Released checkpoints and model sources](#released-checkpoints-and-model-sources)
40
- - [Repository layout](#repository-layout)
41
- - [Environment and authentication](#environment-and-authentication)
42
- - [Download the release](#download-the-release)
43
- - [SD3-Medium and SD3.5-Medium inference](#sd3-medium-and-sd35-medium-inference)
44
- - [SenseFlow-FLUX inference](#senseflow-flux-inference)
45
- - [Wan2.1 text-to-video inference](#wan21-text-to-video-inference)
46
- - [Reproducibility and file verification](#reproducibility-and-file-verification)
47
- - [Troubleshooting](#troubleshooting)
48
- - [Limitations and licensing](#limitations-and-licensing)
49
- - [Citation](#citation)
50
-
51
- ## What ReSeed changes
52
-
53
- ReSeed trains small, per-channel residual biases inside a frozen distilled
54
- student. At inference, the biases are active **only during the first of four
55
- denoising steps**. The original student weights remain unchanged, and the
56
- final three steps run without the biases. No teacher forward pass is needed
57
- when generating with the trained adapter.
58
-
59
- During adapter training, the student's first-step velocity is matched to
60
- the teacher's average velocity over the same time interval. This is a
61
- post-distillation correction; it does not repeat student distillation.
62
-
63
- The adapter is **not** a LoRA and is **not** a complete Diffusers pipeline.
64
- It must be loaded using the ReSeed adapter code and applied to the exact
65
- student it was fitted on. Do not load it with `load_lora_weights`, apply it
66
- to the teacher, or leave it active for all four steps.
67
-
68
- ## Released checkpoints and model sources
69
-
70
- ### Weights included in this branch
71
-
72
- | Model | Adapter training seed | Adapter update | Bias parameters | Matching student |
73
- | --- | ---: | ---: | ---: | --- |
74
- | SD3-Medium | 42 | **1900** | 72,192 | Our DMD, training seed 42, update 2000; included |
75
- | SD3.5-Medium | 42 | **2100** | 72,192 | Our DMD, training seed 42, update 2000; included |
76
- | SenseFlow-FLUX | 42 | **2000** | 233,472 | Public SenseFlow-FLUX; download separately |
77
- | Wan2.1-T2V-1.3B | 42 | **300** | 46,080 | Public Wan DMD2, iteration 1000; download separately |
78
-
79
- Adapter updates and student updates are different counters. For example,
80
- the SD3.5 adapter at update 2100 must be used with the included student at
81
- update 2000, not with an unrelated student checkpoint named `2100`.
82
-
83
- Three different seeds may appear in a workflow:
84
-
85
- 1. **Student training seed:** selects our DMD student run; it is 42 here.
86
- 2. **Adapter training seed:** selects the ReSeed fitting run; it is 42 here.
87
- 3. **Generation seed:** selects the initial noise for an image or video.
88
- This can be any supported seed, such as 0 or 1; it need not be 42.
89
-
90
- These are individual checkpoints, not averaged weights. One selected run
91
- does not by itself reproduce a three-training-seed aggregate in the paper.
92
- GAD and DP-DMD student weights are not included in this release.
93
-
94
- ### Public teachers and external students
95
-
96
- | Component | Original source | Revision or checkpoint |
97
- | --- | --- | --- |
98
- | SD3-Medium teacher and shared components | [stabilityai/stable-diffusion-3-medium-diffusers](https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers) | `ea42f8cef0f178587cf766dc8129abd379c90671` |
99
- | SD3.5-Medium teacher and shared components | [stabilityai/stable-diffusion-3.5-medium](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium) | `b940f670f0eda2d07fbb75229e779da1ad11eb80` |
100
- | FLUX teacher and shared components | [black-forest-labs/FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) | `3de623fc3c33e44ffbe2bad470d0f45bccf2eb21` |
101
- | SenseFlow-FLUX student transformer | [domiso/SenseFlow](https://huggingface.co/domiso/SenseFlow) | Revision `1bbd81546d590acf24470b39ab28f9cd918c3927`, subfolder **`SenseFlow-FLUX`** |
102
- | Wan teacher and shared components | [Wan-AI/Wan2.1-T2V-1.3B-Diffusers](https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B-Diffusers) | Original public teacher; see the video runtime setup |
103
- | Wan DMD2 student | [csusupergear/wan_t2v_checkpoints_release](https://huggingface.co/csusupergear/wan_t2v_checkpoints_release) | Prefix **`dmd2_t2v_vipe/0001000`**; retain its `.pth` and `.net_model.zip` files |
104
-
105
- The SenseFlow transformer is directly in `SenseFlow-FLUX`, not
106
- `SenseFlow-FLUX/transformer`. The Wan checkpoint uses the DFD/fastgen format;
107
- it is not a standalone Diffusers `transformer/` folder.
108
-
109
- The original SD3-M manifest retains the historical internal objective label
110
- `tdm`; its recorded implementation is `sd_distill_classic_v1`. Original
111
- manifests are preserved without silently changing their labels or paths.
112
-
113
- ## Repository layout
114
-
115
- ```text
116
- Wenwu2000/ReSeed (main)/
117
- ├── README.md
118
- ├── requirements.txt
119
- ├── inference.py # Single-prompt SD3-M / SD3.5-M example
120
- ├── release_manifest.json # Pairings, sources, sizes and SHA-256 hashes
121
- ├── adapters/
122
- │ ├── sd3m/seed_42/checkpoint-1900/
123
- │ ├── sd35m/seed_42/checkpoint-2100/
124
- │ ├── flux/seed_42/checkpoint-2000/
125
- │ └── wan21/seed_42/checkpoint-300/
126
- │ # Each adapter directory contains residual_biases.pt and manifest.json
127
- └── students/
128
- ├── sd3m/seed_42/checkpoint-2000/
129
- └── sd35m/seed_42/checkpoint-2000/
130
- ├── manifest.json
131
- └── student/
132
- ├── config.json
133
- ├── diffusion_pytorch_model-00001-of-00002.safetensors
134
- ├── diffusion_pytorch_model-00002-of-00002.safetensors
135
- └── diffusion_pytorch_model.safetensors.index.json
136
- ```
137
 
138
- The complete release is approximately **18.22 GB** in decimal units. Most
139
- of that size is the two self-trained students. Their original inference
140
- files are retained without conversion or quantization. The adapters are small.
141
 
142
- This repository excludes optimizer state, fake-score models, teacher
143
- weights, text encoders, VAEs, datasets, prompt caches and generated media.
144
- It is an inference release, not a resumable training checkpoint archive.
145
- Absolute paths in original manifests are provenance records from our
146
- training system; users do not need to recreate those server paths.
147
 
148
- ## Environment and authentication
149
 
150
- Use Python 3.10 or later and install PyTorch for your CUDA environment.
151
- The image inference examples target a CUDA GPU with BF16 support. FLUX
152
- also needs enough memory for its transformer, text encoders and VAE; the
153
- example generator does not automatically quantize or offload them.
154
 
155
- The image-side reference environment uses:
 
 
 
156
 
157
- | Package | Version |
158
- | --- | --- |
159
- | PyTorch | 2.6.0 in the original generation environment; install for your CUDA setup |
160
- | Diffusers | 0.34.0 |
161
- | Transformers | 4.45.2 |
162
- | Hugging Face Hub | 0.36.2 |
163
 
164
- Install the Hub CLI before downloading:
 
 
 
 
165
 
166
- ```bash
167
- pip install huggingface_hub==0.36.2
168
- hf auth login
169
- hf auth whoami
170
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
171
 
172
- Your account needs access to this private repository. Separately accept
173
- the terms of any gated upstream model before downloading its components.
174
- Access to ReSeed does not automatically grant access to Stability AI or
175
- FLUX checkpoints. Never write an access token into source code or a README.
176
 
177
- ## Download the release
178
 
179
- Run the following from a working directory in which you want to keep the
180
- weights, upstream models and code:
181
 
182
  ```bash
 
183
  hf download Wenwu2000/ReSeed --revision main --local-dir reseed-weights
184
 
185
  git clone https://github.com/TWWinde/ReSeed.git
@@ -188,10 +107,8 @@ pip install -r reseed-weights/requirements.txt
188
  pip install -e ReSeed/image --no-deps
189
  ```
190
 
191
- The code revision above is the public ReSeed version used by these examples.
192
- The weight repository itself does not install the `reseed` Python package.
193
-
194
- For SD3-Medium only, avoid downloading the other student:
195
 
196
  ```bash
197
  hf download Wenwu2000/ReSeed --revision main --local-dir reseed-weights \
@@ -199,26 +116,13 @@ hf download Wenwu2000/ReSeed --revision main --local-dir reseed-weights \
199
  "adapters/sd3m/*" "students/sd3m/*"
200
  ```
201
 
202
- Replace both `sd3m` patterns with `sd35m` for SD3.5-Medium. For FLUX or
203
- Wan, download `adapters/flux/*` or `adapters/wan21/*` and the documentation;
204
- their public students must still be obtained separately.
205
-
206
- Python can download the same branch without cloning a large Git repository:
207
-
208
- ```python
209
- from huggingface_hub import snapshot_download
210
-
211
- weights = snapshot_download(
212
- repo_id="Wenwu2000/ReSeed",
213
- revision="main",
214
- local_dir="reseed-weights",
215
- )
216
- print(weights)
217
- ```
218
 
219
- ## SD3-Medium and SD3.5-Medium inference
220
 
221
- ### Download the matching public components
222
 
223
  ```bash
224
  hf download stabilityai/stable-diffusion-3-medium-diffusers \
@@ -228,93 +132,38 @@ hf download stabilityai/stable-diffusion-3.5-medium \
228
  --revision b940f670f0eda2d07fbb75229e779da1ad11eb80 --local-dir models/sd35m
229
  ```
230
 
231
- The example loads the included DMD transformer in place of the public
232
- teacher transformer. It reuses the public text encoders, tokenizers, VAE
233
- and scheduler. It does not run the teacher during student inference.
234
-
235
- ### Generate with ReSeed
236
 
237
  ```bash
238
  python reseed-weights/inference.py --weights reseed-weights --model sd3m \
239
  --base models/sd3m --prompt "a photo of a cat" --seed 0 \
240
- --output samples/sd3m_reseed_seed0.png
241
 
242
  python reseed-weights/inference.py --weights reseed-weights --model sd35m \
243
  --base models/sd35m --prompt "a photo of a cat" --seed 0 \
244
- --output samples/sd35m_reseed_seed0.png
245
  ```
246
 
247
- `--base` is a **local upstream snapshot directory**. `--weights` is the
248
- downloaded ReSeed branch root. `--model sd3m` or `sd35m` selects the correct
249
- student and adapter from `release_manifest.json` automatically.
250
 
251
- ### Generate the matched DMD baseline
 
 
 
 
 
252
 
253
- Use the same prompt and generation seed, and add `--without-reseed`:
254
-
255
- ```bash
256
- python reseed-weights/inference.py --weights reseed-weights --model sd3m \
257
- --base models/sd3m --prompt "a photo of a cat" --seed 0 \
258
- --without-reseed --output samples/sd3m_dmd_seed0.png
259
-
260
- python reseed-weights/inference.py --weights reseed-weights --model sd35m \
261
- --base models/sd35m --prompt "a photo of a cat" --seed 0 \
262
- --without-reseed --output samples/sd35m_dmd_seed0.png
263
- ```
264
-
265
- The baseline keeps the same DMD weights and initial noise; only the adapter
266
- is omitted. It is not a comparison against the public many-step teacher.
267
- The script refuses to overwrite an existing image, so use separate output names.
268
-
269
- ### Fixed sampler and adapter behavior
270
-
271
- | Setting | Both SD3 student examples |
272
- | --- | --- |
273
- | Resolution | 1024×1024 |
274
- | Solver | Native FlowMatch Euler |
275
- | Number of steps | 4 |
276
- | Student CFG | **1** |
277
- | Scheduler | Static shift 3, dynamic shifting disabled |
278
- | Explicit unshifted sigmas | `[1.0, 0.75, 0.5, 0.25]` |
279
- | Text maximum sequence length | 256 |
280
- | Inference dtype | BF16 |
281
- | Initial noise | CPU FP32 Gaussian draw, then BF16 cast |
282
- | ReSeed gate | Active for step 1 only; disabled for steps 2–4 |
283
-
284
- The `sigmas` argument above is the **unshifted** schedule supplied to the
285
- pipeline; the scheduler applies its shift. Do not replace it with the
286
- already shifted values from an adapter manifest. Those manifests may also
287
- contain BF16-rounded training boundaries.
288
-
289
- The teacher references use **28 steps / CFG 7** for SD3-Medium and **40
290
- steps / CFG 4.5** for SD3.5-Medium. Those guidance settings must not replace
291
- the student's CFG 1. The adapter manifests record the exact first interval
292
- and teacher microstep settings used for fitting.
293
-
294
- For multiple samples, change `--seed` and the output filename. The example
295
- handles one prompt/sample per invocation and resets its gate for each run.
296
- The commands are process-level examples, not single-GPU Slurm submissions.
297
- Our MUSICA jobs request a whole four-H100 node and parallelize independent
298
- samples across its four GPUs.
299
-
300
- ## SenseFlow-FLUX inference
301
-
302
- ### Obtain FLUX components and the released student
303
 
304
  ```bash
305
  hf download black-forest-labs/FLUX.1-dev \
306
  --revision 3de623fc3c33e44ffbe2bad470d0f45bccf2eb21 --local-dir models/flux-dev
307
-
308
  hf download domiso/SenseFlow --revision 1bbd81546d590acf24470b39ab28f9cd918c3927 \
309
  --include "SenseFlow-FLUX/*" --local-dir models/senseflow
310
  ```
311
 
312
- Use the SenseFlow transformer, not the FLUX.1-dev transformer, with this
313
- adapter. The public FLUX snapshot provides shared components and the scheduler.
314
-
315
- ### Prepare a prompt CSV
316
-
317
- Save the following as `prompts.csv` in the working directory:
318
 
319
  ```csv
320
  case_number,evaluation_seed,prompt,prompt_id,sample_id
@@ -322,11 +171,7 @@ case_number,evaluation_seed,prompt,prompt_id,sample_id
322
  1,1,a photo of a cat,0,1
323
  ```
324
 
325
- `case_number` must be unique. `evaluation_seed` is the generation noise
326
- seed. Rows for the same prompt share `prompt_id`; `sample_id` identifies
327
- the sample within that prompt. Quote the prompt field if it contains commas.
328
-
329
- ### Generate SenseFlow + ReSeed
330
 
331
  ```bash
332
  cd ReSeed/image
@@ -335,165 +180,57 @@ python tools/flux_generate.py --model senseflow \
335
  --senseflow-transformer ../../models/senseflow/SenseFlow-FLUX \
336
  --first-interval-residual-bias-checkpoint ../../reseed-weights/adapters/flux/seed_42/checkpoint-2000 \
337
  --prompts ../../prompts.csv --expected-count 2 --samples-per-prompt 2 \
338
- --evaluation-protocol reseed-single-seed-example \
339
  --steps 4 --resolution 1024 --max-sequence-length 512 \
340
  --output ../../samples/flux_reseed
341
  cd ../..
342
  ```
343
 
344
- Omit `--first-interval-residual-bias-checkpoint` and use a different output
345
- directory for the plain SenseFlow baseline. Retain the same CSV and seeds.
346
- The generator writes images, a generation manifest and a protocol record.
347
- `reseed-single-seed-example` is an example run label, not a paper benchmark.
348
 
349
- The native student uses four dynamically shifted Euler steps, approximately
350
- `[1, .904531, .759511, .512844, 0]`, with BF16 inference and text length 512.
351
- Its guidance embedding is **disabled**: do not pass `--guidance 3.5` for
352
- SenseFlow. FLUX.1-dev guidance 3.5 belongs to the teacher target, not this
353
- student. The generator checks the schedule against Diffusers and activates
354
- the bias only during the first step.
355
 
356
- ## Wan2.1 text-to-video inference
357
-
358
- ### Set up DFD / fastgen
359
-
360
- The Wan adapter uses the DMD2 student in the
361
- [DFD checkpoint release](https://huggingface.co/csusupergear/wan_t2v_checkpoints_release)
362
- and runs through [DFD / fastgen](https://github.com/csy2077/data-forcing-distillation).
363
- Follow the [ReSeed video installation guide](https://github.com/TWWinde/ReSeed/blob/eed9a03bd85a0af8a7dc36e8da5b48d2a31f58ca/video/README.md)
364
- to install the ReSeed module and apply its inference patch to a configured
365
- DFD checkout. The image package alone is not sufficient for video generation.
366
- Use DFD's environment and public Wan component setup; do not assume that
367
- the image requirements file provides the complete video environment.
368
-
369
- ### Download the public DMD2 checkpoint
370
 
371
  ```bash
372
  hf download csusupergear/wan_t2v_checkpoints_release \
373
  --include "dmd2_t2v_vipe/0001000*" --local-dir wan-checkpoints
374
  ```
375
 
376
- Keep the original `.pth` and `.net_model.zip` filenames. DFD receives the
377
- checkpoint **prefix** `dmd2_t2v_vipe/0001000`, rather than a renamed weight
378
- file or a Diffusers pipeline directory.
379
-
380
- ### Generate matched DMD2 and DMD2 + ReSeed videos
381
-
382
- Create a UTF-8 text file with one prompt per line, then run the following
383
- from the configured, patched DFD checkout:
384
 
385
  ```bash
386
  export DFD_CKPT_ROOT=/absolute/path/to/wan-checkpoints
387
  export DFD_RESEED_ADAPTER=/absolute/path/to/reseed-weights/adapters/wan21/seed_42/checkpoint-300
388
  export DFD_PROMPT_FILE=/absolute/path/to/video-prompts.txt
389
-
390
  python evaluation/generate_multiseed.py --config evaluation/config.yaml \
391
  --models dmd2 dmd2_reseed --seeds 0 1 \
392
  --output-root evaluation/reseed-example
393
  ```
394
 
395
- The driver passes `FASTGEN_RESEED_ADAPTER` to the patched inference process
396
- for the ReSeed arm. Setting `DFD_RESEED_ADAPTER` without installing that
397
- patch does not add ReSeed support to an unmodified inference script.
398
-
399
- | Setting | Wan DMD2 / DMD2 + ReSeed |
400
- | --- | --- |
401
- | Student solver | Native four-step ODE sampler |
402
- | Student time boundaries | `[.999, .937, .833, .624, 0]` |
403
- | Student CFG | None; no student CFG branch is evaluated |
404
- | Video size | 832×480 |
405
- | Frames / FPS | 81 / 16 |
406
- | Inference dtype | BF16 |
407
- | ReSeed gate | First step only |
408
- | Teacher reference | 50 steps, CFG 5, shift 5 |
409
-
410
- Do not substitute the public DFD post-trained checkpoint for DMD2: the
411
- included ReSeed adapter was fitted on **DMD2 iteration 1000**. The DFD
412
- post-trained model is a different comparison method, not this adapter's base.
413
-
414
- ## Reproducibility and file verification
415
-
416
- `release_manifest.json` records each model's teacher source, student origin,
417
- selected adapter update, matching student path, training seed provenance,
418
- original checkpoint location and per-file SHA-256 hashes. Original
419
- checkpoint manifests remain beside their weights.
420
- The release manifest retains `revision: single-seed`, identifying the
421
- original release branch. The same checkpoint files are also available on
422
- `main`; their hashes and model pairings are unchanged.
423
-
424
- After a **full** download, verify all inference files listed in the manifest:
425
-
426
- ```python
427
- import hashlib
428
- import json
429
- from pathlib import Path
430
-
431
- root = Path("reseed-weights")
432
- manifest = json.loads((root / "release_manifest.json").read_text())
433
- for entry in manifest["files"]:
434
- path = root / entry["path"]
435
- assert path.is_file(), f"Missing: {path}"
436
- assert path.stat().st_size == entry["bytes"], f"Wrong size: {path}"
437
- digest = hashlib.sha256()
438
- with path.open("rb") as handle:
439
- for block in iter(lambda: handle.read(8 * 1024 * 1024), b""):
440
- digest.update(block)
441
- assert digest.hexdigest() == entry["sha256"], f"Hash mismatch: {path}"
442
- print("All listed inference files verified.")
443
- ```
444
 
445
- For a selective download, verify only the manifest entries you deliberately
446
- downloaded; the full check above will correctly report the omitted models
447
- as missing. Documentation can be updated independently of the weight
448
- manifest. Hashes identify the released files, not generated outputs.
449
-
450
- For comparisons, keep the prompt, initial noise, scheduler, number of steps,
451
- resolution, dtype and guidance unchanged, and toggle only ReSeed. A matching
452
- integer seed is not sufficient if different runtimes draw noise differently.
453
- The supplied image example explicitly draws FP32 noise on CPU before the
454
- model-dtype cast. Do not normalize or interpolate that noise unless this is
455
- an explicitly separate experiment.
456
-
457
- Exact pixel equality across hardware, attention kernels or library versions
458
- is not guaranteed. BF16/FP32 changes can also change generated samples.
459
- Use the reference environment and pinned public snapshots when checking a
460
- reported comparison. The example scripts are inference entry points; they
461
- do not by themselves run COCO quality scoring, diversity evaluation or VBench.
462
-
463
- ## Troubleshooting
464
-
465
- | Symptom | Check |
466
- | --- | --- |
467
- | `401`, `403` or a private repository appears missing | Log in with an authorized account; check private-repo access and upstream gated-model approval separately. |
468
- | Missing weights in an old local download | Refresh the snapshot with `--revision main`; check any selective-download patterns. |
469
- | `No module named reseed` | Install `ReSeed/image` in the same Python environment used for inference. |
470
- | A LoRA loader rejects `residual_biases.pt` | Use the ReSeed residual-bias loader, not PEFT or `load_lora_weights`. |
471
- | Missing shard or incomplete student | Download both safetensor shards, their index and `config.json`; do not copy a single shard alone. |
472
- | SD3 sampler contract error | Use the matching pinned base, static shift 3, the supplied unshifted sigmas and student CFG 1. |
473
- | Unexpected FLUX guidance error | Use `--model senseflow`, the `SenseFlow-FLUX` folder, and no guidance embedding. |
474
- | Wan adapter is ignored | Install the ReSeed inference patch and use the `dmd2_reseed` arm. |
475
- | Wan checkpoint loader fails | Preserve the public `.pth` / `.net_model.zip` pair and pass the correct checkpoint prefix. |
476
- | CUDA out of memory | Check which model is loaded and available GPU memory. The example does not silently change resolution, precision, quantization or offload settings. |
477
- | Output image already exists | Select a new output filename; the SD3 example intentionally refuses overwrites. |
478
-
479
- ## Limitations and licensing
480
-
481
- These adapters are tied to their listed students and native sampling
482
- settings. Applying one to another backbone, another distillation run, a
483
- different step count or a different resolution is outside this release's
484
- documented setup. A diversity gain on an evaluation bank does not guarantee
485
- that every prompt, quality metric or generated sample improves.
486
-
487
- This branch provides one selected run per model, not training data, complete
488
- training state or all experimental variants. It does not include weights
489
- for GAD, DP-DMD, SDXL or MiniMax experiments.
490
-
491
- The ReSeed implementation is MIT-licensed in the code repository. That does
492
- **not** replace the licenses of model weights or dependencies. Consult each
493
- linked upstream model's terms and the code repository's
494
- [NOTICE](https://github.com/TWWinde/ReSeed/blob/eed9a03bd85a0af8a7dc36e8da5b48d2a31f58ca/NOTICE.md).
495
- Keeping this repository private does not grant additional redistribution or
496
- commercial-use rights. This documentation update does not make the repository public.
497
 
498
  ## Citation
499
 
@@ -508,6 +245,3 @@ commercial-use rights. This documentation update does not make the repository pu
508
  url={https://arxiv.org/abs/2610.37110}
509
  }
510
  ```
511
-
512
- Please also acknowledge the teacher models, external distilled students
513
- and DFD / fastgen software when using them in a new study.
 
16
  - Wan-AI/Wan2.1-T2V-1.3B-Diffusers
17
  ---
18
 
19
+ # ReSeed
20
 
21
  **A First-Step Bias Restores Lost Diversity in Diffusion Model Distillation**
22
 
23
  [Paper](https://arxiv.org/abs/2610.37110) · [Project page](https://twwinde.github.io/reseed-project-page/) · [Code](https://github.com/TWWinde/ReSeed)
24
 
25
+ ReSeed improves diversity in a frozen four-step student by fitting small
26
+ residual biases to the teacher's first-interval velocity. The biases are
27
+ enabled only in the first denoising step; the original weights and final
28
+ three steps remain unchanged.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
29
 
30
+ This repository provides four ReSeed adapters and two self-trained SD3 DMD
31
+ students. The selected adapter runs and our two student runs use training
32
+ seed 42. Generation seeds can be chosen independently.
33
 
34
+ ## Student models and their sources
 
 
 
 
35
 
36
+ ### SD3-Medium and SD3.5-Medium: trained by us
37
 
38
+ We trained these four-step students in our CRRA `unified_distill` codebase
39
+ from the corresponding Stability AI models. Both released students are
40
+ **seed 42, checkpoint-2000**. Their full transformer weights are included:
 
41
 
42
+ | Student | Released weights | Teacher | Teacher sampling reference |
43
+ | --- | --- | --- | --- |
44
+ | SD3-Medium DMD | [checkpoint-2000](https://huggingface.co/Wenwu2000/ReSeed/tree/main/students/sd3m/seed_42/checkpoint-2000) | [SD3-Medium](https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers) | 28 steps, CFG 7 |
45
+ | SD3.5-Medium DMD | [checkpoint-2000](https://huggingface.co/Wenwu2000/ReSeed/tree/main/students/sd35m/seed_42/checkpoint-2000) | [SD3.5-Medium](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium) | 40 steps, CFG 4.5 |
46
 
47
+ These are our experimental students, **not checkpoints from the official
48
+ DMD2 release**. They use four-step FlowMatch Euler inference at CFG 1.
49
+ For the underlying methods, see [DMD](https://arxiv.org/abs/2311.18828) and
50
+ [DMD2](https://github.com/tianweiy/DMD2).
 
 
51
 
52
+ The original training manifests are included beside each student. The
53
+ SD3-Medium manifest retains the historical objective label `tdm` and records
54
+ the implementation `sd_distill_classic_v1`; the SD3.5-Medium manifest records
55
+ `dmd`. These records have not been relabelled. The release's exact student
56
+ and adapter pairings are listed in [release_manifest.json](release_manifest.json).
57
 
58
+ ### Wan2.1 DMD2: public checkpoint from the DFD authors
59
+
60
+ The video student is the Wan2.1-T2V-1.3B DMD2 checkpoint released by the
61
+ [Data-Forcing Distillation authors](https://github.com/csy2077/data-forcing-distillation).
62
+ We use **`dmd2_t2v_vipe/0001000`**, trained on the ViPE `long` set:
63
+
64
+ - [Checkpoint files](https://huggingface.co/csusupergear/wan_t2v_checkpoints_release/tree/main/dmd2_t2v_vipe)
65
+ - [Teacher: Wan2.1-T2V-1.3B](https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B-Diffusers)
66
+ - [DFD / fastgen code and setup](https://github.com/csy2077/data-forcing-distillation)
67
+
68
+ We did not train this DMD2 student. We trained the ReSeed adapter on top of
69
+ it and provide only that adapter here. The separately released DFD
70
+ post-trained model (`ours_wan_t2v_vipe/0000040`) is a different baseline,
71
+ not the student used by this adapter.
72
+
73
+ ### FLUX: public SenseFlow student
74
+
75
+ We use the four-step **SenseFlow-FLUX** transformer from
76
+ [domiso/SenseFlow](https://huggingface.co/domiso/SenseFlow/tree/main/SenseFlow-FLUX),
77
+ with shared components from [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev).
78
+ This student was released by the SenseFlow authors, not trained by us.
79
+ Download it separately; this repository contains its ReSeed adapter only.
80
+
81
+ ## Released adapters
82
+
83
+ | Student | Adapter directory | Adapter updates | Bias parameters |
84
+ | --- | --- | ---: | ---: |
85
+ | SD3-Medium DMD | `adapters/sd3m/seed_42/checkpoint-1900` | 1900 | 72,192 |
86
+ | SD3.5-Medium DMD | `adapters/sd35m/seed_42/checkpoint-2100` | 2100 | 72,192 |
87
+ | SenseFlow-FLUX | `adapters/flux/seed_42/checkpoint-2000` | 2000 | 233,472 |
88
+ | Wan2.1 DMD2 | `adapters/wan21/seed_42/checkpoint-300` | 300 | 46,080 |
89
 
90
+ Each directory contains `residual_biases.pt` and `manifest.json`. Adapter
91
+ updates count ReSeed fitting, not student distillation: both included SD3
92
+ students remain at update 2000. Use each adapter with its listed student.
93
+ These are residual-bias adapters, not LoRA weights.
94
 
95
+ ## Download and install
96
 
97
+ Use Python 3.10+ and a CUDA/BF16-capable PyTorch installation. Our image
98
+ runtime used PyTorch 2.6.0, Diffusers 0.34.0 and Transformers 4.45.2.
99
 
100
  ```bash
101
+ pip install huggingface_hub==0.36.2
102
  hf download Wenwu2000/ReSeed --revision main --local-dir reseed-weights
103
 
104
  git clone https://github.com/TWWinde/ReSeed.git
 
107
  pip install -e ReSeed/image --no-deps
108
  ```
109
 
110
+ The full download is about 18.22 GB, mostly the two student transformers.
111
+ To download only SD3-Medium, use:
 
 
112
 
113
  ```bash
114
  hf download Wenwu2000/ReSeed --revision main --local-dir reseed-weights \
 
116
  "adapters/sd3m/*" "students/sd3m/*"
117
  ```
118
 
119
+ Use `sd35m` instead of `sd3m` for SD3.5-Medium. For FLUX or Wan, only the
120
+ corresponding adapter is needed from this repository. Upstream gated models
121
+ require accepting their terms and logging in with `hf auth login`.
 
 
 
 
 
 
 
 
 
 
 
 
 
122
 
123
+ ## SD3 image generation
124
 
125
+ Download the public components for the model you want to run:
126
 
127
  ```bash
128
  hf download stabilityai/stable-diffusion-3-medium-diffusers \
 
132
  --revision b940f670f0eda2d07fbb75229e779da1ad11eb80 --local-dir models/sd35m
133
  ```
134
 
135
+ Generate an image with the matching DMD student and ReSeed:
 
 
 
 
136
 
137
  ```bash
138
  python reseed-weights/inference.py --weights reseed-weights --model sd3m \
139
  --base models/sd3m --prompt "a photo of a cat" --seed 0 \
140
+ --output samples/sd3m_reseed.png
141
 
142
  python reseed-weights/inference.py --weights reseed-weights --model sd35m \
143
  --base models/sd35m --prompt "a photo of a cat" --seed 0 \
144
+ --output samples/sd35m_reseed.png
145
  ```
146
 
147
+ Add `--without-reseed` and choose a different output filename to generate
148
+ the plain DMD baseline with the same prompt and initial noise.
 
149
 
150
+ The script loads our student transformer and the public model's text
151
+ encoders, tokenizers, VAE and scheduler. It uses 1024×1024 BF16 inference,
152
+ four Euler steps, CFG 1 and static shift 3. The pipeline receives unshifted
153
+ sigmas `[1, .75, .5, .25]`; do not substitute the shifted boundaries from
154
+ the training manifest. Initial noise is drawn in FP32 on CPU, then cast to
155
+ BF16. The adapter is disabled after the first step.
156
 
157
+ ## FLUX image generation
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
158
 
159
  ```bash
160
  hf download black-forest-labs/FLUX.1-dev \
161
  --revision 3de623fc3c33e44ffbe2bad470d0f45bccf2eb21 --local-dir models/flux-dev
 
162
  hf download domiso/SenseFlow --revision 1bbd81546d590acf24470b39ab28f9cd918c3927 \
163
  --include "SenseFlow-FLUX/*" --local-dir models/senseflow
164
  ```
165
 
166
+ Create `prompts.csv`:
 
 
 
 
 
167
 
168
  ```csv
169
  case_number,evaluation_seed,prompt,prompt_id,sample_id
 
171
  1,1,a photo of a cat,0,1
172
  ```
173
 
174
+ Then run:
 
 
 
 
175
 
176
  ```bash
177
  cd ReSeed/image
 
180
  --senseflow-transformer ../../models/senseflow/SenseFlow-FLUX \
181
  --first-interval-residual-bias-checkpoint ../../reseed-weights/adapters/flux/seed_42/checkpoint-2000 \
182
  --prompts ../../prompts.csv --expected-count 2 --samples-per-prompt 2 \
183
+ --evaluation-protocol reseed-example \
184
  --steps 4 --resolution 1024 --max-sequence-length 512 \
185
  --output ../../samples/flux_reseed
186
  cd ../..
187
  ```
188
 
189
+ Omit the adapter argument for the plain SenseFlow baseline. This student
190
+ uses its native four-step dynamically shifted Euler schedule with the
191
+ guidance embedding disabled. Do not pass the teacher's guidance 3.5 to the
192
+ student. The transformer is in `SenseFlow-FLUX`, not a nested `transformer/` folder.
193
 
194
+ ## Wan video generation
 
 
 
 
 
195
 
196
+ First follow the [ReSeed video setup](https://github.com/TWWinde/ReSeed/blob/eed9a03bd85a0af8a7dc36e8da5b48d2a31f58ca/video/README.md)
197
+ to install the video code and apply the ReSeed inference patch to DFD.
198
+ The video runtime has separate dependencies from the image package.
 
 
 
 
 
 
 
 
 
 
 
199
 
200
  ```bash
201
  hf download csusupergear/wan_t2v_checkpoints_release \
202
  --include "dmd2_t2v_vipe/0001000*" --local-dir wan-checkpoints
203
  ```
204
 
205
+ Keep the `.pth` and `.net_model.zip` files under their original names.
206
+ From the configured DFD checkout, use a text file with one prompt per line:
 
 
 
 
 
 
207
 
208
  ```bash
209
  export DFD_CKPT_ROOT=/absolute/path/to/wan-checkpoints
210
  export DFD_RESEED_ADAPTER=/absolute/path/to/reseed-weights/adapters/wan21/seed_42/checkpoint-300
211
  export DFD_PROMPT_FILE=/absolute/path/to/video-prompts.txt
 
212
  python evaluation/generate_multiseed.py --config evaluation/config.yaml \
213
  --models dmd2 dmd2_reseed --seeds 0 1 \
214
  --output-root evaluation/reseed-example
215
  ```
216
 
217
+ This generates matched DMD2 and DMD2 + ReSeed videos. The student uses four
218
+ ODE steps with boundaries `[.999, .937, .833, .624, 0]`, no student CFG,
219
+ 81 frames at 832×480, 16 FPS and BF16. The teacher reference uses 50 UniPC
220
+ steps, CFG 5 and shift 5. ReSeed is active only in the first student step.
221
+
222
+ ## Files and licenses
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
223
 
224
+ The release contains inference weights, not optimizer state, fake-score
225
+ models, public teacher weights, datasets or generated samples. Text encoders
226
+ and VAEs come from the linked base models. File hashes, pinned source
227
+ revisions and checkpoint pairings are recorded in `release_manifest.json`;
228
+ its branch and visibility fields describe the original private release.
229
+ The repository is now public, with current instructions on `main` and the
230
+ original `single-seed` branch retained as an archive.
231
+
232
+ The ReSeed code is MIT-licensed. Model weights retain their upstream
233
+ licenses and access conditions; see [NOTICE](https://github.com/TWWinde/ReSeed/blob/main/NOTICE.md).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
234
 
235
  ## Citation
236
 
 
245
  url={https://arxiv.org/abs/2610.37110}
246
  }
247
  ```