Add MNIST + CIFAR-10 DDPM weights (EMA and raw, safetensors)
Browse files- README.md +57 -0
- cifar10_cond/config.json +28 -0
- cifar10_cond/ema.safetensors +3 -0
- cifar10_cond/model.safetensors +3 -0
- cifar10_uncond/config.json +26 -0
- cifar10_uncond/ema.safetensors +3 -0
- cifar10_uncond/model.safetensors +3 -0
- mnist/config.json +21 -0
- mnist/ema.safetensors +3 -0
- mnist/model.safetensors +3 -0
README.md
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
tags:
|
| 4 |
+
- diffusion
|
| 5 |
+
- ddpm
|
| 6 |
+
- ddim
|
| 7 |
+
- from-scratch
|
| 8 |
+
- pytorch
|
| 9 |
+
- unconditional-image-generation
|
| 10 |
+
pipeline_tag: unconditional-image-generation
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# diffusion-from-scratch — trained checkpoints
|
| 14 |
+
|
| 15 |
+
Weights for [github.com/adimunot21/diffusion-from-scratch](https://github.com/adimunot21/diffusion-from-scratch),
|
| 16 |
+
a DDPM/DDIM implementation written from scratch in PyTorch — forward process, noise
|
| 17 |
+
schedules, U-Net denoiser, samplers and classifier-free guidance, no diffusers dependency.
|
| 18 |
+
|
| 19 |
+
Three models. Each folder has `config.json` plus two weight files:
|
| 20 |
+
|
| 21 |
+
- **`ema.safetensors`** — exponential moving average of the weights (`ema_decay=0.9999`). **Use these for sampling.**
|
| 22 |
+
- **`model.safetensors`** — the raw final training weights, kept for completeness.
|
| 23 |
+
|
| 24 |
+
| Folder | Model | Params | Schedule | Epochs | Final train loss | FID |
|
| 25 |
+
|---|---|---|---|---|---|---|
|
| 26 |
+
| `mnist/` | MNIST, unconditional | 9.53 M | linear, T=1000 | 50 | 0.0210 | 34.1 |
|
| 27 |
+
| `cifar10_uncond/` | CIFAR-10, unconditional | 46.03 M | cosine, T=1000 | 100 | 0.0547 | 71.2 |
|
| 28 |
+
| `cifar10_cond/` | CIFAR-10, class-conditional | 46.03 M | cosine, T=1000 | 100 | 0.0557 | 65.3 |
|
| 29 |
+
|
| 30 |
+
The CIFAR-10 U-Nets use channels `[128, 256, 256, 512]`, 2 residual blocks per level and
|
| 31 |
+
4-head self-attention at the two middle resolutions. The conditional model was trained
|
| 32 |
+
with classifier-free guidance (`uncond_prob=0.1`, 10 classes).
|
| 33 |
+
|
| 34 |
+
## Loading
|
| 35 |
+
|
| 36 |
+
These are plain `state_dict`s for the `UNet` class in the source repo — not a
|
| 37 |
+
`diffusers` pipeline. Build the model from the repo, then load:
|
| 38 |
+
|
| 39 |
+
```python
|
| 40 |
+
from huggingface_hub import hf_hub_download
|
| 41 |
+
from safetensors.torch import load_file
|
| 42 |
+
|
| 43 |
+
path = hf_hub_download("adimunot/diffusion-from-scratch",
|
| 44 |
+
"cifar10_uncond/ema.safetensors")
|
| 45 |
+
model.load_state_dict(load_file(path)) # model = UNet(**config) from the repo
|
| 46 |
+
model.eval()
|
| 47 |
+
```
|
| 48 |
+
|
| 49 |
+
`config.json` carries the exact architecture and training hyperparameters each
|
| 50 |
+
checkpoint was produced with, so it can be fed straight into the constructor.
|
| 51 |
+
|
| 52 |
+
## Notes
|
| 53 |
+
|
| 54 |
+
- FID was computed during the project's evaluation phase; treat the numbers as
|
| 55 |
+
self-reported and comparable only within this repo.
|
| 56 |
+
- These are learning-project models trained on a single GPU, not competitive baselines.
|
| 57 |
+
CIFAR-10 FID in the 65–71 range reflects the small model and 100-epoch budget.
|
cifar10_cond/config.json
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"num_timesteps": 1000,
|
| 3 |
+
"schedule_type": "cosine",
|
| 4 |
+
"channel_list": [
|
| 5 |
+
128,
|
| 6 |
+
256,
|
| 7 |
+
256,
|
| 8 |
+
512
|
| 9 |
+
],
|
| 10 |
+
"time_dim": 256,
|
| 11 |
+
"num_res_blocks": 2,
|
| 12 |
+
"attention_levels": [
|
| 13 |
+
false,
|
| 14 |
+
true,
|
| 15 |
+
true,
|
| 16 |
+
false
|
| 17 |
+
],
|
| 18 |
+
"num_heads": 4,
|
| 19 |
+
"num_classes": 10,
|
| 20 |
+
"epochs": 100,
|
| 21 |
+
"batch_size": 128,
|
| 22 |
+
"lr": 0.0002,
|
| 23 |
+
"grad_clip": 1.0,
|
| 24 |
+
"ema_decay": 0.9999,
|
| 25 |
+
"uncond_prob": 0.1,
|
| 26 |
+
"epoch": 100,
|
| 27 |
+
"final_epoch_loss": 0.05565971454175619
|
| 28 |
+
}
|
cifar10_cond/ema.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1167c1544c46097fa2c34e020dc159b319ef29c3f8c18fa5e140508e22879525
|
| 3 |
+
size 184143500
|
cifar10_cond/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cb9674336115e2a2872a7b6258ea58e40cde1fdfcea39039e1acd38ffd2c0c80
|
| 3 |
+
size 184143500
|
cifar10_uncond/config.json
ADDED
|
@@ -0,0 +1,26 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"num_timesteps": 1000,
|
| 3 |
+
"schedule_type": "cosine",
|
| 4 |
+
"channel_list": [
|
| 5 |
+
128,
|
| 6 |
+
256,
|
| 7 |
+
256,
|
| 8 |
+
512
|
| 9 |
+
],
|
| 10 |
+
"time_dim": 256,
|
| 11 |
+
"num_res_blocks": 2,
|
| 12 |
+
"attention_levels": [
|
| 13 |
+
false,
|
| 14 |
+
true,
|
| 15 |
+
true,
|
| 16 |
+
false
|
| 17 |
+
],
|
| 18 |
+
"num_heads": 4,
|
| 19 |
+
"epochs": 100,
|
| 20 |
+
"batch_size": 128,
|
| 21 |
+
"lr": 0.0002,
|
| 22 |
+
"grad_clip": 1.0,
|
| 23 |
+
"ema_decay": 0.9999,
|
| 24 |
+
"epoch": 100,
|
| 25 |
+
"final_epoch_loss": 0.05466018270414609
|
| 26 |
+
}
|
cifar10_uncond/ema.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f012186a8c1d7ced62f780b5af76e59f14f5940048cea482e3284e0f60701a1b
|
| 3 |
+
size 184132148
|
cifar10_uncond/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:12c37d9150ff716f24c6b96851237e517afda78b53a2cc1da5461d049354965f
|
| 3 |
+
size 184132148
|
mnist/config.json
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"num_timesteps": 1000,
|
| 3 |
+
"schedule_type": "linear",
|
| 4 |
+
"beta_start": 0.0001,
|
| 5 |
+
"beta_end": 0.02,
|
| 6 |
+
"channel_list": [
|
| 7 |
+
64,
|
| 8 |
+
128,
|
| 9 |
+
256
|
| 10 |
+
],
|
| 11 |
+
"time_dim": 128,
|
| 12 |
+
"num_res_blocks": 2,
|
| 13 |
+
"epochs": 50,
|
| 14 |
+
"batch_size": 128,
|
| 15 |
+
"lr": 0.0002,
|
| 16 |
+
"grad_clip": 1.0,
|
| 17 |
+
"ema_decay": 0.9999,
|
| 18 |
+
"epoch": 50,
|
| 19 |
+
"global_step": 23400,
|
| 20 |
+
"final_epoch_loss": 0.020972778296305075
|
| 21 |
+
}
|
mnist/ema.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7e9737724d63342c228dda87352ab7b571781b48bd209cbc10fd3d0cd318cfd1
|
| 3 |
+
size 38131780
|
mnist/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3285bc61750268a2bfb02b80560b3b1398ee52d825a96583e8d865711afcfe8e
|
| 3 |
+
size 38131780
|