adimunot commited on
Commit
a8a3268
·
verified ·
1 Parent(s): 5cb394d

Add MNIST + CIFAR-10 DDPM weights (EMA and raw, safetensors)

Browse files
README.md ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - diffusion
5
+ - ddpm
6
+ - ddim
7
+ - from-scratch
8
+ - pytorch
9
+ - unconditional-image-generation
10
+ pipeline_tag: unconditional-image-generation
11
+ ---
12
+
13
+ # diffusion-from-scratch — trained checkpoints
14
+
15
+ Weights for [github.com/adimunot21/diffusion-from-scratch](https://github.com/adimunot21/diffusion-from-scratch),
16
+ a DDPM/DDIM implementation written from scratch in PyTorch — forward process, noise
17
+ schedules, U-Net denoiser, samplers and classifier-free guidance, no diffusers dependency.
18
+
19
+ Three models. Each folder has `config.json` plus two weight files:
20
+
21
+ - **`ema.safetensors`** — exponential moving average of the weights (`ema_decay=0.9999`). **Use these for sampling.**
22
+ - **`model.safetensors`** — the raw final training weights, kept for completeness.
23
+
24
+ | Folder | Model | Params | Schedule | Epochs | Final train loss | FID |
25
+ |---|---|---|---|---|---|---|
26
+ | `mnist/` | MNIST, unconditional | 9.53 M | linear, T=1000 | 50 | 0.0210 | 34.1 |
27
+ | `cifar10_uncond/` | CIFAR-10, unconditional | 46.03 M | cosine, T=1000 | 100 | 0.0547 | 71.2 |
28
+ | `cifar10_cond/` | CIFAR-10, class-conditional | 46.03 M | cosine, T=1000 | 100 | 0.0557 | 65.3 |
29
+
30
+ The CIFAR-10 U-Nets use channels `[128, 256, 256, 512]`, 2 residual blocks per level and
31
+ 4-head self-attention at the two middle resolutions. The conditional model was trained
32
+ with classifier-free guidance (`uncond_prob=0.1`, 10 classes).
33
+
34
+ ## Loading
35
+
36
+ These are plain `state_dict`s for the `UNet` class in the source repo — not a
37
+ `diffusers` pipeline. Build the model from the repo, then load:
38
+
39
+ ```python
40
+ from huggingface_hub import hf_hub_download
41
+ from safetensors.torch import load_file
42
+
43
+ path = hf_hub_download("adimunot/diffusion-from-scratch",
44
+ "cifar10_uncond/ema.safetensors")
45
+ model.load_state_dict(load_file(path)) # model = UNet(**config) from the repo
46
+ model.eval()
47
+ ```
48
+
49
+ `config.json` carries the exact architecture and training hyperparameters each
50
+ checkpoint was produced with, so it can be fed straight into the constructor.
51
+
52
+ ## Notes
53
+
54
+ - FID was computed during the project's evaluation phase; treat the numbers as
55
+ self-reported and comparable only within this repo.
56
+ - These are learning-project models trained on a single GPU, not competitive baselines.
57
+ CIFAR-10 FID in the 65–71 range reflects the small model and 100-epoch budget.
cifar10_cond/config.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "num_timesteps": 1000,
3
+ "schedule_type": "cosine",
4
+ "channel_list": [
5
+ 128,
6
+ 256,
7
+ 256,
8
+ 512
9
+ ],
10
+ "time_dim": 256,
11
+ "num_res_blocks": 2,
12
+ "attention_levels": [
13
+ false,
14
+ true,
15
+ true,
16
+ false
17
+ ],
18
+ "num_heads": 4,
19
+ "num_classes": 10,
20
+ "epochs": 100,
21
+ "batch_size": 128,
22
+ "lr": 0.0002,
23
+ "grad_clip": 1.0,
24
+ "ema_decay": 0.9999,
25
+ "uncond_prob": 0.1,
26
+ "epoch": 100,
27
+ "final_epoch_loss": 0.05565971454175619
28
+ }
cifar10_cond/ema.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1167c1544c46097fa2c34e020dc159b319ef29c3f8c18fa5e140508e22879525
3
+ size 184143500
cifar10_cond/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb9674336115e2a2872a7b6258ea58e40cde1fdfcea39039e1acd38ffd2c0c80
3
+ size 184143500
cifar10_uncond/config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "num_timesteps": 1000,
3
+ "schedule_type": "cosine",
4
+ "channel_list": [
5
+ 128,
6
+ 256,
7
+ 256,
8
+ 512
9
+ ],
10
+ "time_dim": 256,
11
+ "num_res_blocks": 2,
12
+ "attention_levels": [
13
+ false,
14
+ true,
15
+ true,
16
+ false
17
+ ],
18
+ "num_heads": 4,
19
+ "epochs": 100,
20
+ "batch_size": 128,
21
+ "lr": 0.0002,
22
+ "grad_clip": 1.0,
23
+ "ema_decay": 0.9999,
24
+ "epoch": 100,
25
+ "final_epoch_loss": 0.05466018270414609
26
+ }
cifar10_uncond/ema.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f012186a8c1d7ced62f780b5af76e59f14f5940048cea482e3284e0f60701a1b
3
+ size 184132148
cifar10_uncond/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:12c37d9150ff716f24c6b96851237e517afda78b53a2cc1da5461d049354965f
3
+ size 184132148
mnist/config.json ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "num_timesteps": 1000,
3
+ "schedule_type": "linear",
4
+ "beta_start": 0.0001,
5
+ "beta_end": 0.02,
6
+ "channel_list": [
7
+ 64,
8
+ 128,
9
+ 256
10
+ ],
11
+ "time_dim": 128,
12
+ "num_res_blocks": 2,
13
+ "epochs": 50,
14
+ "batch_size": 128,
15
+ "lr": 0.0002,
16
+ "grad_clip": 1.0,
17
+ "ema_decay": 0.9999,
18
+ "epoch": 50,
19
+ "global_step": 23400,
20
+ "final_epoch_loss": 0.020972778296305075
21
+ }
mnist/ema.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e9737724d63342c228dda87352ab7b571781b48bd209cbc10fd3d0cd318cfd1
3
+ size 38131780
mnist/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3285bc61750268a2bfb02b80560b3b1398ee52d825a96583e8d865711afcfe8e
3
+ size 38131780