| license: mit | |
| Semantic VAE | |
| Purpose: Extremely fast convergence on illustraton data for a downstream DiT model | |
| Trained from DINOv2 in 12 hours as follows: | |
| Frozen DINOv2 -> LayerNorm -> 32 channels | |
| Frozen DINOv2 with patch embed unfrozen -> LayerNorm -> 32 channels | |
| 64 channel bottleneck with Sigreg loss (1e-3 weight) from LeJEPA | |
| VA-VAE style convolution decoder | |
| Loss is all layers of DINOv2 and DINOv3 equally weighted MSE on input vs reconstruction. | |
| There are no pixel losses/LPIPS/GAN used, this is DINO space loss only. |