--- license: gpl-3.0 datasets: - stanford-vision-lab/gpic language: - en pipeline_tag: image-text-to-image tags: - VAE - mmDiT --- Weights for mmDiT and VAE from [Diffusion Transformer](https://github.com/someonecantcode/diffusion-transformer). Weights structure: ```python flow_match_checkpoint = { 'model': raw_model.state_dict(), 'optimizer': optimizer.state_dict(), 'ema': ema.state_dict, 'step': step } vae_checkpoint = vae.state_dict(), ``` Hyperparameters used for model. ```python @dataclass class TrainConfig: lr: float = 1e-4 batch_size: int = 8 total_steps: int = 10000 # 50k steps is ideal. lr_cooldown_steps: int = 20000 starting_step: int = 0 logging_interval: int = 100 grad_accum_steps: int = 2 # not really needed save_dir: str = "./saved_models/" ckpt: str = "flowmatch-ckpt.pt" @dataclass class VAEConfig: kl_beta: float = 5e-2 fft_weight: float = 0.05 lpips_weight: float = 0.5 latent_channels: tuple[int, ...] = (128, 256, 512, 512) z_channels: int = 32 n_heads: int = 8 resnet_blocks: int = 2 @dataclass class mmDiTConfig: d_model: int = 768 n_heads: int = 32 n_blocks: int = 16 patch_size: int = 2 color_channels: int = 32 # z_channels height: int = 32 # latent width: int = 32 # latent t_channels: int = 64 ```