Instructions to use honghong3/VER9 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use honghong3/VER9 with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("honghong3/VER9", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
|
Download README.md from honghong3/VER9: direct link, hf CLI and curl.
- Browser
- Download file 4.29 kB
-
https://huggingface.co/honghong3/VER9/resolve/main/README.md
- Command line
-
hf download hf://honghong3/VER9/README.md
-
curl -L -o README.md https://huggingface.co/honghong3/VER9/resolve/main/README.md
4.29 kB
metadata
license: mit
library_name: diffusers
pipeline_tag: text-to-image
tags:
- mmdit
- text-to-image
- flow-matching
- dc-ae
- t5
language:
- en
- ko
Diffusion MMDiT with DC-AE f32c32
Inference Requirements
- Inference: Min VRAM 15GB
- Storage: Min 30GB free disk space (excluding
requirements.txtpackages) - Environment: Local & Google Colab supported
Update
CPU Offloading Mode
If your GPU has less than 15GB VRAM, run inference in CPU offloading mode. The model stays on CPU; transformer blocks are streamed to the GPU in chunks.
On startup, choose n:
model upload on gpu (y or n): n
Then set the chunk size (layers loaded onto the GPU at once):
block chunk size (1~8, recommended 4):
| CHUNK | Layers on GPU | With prefetch | Approx. VRAM |
|---|---|---|---|
| 1 | 1 | 2 | ~0.5GB |
| 2 | 2 | 4 | ~1.1GB |
| 4 | 4 | 8 | ~2.2GB |
| 8 | 8 | 16 | ~4.3GB |
- Larger chunk = faster, more VRAM.
- Smaller chunk = safer on low VRAM.
- Up to
2 × CHUNKlayers are resident with prefetch.
Trade-off: VRAM drops from ~15GB to ~3GB, but inference is slower due to CPU-GPU PCIe transfers. Output quality may also be slightly lower than full GPU mode (differences in floating-point precision and kernel accumulation order across CPU↔GPU transfers).
Installation
python -m pip install -r requirements.txt
Architecture
- Autoencoder: DC-AE f32c32 (32× compression, 32 latent channels)
- Text Encoder: T5-Large
- Backbone: MMDiT with Joint Attention
- Implementation: Custom implementation (no pretrained base model)
- Parameters: 4B
Train Environment
- Framework: PyTorch (no custom CUDA kernels)
- GPU: NVIDIA RTX PRO 6000 (96GB VRAM)
- Platform: Google Colab
Training
Configuration
- Batch Size: Micro-batch 4 × Gradient Accumulation 8 = Effective Batch 32
- Training Speed: ~2.3 s / optimizer step
- Max Resolution: 1024 × 1024
- Dataset: 2M images
- Objective: Flow Matching (shift = 3.0)
- Precision: BF16
- Optimizer: AdamW
- Learning Rate: 1e-4
- EMA: Enabled
Training Stages
| Stage | Description | Steps | Patch Size |
|---|---|---|---|
| Stage 1 | Pre-training | 600K | 2 |
| Stage 2 | Fine-tuning | 11K | 1 |
Stage 2 Notes:
- Reset patch embedding weights.
- Reset output projection weights.
Loss by Noise Level
| Noise Level | Average Loss |
|---|---|
| 1.0 – 0.9 | 4.0 ~ 4.3 |
| 0.9 – 0.8 | 2.9 ~ 3.1 |
| 0.8 – 0.7 | 2.4 ~ 2.6 |
| 0.7 – 0.6 | 2.0 ~ 2.1 |
| 0.6 – 0.5 | 1.8 ~ 1.9 |
| 0.5 – 0.4 | 1.6 ~ 1.7 |
| 0.4 – 0.3 | 1.3 ~ 1.6 |
| 0.3 – 0.2 | 1.2 ~ 1.4 |
| 0.2 – 0.1 | 1.0 ~ 1.1 |
Generation Results
Patch Size 2 + DC-AE f32c32
- Issue: Facial features, particularly the eyes, tend to collapse or become distorted.
- Observation: Limited spatial detail makes accurate reconstruction of fine facial features challenging.
Patch Size 1 + DC-AE f32c32
- Fine-tuning Result: Fine-tuning with patch size 1 improved facial feature reconstruction.
- Improvement: Eye and facial distortions were partially resolved, with better preservation of fine details.
Current Limitations
- Full-body Generation: Full-body generation remains challenging due to insufficient full-body samples in the training dataset.
- Observation: The current dataset contains insufficient full-body examples to learn consistent full-body composition and anatomical details.
- Future Work: Increase the proportion and diversity of full-body images in the training dataset.
Train Architecture
- Use Cache
- Cpu Multi Threading
Inference Architecture
UI
- Sampling
- Inpainting
- Result







