| license: apache-2.0 | |
| base_model: OpenMOSS-Team/MOSS-Image-SFT | |
| tags: | |
| - diffusers | |
| - text-to-image | |
| - 4bit | |
| - bitsandbytes | |
| - nf4 | |
| - moss-image | |
| # MOSS-Image-SFT-4bit | |
| This is an optimized 4-bit (NF4) quantized version of [OpenMOSS-Team/MOSS-Image-SFT](https://huggingface.co/OpenMOSS-Team/MOSS-Image-SFT), created for efficient inference on 16GB GPUs (such as NVIDIA Tesla T4, RTX 3080/3090/4080/4090). | |
| ## Quantization Details | |
| - **Text Encoder**: Qwen3.5-9B quantized to 4-bit NF4 via `bitsandbytes` (~5.5 GB). | |
| - **Transformer Backbone**: 5B DiT quantized to 4-bit NF4 with small routing and time-embedding modules preserved in FP16 (~2.6 GB). | |
| - **VAE**: Kept at full precision FP32 (FLUX.2 VAE). | |
| - **Total Footprint**: ~8.5 GB (compressed from ~30 GB, >70% reduction). | |
| - **Compute Precision**: FP16 (optimized for Tesla T4 Tensor Cores). | |
| ## Usage with Diffusers | |
| ```python | |
| import torch | |
| from diffusers import DiffusionPipeline | |
| pipe = DiffusionPipeline.from_pretrained( | |
| "eramth/MOSS-Image-SFT-4bit", | |
| custom_pipeline="pipeline_moss_image", | |
| trust_remote_code=True, | |
| torch_dtype={"default": torch.float16, "vae": torch.float32} | |
| ) | |
| pipe.to("cuda") | |
| prompt = "a photorealistic portrait of an astronaut floating in deep cosmic space" | |
| image = pipe(prompt, height=1024, width=1024, num_inference_steps=25).images[0] | |
| image.save("output.png") | |
| ``` | |