Upload 4-bit NF4 quantized MOSS-Image-SFT

c44ace8 verified 6 days ago

1.35 kB

license: apache-2.0
base_model: OpenMOSS-Team/MOSS-Image-SFT
tags:
  - diffusers
  - text-to-image
  - 4bit
  - bitsandbytes
  - nf4
  - moss-image

MOSS-Image-SFT-4bit

This is an optimized 4-bit (NF4) quantized version of OpenMOSS-Team/MOSS-Image-SFT, created for efficient inference on 16GB GPUs (such as NVIDIA Tesla T4, RTX 3080/3090/4080/4090).

Quantization Details

Text Encoder: Qwen3.5-9B quantized to 4-bit NF4 via bitsandbytes (~5.5 GB).
Transformer Backbone: 5B DiT quantized to 4-bit NF4 with small routing and time-embedding modules preserved in FP16 (~2.6 GB).
VAE: Kept at full precision FP32 (FLUX.2 VAE).
Total Footprint: ~8.5 GB (compressed from ~30 GB, >70% reduction).
Compute Precision: FP16 (optimized for Tesla T4 Tensor Cores).

Usage with Diffusers

import torch
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(
    "eramth/MOSS-Image-SFT-4bit",
    custom_pipeline="pipeline_moss_image",
    trust_remote_code=True,
    torch_dtype={"default": torch.float16, "vae": torch.float32}
)
pipe.to("cuda")

prompt = "a photorealistic portrait of an astronaut floating in deep cosmic space"
image = pipe(prompt, height=1024, width=1024, num_inference_steps=25).images[0]
image.save("output.png")