--- base_model: hustvl/Moebius base_model_relation: quantized license: mit pipeline_tag: image-to-image library_name: onnx tags: - onnx - moonbox - moebius - inpainting - object-removal - image-to-image - diffusion --- # Moebius ONNX ONNX builds of [Moebius](https://huggingface.co/hustvl/Moebius), made by Moonbox for on-device use with ONNX Runtime (CUDA, DirectML, CPU). Each precision is a folder with one graph per part of the pipeline; large weights sit next to each graph as ONNX external data (`*.onnx.data`), so keep a folder's files together. ## Files The last column is how far each graph's output is from the original PyTorch model run in fp32 on the same inputs (relative RMS error). fp32 is the yardstick because the original's own bf16 run is not exact either. Image-quality scores are added as each precision passes the image eval. | Precision | File | Size | sha256 | Checked against fp32 | |---|---|---|---|---| | fp16 | [`fp16/unet/model.onnx`](https://huggingface.co/moonbox-dev/Moebius-ONNX/blob/main/fp16/unet/model.onnx) | 0.91 GB | `3ab9796281da` | fp32 graph (fp16 compute: 10 % off) | | fp16 | [`fp16/vae_decoder/model.onnx`](https://huggingface.co/moonbox-dev/Moebius-ONNX/blob/main/fp16/vae_decoder/model.onnx) | 0.10 GB | `2818a7cbd433` | 1.9e-04 | | fp16 | [`fp16/vae_encoder/model.onnx`](https://huggingface.co/moonbox-dev/Moebius-ONNX/blob/main/fp16/vae_encoder/model.onnx) | 0.07 GB | `8e37212ea885` | 6.3e-05 | | int8 | [`int8/unet/model.onnx`](https://huggingface.co/moonbox-dev/Moebius-ONNX/blob/main/int8/unet/model.onnx) | 0.27 GB | `6bed99be95b9` | 2.3 % (int8 weights) | | int8 | [`int8/vae_decoder/model.onnx`](https://huggingface.co/moonbox-dev/Moebius-ONNX/blob/main/int8/vae_decoder/model.onnx) | 0.10 GB | `2818a7cbd433` | 1.9e-04 (fp16 graph) | | int8 | [`int8/vae_encoder/model.onnx`](https://huggingface.co/moonbox-dev/Moebius-ONNX/blob/main/int8/vae_encoder/model.onnx) | 0.07 GB | `8e37212ea885` | 6.3e-05 (fp16 graph) | ## How to run Moebius (0.22 B) is an image-inpainting / object-removal model: give it a picture and a mask (white = remove / repaint) and it fills the masked region from the surroundings. No text prompt. These folders hold its three graphs at 512 px (the short side the original pipeline uses): `vae_encoder`, `unet` (with classifier-free guidance and the learned prompt embeddings inside) and `vae_decoder`. The VAE is PixelHacker's (the SDXL VAE, scaling 0.13025); it stays fp32 where fp16 overflows (see the table). **Run it** (ONNX Runtime + diffusers' DDIM scheduler; the original's defaults: 20 steps at strength 0.99, CFG 2.5, noise offset 0.0357, result pasted back under a blurred mask): ```python import numpy as np, onnxruntime as ort, torch from diffusers import DDIMScheduler from PIL import Image, ImageFilter d, prov = "int8", ["CUDAExecutionProvider", "CPUExecutionProvider"] enc, unet, dec = (ort.InferenceSession(f"{d}/{p}/model.onnx", providers=prov) for p in ("vae_encoder", "unet", "vae_decoder")) img = Image.open("photo.jpg").convert("RGB").resize((512, 512), Image.LANCZOS) # graphs are 512 x 512 mask = Image.open("mask.png").convert("L").resize((512, 512), Image.NEAREST) x = np.asarray(img, np.float32).transpose(2, 0, 1)[None] / 127.5 - 1 m = (np.asarray(mask, np.float32)[None, None] / 255 >= 0.5).astype(np.float32) lat, mlat = enc.run(None, {"image": x})[0], enc.run(None, {"image": x * (1 - m)})[0] m64 = m[:, :, ::8, ::8].copy() s = DDIMScheduler(beta_start=0.00085, beta_end=0.012, beta_schedule="scaled_linear", clip_sample=False) s.set_timesteps(20); ts = s.timesteps[1:] # strength 0.99: 19 steps g = torch.Generator().manual_seed(0) noise = torch.randn(lat.shape, generator=g) + 0.0357 * torch.randn((1, 4, 1, 1), generator=g) z = s.add_noise(torch.from_numpy(lat), noise, ts[:1]) for t in ts: eps = unet.run(None, {"latent": z.numpy(), "mask": m64, "masked_latent": mlat, "timestep": np.array([float(t)], np.float32), "guidance": np.array([2.5], np.float32)})[0] z = s.step(torch.from_numpy(eps), t, z).prev_sample y = dec.run(None, {"latent": (z.numpy() / 0.13025).astype(np.float32)})[0][0] out = Image.fromarray((np.clip((y.transpose(1, 2, 0) + 1) / 2, 0, 1) * 255).astype(np.uint8)) blur = np.asarray(mask.convert("RGB").filter(ImageFilter.GaussianBlur(3)), np.float32) / 255 Image.fromarray((np.asarray(out) * blur + np.asarray(img) * (1 - blur)).astype(np.uint8)).save("filled.png") ``` **Precisions.** fp16: every graph in fp16 where it stays within 5 % of fp32 on a test input, otherwise fp32 (the table says which). int8: the UNet's weights int8 per output channel (weight-only), VAE graphs as in fp16. int4: the UNet's MatMul weights 4-bit (MatMulNBits, block 32) and its other weights int8, VAE graphs as in fp16. No bf16 (no bf16 convolution in ONNX Runtime's CUDA provider). **Benchmark.** BSD100's 100 photos with seeded free-form masks (10-26 % of the picture), the pipeline above on an RTX 3080 Laptop GPU with ONNX Runtime 1.30: PSNR / SSIM / LPIPS against the original photo and PSNR against the fp32 pipeline's result (same seed). Inpainting has no single right answer, so the ground-truth numbers are for comparing precisions, not a quality score. | Pipeline | Size | vs fp32 (PSNR, same seed) | PSNR / SSIM / LPIPS vs original | Time / picture | Peak VRAM | |---|---|---|---|---|---| | fp32 (reference, not published) | 1.24 GB | - | 26.35 / 0.906 / 0.067 | 3.8 s | 7.8 GB | | fp16 (UNet fp32) | 1.07 GB | 46.5 dB (worst 35.6) | 26.35 / 0.906 / 0.067 | 3.6 s | 5.9 GB | | int8 | 0.44 GB | 40.3 dB (worst 32.5) | 26.38 / 0.906 / 0.066 | 3.9 s | 5.9 GB | ## How these were made From [Moebius](https://huggingface.co/hustvl/Moebius) at revision `cd01f47fb6`: each part exported with `torch.onnx`, then converted or quantized with ONNX Runtime's tools (fp16 / bf16 casts, int8 and int4 weight quantization). Every graph was checked against the original before upload; the `sha256` column above is what the files must hash to. ## License MIT (weights; the GitHub code is Apache-2.0). These files are a derivative of the original model and carry its license: it is copied in this repository, and it governs your use of these files. The original model card follows unchanged. --- # Original model card 《Moebius: 0.2B Lightweight Image Inpainting Framework with 10B-Level Performance》 Project Page: https://hustvl.github.io/Moebius Paper: https://arxiv.org/abs/2606.19195 Code: https://github.com/hustvl/Moebius weight: https://huggingface.co/hustvl/Moebius/tree/main