Moebius-MLX / README.md
jennyxue's picture
Model card and the original license
441ac81 verified
|
Raw History Blame Contribute Delete
6.18 kB
---
base_model: hustvl/Moebius
base_model_relation: quantized
license: mit
pipeline_tag: image-to-image
library_name: mlx
tags:
- mlx
- moonbox
- moebius
- inpainting
- object-removal
---
# Moebius MLX
MLX builds of [Moebius](https://huggingface.co/hustvl/Moebius), made by Moonbox for Apple Silicon Macs. Each precision is a folder; keep a folder's
files together. Only the parts listed below are converted: the rest of the pipeline comes from the original
repository unless the section below says otherwise.
## Files
The last column is how far each build is from the original PyTorch model on the same inputs (see the section below
for what was measured).
| Precision | File | Size | sha256 | Checked against the original |
|---|---|---|---|---|
| bf16 | [`bf16/temb.npz`](https://huggingface.co/moonbox-ai/Moebius-MLX/blob/main/bf16/temb.npz) | 5.1 MB | `b5b4a28cb0ba` | PSNR 32.2 dB mean / 27.2 worst, LPIPS <= 0.050 |
| bf16 | [`bf16/unet.safetensors`](https://huggingface.co/moonbox-ai/Moebius-MLX/blob/main/bf16/unet.safetensors) | 476.8 MB | `13440a538271` | PSNR 32.2 dB mean / 27.2 worst, LPIPS <= 0.050 |
| bf16 | [`bf16/vae.safetensors`](https://huggingface.co/moonbox-ai/Moebius-MLX/blob/main/bf16/vae.safetensors) | 167.3 MB | `20493a16c12c` | PSNR 32.2 dB mean / 27.2 worst, LPIPS <= 0.050 |
| fp16 | [`fp16/temb.npz`](https://huggingface.co/moonbox-ai/Moebius-MLX/blob/main/fp16/temb.npz) | 5.1 MB | `b5b4a28cb0ba` | PSNR 41.0 dB mean / 33.9 worst, LPIPS <= 0.008 |
| fp16 | [`fp16/unet.safetensors`](https://huggingface.co/moonbox-ai/Moebius-MLX/blob/main/fp16/unet.safetensors) | 476.8 MB | `e75b9e1549e4` | PSNR 41.0 dB mean / 33.9 worst, LPIPS <= 0.008 |
| fp16 | [`fp16/vae.safetensors`](https://huggingface.co/moonbox-ai/Moebius-MLX/blob/main/fp16/vae.safetensors) | 167.3 MB | `81593a7667f4` | PSNR 41.0 dB mean / 33.9 worst, LPIPS <= 0.008 |
| 8bit | [`8bit/temb.npz`](https://huggingface.co/moonbox-ai/Moebius-MLX/blob/main/8bit/temb.npz) | 5.1 MB | `b5b4a28cb0ba` | PSNR 37.4 dB mean / 33.0 worst, LPIPS <= 0.022 |
| 8bit | [`8bit/unet.safetensors`](https://huggingface.co/moonbox-ai/Moebius-MLX/blob/main/8bit/unet.safetensors) | 322.2 MB | `57339be34c7f` | PSNR 37.4 dB mean / 33.0 worst, LPIPS <= 0.022 |
| 8bit | [`8bit/vae.safetensors`](https://huggingface.co/moonbox-ai/Moebius-MLX/blob/main/8bit/vae.safetensors) | 167.3 MB | `c7eaffa5125e` | PSNR 37.4 dB mean / 33.0 worst, LPIPS <= 0.022 |
| 6bit | [`6bit/temb.npz`](https://huggingface.co/moonbox-ai/Moebius-MLX/blob/main/6bit/temb.npz) | 5.1 MB | `b5b4a28cb0ba` | PSNR 32.9 dB mean / 26.4 worst, LPIPS <= 0.066 |
| 6bit | [`6bit/unet.safetensors`](https://huggingface.co/moonbox-ai/Moebius-MLX/blob/main/6bit/unet.safetensors) | 281.0 MB | `f4d5da54ee83` | PSNR 32.9 dB mean / 26.4 worst, LPIPS <= 0.066 |
| 6bit | [`6bit/vae.safetensors`](https://huggingface.co/moonbox-ai/Moebius-MLX/blob/main/6bit/vae.safetensors) | 167.3 MB | `4d8ad4179f1c` | PSNR 32.9 dB mean / 26.4 worst, LPIPS <= 0.066 |
| 4bit | [`4bit/temb.npz`](https://huggingface.co/moonbox-ai/Moebius-MLX/blob/main/4bit/temb.npz) | 5.1 MB | `b5b4a28cb0ba` | below threshold (experimental): PSNR 22.4 dB mean / 16.7 worst, LPIPS <= 0.380 |
| 4bit | [`4bit/unet.safetensors`](https://huggingface.co/moonbox-ai/Moebius-MLX/blob/main/4bit/unet.safetensors) | 239.8 MB | `775f131ac8b5` | below threshold (experimental): PSNR 22.4 dB mean / 16.7 worst, LPIPS <= 0.380 |
| 4bit | [`4bit/vae.safetensors`](https://huggingface.co/moonbox-ai/Moebius-MLX/blob/main/4bit/vae.safetensors) | 167.3 MB | `2c9ec84a4993` | below threshold (experimental): PSNR 22.4 dB mean / 16.7 worst, LPIPS <= 0.380 |
## How to run
Moebius (the `pretrained` general model) with the PixelHacker VAE, converted for the MLX port in Moonbox's Moebius
engine. Each folder holds:
| File | What |
|---|---|
| `unet.safetensors` | the 0.22 B UNet: BatchNorms, norm affines and the constant conditioning folded into the weights; 1x1 convs and attention projections stored as `[in, out]` matrices (MLX affine-quantized, group 64, in the `8bit` / `6bit` / `4bit` folders: `<name>.q`, `.scales`, `.biases`) |
| `vae.safetensors` | PixelHacker's VAE (fp16 in the quantized folders) |
| `temb.npz` | `silu(time_embedding(t))` for t = 0..999 |
The layout is the one `moebius_od/mlx_unet.py` and `mlx_vae.py` in Moonbox read (NHWC, 512x512, 20 DDIM steps, CFG 2,
as upstream's README). That runtime isn't part of this repository.
**How they were checked.** Through the same MLX engine against the float32 weights, removing the main subject of 8
photos (seed 0): PSNR and LPIPS of the 512 px output. Pass: LPIPS <= 0.15 on every image. The MLX port itself was
checked against the official PyTorch pipeline when the engine was written.
| Precision | Size | PSNR vs float32 (mean / worst) | LPIPS (worst) | Status |
|---|---|---|---|---|
| `fp16/` | 649 MB | 41.0 / 33.9 dB | 0.008 | passes |
| `bf16/` | 649 MB | 32.2 / 27.2 dB | 0.050 | passes |
| `8bit/` | 495 MB | 37.4 / 33.0 dB | 0.022 | passes |
| `6bit/` | 453 MB | 32.9 / 26.4 dB | 0.066 | passes |
| `4bit/` | 412 MB | 22.4 / 16.7 dB | 0.380 | **below threshold, experimental** |
About 4-5 s per inpainting (20 steps, 512 px) on an Apple M5.
## How these were made
From [Moebius](https://huggingface.co/hustvl/Moebius) at revision `cd01f47fb6`: the original weights loaded in MLX and stored as bf16 or fp16, or
quantized with MLX's affine quantization (`8bit`, `6bit`, `4bit`, group size 64; layers MLX can't quantize, such as
convolutions and norms, stay bf16). Every build was checked against the original before upload; the `sha256` column
above is what the files must hash to.
## License
Apache-2.0. These files are a derivative of the original model and carry its license: it is copied in this
repository, and it governs your use of these files. The original model card follows unchanged.
---
# Original model card
《Moebius: 0.2B Lightweight Image Inpainting Framework with 10B-Level Performance》
Project Page: https://hustvl.github.io/Moebius
Paper: https://arxiv.org/abs/2606.19195
Code: https://github.com/hustvl/Moebius
weight: https://huggingface.co/hustvl/Moebius/tree/main