base_model: black-forest-labs/FLUX.1-Fill-dev
base_model_relation: quantized
license: other
license_name: flux-1-dev-non-commercial-license
license_link: LICENSE.md
pipeline_tag: image-to-image
library_name: mlx
tags:
- mlx
- moonbox
- flux
- inpainting
- outpainting
- flux-fill
FLUX.1 Fill Dev MLX
MLX builds of FLUX.1 Fill Dev, made by Moonbox for Apple Silicon Macs. Each precision is a folder; keep a folder's files together. Only the parts listed below are converted: the rest of the pipeline comes from the original repository unless the section below says otherwise.
Files
The last column is how far each build is from the original PyTorch model on the same inputs (see the section below for what was measured).
| Precision | File | Size | sha256 | Checked against the original |
|---|---|---|---|---|
| 8bit | 8bit/mflux_model.json |
0.4 kB | f4ff5fbab1ee |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/text_encoder/0.safetensors |
130.9 MB | 29daf483b500 |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/text_encoder/model.safetensors.index.json |
25.0 kB | 2496ca8b591e |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/text_encoder_2/0.safetensors |
2.15 GB | c93d0244058a |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/text_encoder_2/1.safetensors |
2.15 GB | 49b24e32092e |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/text_encoder_2/2.safetensors |
767.7 MB | 4a896dfbcca7 |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/text_encoder_2/model.safetensors.index.json |
44.1 kB | 310dde2f9694 |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/tokenizer/tokenizer.json |
3.6 MB | 6d9109cc8389 |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/tokenizer/tokenizer_config.json |
0.4 kB | d0eda7377bb9 |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/tokenizer_2/tokenizer.json |
2.4 MB | 2432e8aa8341 |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/tokenizer_2/tokenizer_config.json |
2.5 kB | e31ed19bc545 |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/transformer/0.safetensors |
2.14 GB | 4c1b79237ceb |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/transformer/1.safetensors |
2.13 GB | 56a82a0053d3 |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/transformer/2.safetensors |
2.13 GB | b2b5f93d2ea6 |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/transformer/3.safetensors |
2.13 GB | 09efe1817336 |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/transformer/4.safetensors |
2.11 GB | 632c4607b56b |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/transformer/5.safetensors |
2.03 GB | 68a5b71bbc50 |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/transformer/model.safetensors.index.json |
0.1 MB | a47d31736f65 |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/vae/0.safetensors |
165.7 MB | 6ddcda403d28 |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 8bit | 8bit/vae/model.safetensors.index.json |
17.4 kB | 0a98a344cba2 |
reference build: not compared with bf16 (doesn't run in 24 GB); inpaints correctly on the 3 photos |
| 6bit | 6bit/mflux_model.json |
0.4 kB | 9171d42971d1 |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/text_encoder/0.safetensors |
100.2 MB | 021972a101b1 |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/text_encoder/model.safetensors.index.json |
25.0 kB | bb76a6a32514 |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/text_encoder_2/0.safetensors |
2.15 GB | d48839ac576b |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/text_encoder_2/1.safetensors |
1.72 GB | 4a9a542456a7 |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/text_encoder_2/model.safetensors.index.json |
44.1 kB | ac2d435419f7 |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/tokenizer/tokenizer.json |
3.6 MB | 6d9109cc8389 |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/tokenizer/tokenizer_config.json |
0.4 kB | d0eda7377bb9 |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/tokenizer_2/tokenizer.json |
2.4 MB | 2432e8aa8341 |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/tokenizer_2/tokenizer_config.json |
2.5 kB | e31ed19bc545 |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/transformer/0.safetensors |
2.12 GB | ff829e699345 |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/transformer/1.safetensors |
2.15 GB | 10eb8c736234 |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/transformer/2.safetensors |
2.13 GB | e551955f75c7 |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/transformer/3.safetensors |
2.14 GB | f2c528bcbb50 |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/transformer/4.safetensors |
1.14 GB | c7ee01b73467 |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/transformer/model.safetensors.index.json |
0.1 MB | 302702931b26 |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/vae/0.safetensors |
165.2 MB | 5fb147b7776d |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 6bit | 6bit/vae/model.safetensors.index.json |
17.4 kB | 20fb1f73bf5e |
below threshold (experimental): vs 8bit PSNR 37.8 dB mean, LPIPS 0.056 mean / 0.163 worst |
| 4bit | 4bit/mflux_model.json |
0.4 kB | 9a500615e1c2 |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/text_encoder/0.safetensors |
69.4 MB | 80401ef4d0cb |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/text_encoder/model.safetensors.index.json |
25.0 kB | 87918a51d110 |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/text_encoder_2/0.safetensors |
2.15 GB | daf9b170b819 |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/text_encoder_2/1.safetensors |
533.3 MB | 3db2a6e9309a |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/text_encoder_2/model.safetensors.index.json |
44.1 kB | 03f5a8751ab8 |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/tokenizer/tokenizer.json |
3.6 MB | 6d9109cc8389 |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/tokenizer/tokenizer_config.json |
0.4 kB | d0eda7377bb9 |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/tokenizer_2/tokenizer.json |
2.4 MB | 2432e8aa8341 |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/tokenizer_2/tokenizer_config.json |
2.5 kB | e31ed19bc545 |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/transformer/0.safetensors |
2.13 GB | dbf7be651fcf |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/transformer/1.safetensors |
2.14 GB | ae7fa499db8e |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/transformer/2.safetensors |
2.13 GB | 0bdccd27d8fb |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/transformer/3.safetensors |
297.6 MB | 6437f5d4cba1 |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/transformer/model.safetensors.index.json |
0.1 MB | ef05979b2043 |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/vae/0.safetensors |
164.7 MB | 53aa93d75594 |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
| 4bit | 4bit/vae/model.safetensors.index.json |
17.4 kB | 56348f08a8c9 |
vs 8bit PSNR 32.0 dB mean, LPIPS 0.051 mean / 0.107 worst |
How to run
Each folder is the whole FLUX.1 Fill [dev] pipeline (T5 and CLIP text encoders, transformer, VAE, tokenizers) saved by mflux 0.24.0 from the original weights at the revision below. Run it with mflux; the mask is white where the picture should be repainted:
pip install mflux==0.24.0
hf download moonbox-ai/FLUX.1-Fill-dev-MLX --include "8bit/*" --local-dir fill
mflux-generate-fill --model fill/8bit --base-model dev-fill --image-path photo.png --masked-image-path mask.png \
--prompt "an empty street" --steps 20 --seed 7 --output out.png
Add --low-ram on a 24 GB Mac. 8bit/, 6bit/ and 4bit/ quantize the transformer's and text encoders' linear
layers (MLX affine, group size 64); the VAE stays bf16.
No bf16/ folder. The lab's Mac (24 GB) can't run the 12B transformer unquantized (it thrashes swap), so a bf16
build couldn't be checked, and building it needs ~85 GB of free disk. The original repo's weights are already bf16
and mflux runs them directly (--model black-forest-labs/FLUX.1-Fill-dev).
How they were checked. Three photos with their subject masked out, 512x512, guidance 30, 20 steps, the same
seed: 6bit/ and 4bit/ against 8bit/ (nothing bigger fits in 24 GB). Pass: LPIPS <= 0.15 on every picture.
8bit/ itself wasn't compared with bf16; it fills the masked areas correctly.
| Precision | Size | PSNR vs 8bit (mean) | LPIPS (mean / worst) | Status |
|---|---|---|---|---|
8bit/ |
17.0 GB | reference | - | not compared with bf16 |
6bit/ |
13.8 GB | 37.8 dB | 0.056 / 0.163 | just below threshold, experimental |
4bit/ |
9.6 GB | 32.0 dB | 0.051 / 0.107 | passes |
About 5 minutes per 512x512 picture (20 steps) for 8bit/ on an Apple M5 with 24 GB in low-RAM mode.
How these were made
From FLUX.1 Fill Dev at revision 358293da03: the original weights loaded in MLX and stored as bf16 or fp16, or
quantized with MLX's affine quantization (8bit, 6bit, 4bit, group size 64; layers MLX can't quantize, such as
convolutions and norms, stay bf16). Every build was checked against the original before upload; the sha256 column
above is what the files must hash to.
License
FLUX.1-dev Non-Commercial (gated). These files are a derivative of the original model and carry its license: it is copied in this repository, and it governs your use of these files. The original model card follows unchanged.
Original model card
FLUX.1 Fill [dev] is a 12 billion parameter rectified flow transformer capable of filling areas in existing images based on a text description.
For more information, please read our blog post.
Key Features
- Cutting-edge output quality, second only to our state-of-the-art model
FLUX.1 Fill [pro]. - Blends impressive prompt following with completing the structure of your source image.
- Trained using guidance distillation, making
FLUX.1 Fill [dev]more efficient. - Open weights to drive new scientific research, and empower artists to develop innovative workflows.
- Generated outputs can be used for personal, scientific, and commercial purposes as described in the
FLUX.1 [dev]Non-Commercial License.
Usage
We provide a reference implementation of FLUX.1 Fill [dev], as well as sampling code, in a dedicated github repository.
Developers and creatives looking to build on top of FLUX.1 Fill [dev] are encouraged to use this as a starting point.
API Endpoints
The FLUX.1 models are also available in our API bfl.ml
Diffusers
To use FLUX.1 Fill [dev] with the 🧨 diffusers python library, first install or upgrade diffusers
pip install -U diffusers
Then you can use FluxFillPipeline to run the model
import torch
from diffusers import FluxFillPipeline
from diffusers.utils import load_image
image = load_image("https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/cup.png")
mask = load_image("https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/cup_mask.png")
pipe = FluxFillPipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", torch_dtype=torch.bfloat16).to("cuda")
image = pipe(
prompt="a white paper cup",
image=image,
mask_image=mask,
height=1632,
width=1232,
guidance_scale=30,
num_inference_steps=50,
max_sequence_length=512,
generator=torch.Generator("cpu").manual_seed(0)
).images[0]
image.save(f"flux-fill-dev.png")
To learn more check out the diffusers documentation
Limitations
- This model is not intended or able to provide factual information.
- As a statistical model this checkpoint might amplify existing societal biases.
- The model may fail to generate output that matches the prompts.
- Prompt following is heavily influenced by the prompting-style.
- There may be slight-color shifts in areas that are not filled in
- Filling in complex textures may produce lines at the edges of the filled-area.
Out-of-Scope Use
The model and its derivatives may not be used
- In any way that violates any applicable national, federal, state, local or international law or regulation.
- For the purpose of exploiting, harming or attempting to exploit or harm minors in any way; including but not limited to the solicitation, creation, acquisition, or dissemination of child exploitative content.
- To generate or disseminate verifiably false information and/or content with the purpose of harming others.
- To generate or disseminate personal identifiable information that can be used to harm an individual.
- To harass, abuse, threat