--- license: cc-by-nc-4.0 base_model: briaai/Fibo-2.0-dev base_model_relation: quantized pipeline_tag: text-to-image library_name: diffusers language: - en tags: - orbitquant - w4a8 - 4-bit - gptq - quantized - fibo - vgl - structured-prompts - text-to-image - image-editing - image-to-image - arxiv:2607.02461 --- # Fibo 2.0 Dev OrbitQuant W4A8 [OrbitQuant](https://github.com/iamwavecut/OrbitQuant) 4-bit weights for [Fibo 2.0 Dev](https://huggingface.co/briaai/Fibo-2.0-dev), the 8B text-to-image and image-editing model of BRIA AI that reads structured JSON captions (VGL). The transformer and the language model of the Qwen3-VL-4B text encoder both run 8-bit activations over 4-bit weights. The 4-bit codes were chosen with GPTQ rounding against calibration captions that are not in the evaluation set. The runtime is the original `BriaFibo2Pipeline` of Diffusers (50 steps, guidance 5) and the ComfyUI-OrbitQuant nodes. The bundle is 8.1 GB. The original is 25.5 GB. On one RTX 5090 a 1024×1024 image (50 steps with guidance 5) takes 20.4 s with 11.4 GiB of device memory at the peak; the original in BF16 takes 46.2 s and 27.6 GiB. With `--offload` the peak is 5.9 GiB and it takes 22.7 s. The [Performance](#performance) section has the details. ![Generation time, device memory peak and checkpoint size: BF16 original and OrbitQuant W4A8 on one RTX 5090](assets/latency_and_memory.png) ## Usage Use Linux, Python 3.12 and an NVIDIA GPU with compute capability 8.0 or newer. The tested stack is Torch 2.12.1 with CUDA 13.0 on an RTX 5090. ```bash python3.12 -m venv .venv && . .venv/bin/activate pip install torch==2.12.1 torchvision==0.27.1 --index-url https://download.pytorch.org/whl/cu130 pip install huggingface_hub hf download WaveCut/Fibo-2.0-dev-OrbitQuant-W4A8 --local-dir fibo2-orbitquant cd fibo2-orbitquant pip install -r requirements.txt ``` Fibo 2.0 reads a structured VGL caption: a JSON object with `short_description`, `objects`, `background_setting`, `lighting`, `aesthetics`, `photographic_characteristics`, `style_medium` and, for text in the image, `text_render`. [`bria-vgl`](https://github.com/Bria-AI/vgl) writes such captions from plain language, and the [caption guide](https://huggingface.co/briaai/Fibo-2.0-dev/blob/main/vgl/README.md) of the original describes every field. Plain text also runs, with weaker results. Text to image, with the caption in `caption.json`: ```bash python run_fibo2.py --prompt caption.json --width 1024 --height 1024 --seed 42 --out image.png ``` Edit: write the caption of the result with an `edit_instruction` and pass the images (up to 4) and an optional mask: ```bash python run_fibo2.py --prompt edit.json --image photo.jpg --out edited.png python run_fibo2.py --prompt edit.json --image photo.jpg --mask mask.png --out edited.png ``` Python, inside `fibo2-orbitquant`: ```python import json import torch from fibo2_orbitquant import NEGATIVE_T2I, fonts, load_pipeline pipe = load_pipeline(".") caption = json.load(open("caption.json")) image = pipe(prompt=json.dumps(caption), negative_prompt=NEGATIVE_T2I, width=1024, height=1024, num_inference_steps=50, guidance_scale=5.0, fonts=fonts("."), generator=torch.Generator("cpu").manual_seed(42)).images[0] image.save("image.png") ``` `load_pipeline` returns the original `BriaFibo2Pipeline`; call it with the original arguments (`image=` and `mask=` for edits; `NEGATIVE_EDIT` is the negative caption for edits). Options of `run_fibo2.py` (`load_pipeline` takes the same settings as arguments): - `--offload`: keep the transformer, the text encoder and the autoencoder in host memory. Each moves to the GPU for its own stage. - `--attention int8`: INT8 Q·K attention. It takes 18.5 s instead of 20.4 s here and changes the image slightly. - `--width`, `--height` (multiples of 16), `--steps`, `--cfg`, `--seed`, `--negative-prompt`. Sampling defaults follow the original model card: 50 steps, guidance 5 with the card's negative caption for text to image and the empty caption for edits. Without `--width` and `--height`, an edit keeps the aspect ratio of the first image at about one megapixel. ### ComfyUI [ComfyUI-OrbitQuant](https://github.com/iamwavecut/ComfyUI-OrbitQuant) 0.9.0 lists this release as `WaveCut/Fibo-2.0-dev-OrbitQuant-W4A8`. Connect **OrbitQuant Model Loader** to **OrbitQuant Generate Image** and save the result with a standard save node. The prompt is the JSON caption. For an edit, connect a **Load Image** node to `reference_images`. The node package includes the example workflow `Fibo 2.0 dev - image.json`. The generator runs in the Python environment that `python_executable` selects. Install the packages of the Usage section there. - `profile`: `default`, `model-offload` or `int8-attention`. - `keep_loaded` (on by default): the worker keeps the model for the next request. On the RTX 5090 the first 1024×1024 image took 30.3 s with loading, the next image 21.2 s and an edit with the kept model 53.5 s. - `steps`, `width` and `height` set to 0 use the defaults of the Usage section. Through ComfyUI v0.42.1 the device peak with `default` was 12.7 GiB for text to image and 12.8 GiB for the edit, including the 1.0 GiB that the idle ComfyUI server holds. `model-offload` took 39.3 s in a new worker. With `default` and `model-offload`, the images were pixel-identical to the output of `run_fibo2.py` for the same caption, image and seed. ## Quality All numbers compare against the original BF16 model with the same caption, seed, image and sampler. LPIPS (AlexNet) is lower when two images are closer. | Measure | Result | | --- | --- | | Mean LPIPS to BF16, 12 text-to-image captions | 0.187 | | Same with nearest rounding (no calibration) | 0.256 | | BF16 weights through the fused kernels, same captions | 0.011 | | Mean LPIPS to BF16, 8 edits | 0.037 | | Same with nearest rounding | 0.070 | Over 50 steps with guidance 5, the rounding noise of the 4-bit weights and 8-bit activations moves objects and poses, so the text-to-image LPIPS mostly measures composition drift; the 20 images show no visible noise, banding or broken text. The fused kernels alone do not cause it: with the unchanged BF16 weights they stay at the floor in the table. The image to edit anchors the layout, so edits stay much closer to BF16. Every caption with text (the shop sign, the neon sign, the travel poster, the Cyrillic chalkboard and the whiteboard) rendered the same words as BF16. On the chalkboard BF16 misspells both dishes and W4A8 one of them. The text encoder's six hidden-state bundles, which condition the transformer, reach a mean cosine of 0.996 to BF16 over the 20 evaluation captions (0.982 with nearest rounding). [![BF16 original and OrbitQuant W4A8 for four text-to-image captions](assets/original_vs_orbitquant_w4a8_preview.webp)](assets/original_vs_orbitquant_w4a8.webp) [![BF16 original and OrbitQuant W4A8 for four edits](assets/edits_original_vs_orbitquant_w4a8_preview.webp)](assets/edits_original_vs_orbitquant_w4a8.webp) Click a preview for all cases. `assets/evaluation_cases.json` holds the 12 captions and 8 edits with their seeds and sizes. The images to edit are crops of the original model card's examples, plus one BF16 output for the sign edit. ## Performance One RTX 5090 (575 W), Torch 2.12.1 with CUDA 13.0. One process per configuration, batch size 1. Time is the median of 3 runs after 1 warm-up run; it covers prompt encoding, sampling and VAE decoding, with CUDA synchronized at both ends. Device peak is the total from NVML at 5 ms intervals, including the CUDA context. Tensor peak is the PyTorch allocator peak. Host RAM is the peak resident set of the process, including loading. Text to image, 1024×1024, 50 steps, guidance 5: | Configuration | Time | Device peak | Tensor peak | Host RAM peak | Checkpoint | | --- | ---: | ---: | ---: | ---: | ---: | | **OrbitQuant W4A8** | **20.4 s** | **11.4 GiB** | **9.2 GiB** | **5.0 GiB** | **8.1 GB** | | OrbitQuant W4A8, INT8 attention | 18.5 s | 11.4 GiB | 9.2 GiB | 5.0 GiB | 8.1 GB | | **OrbitQuant W4A8, offload** | **22.7 s** | **5.9 GiB** | **4.8 GiB** | **12.6 GiB** | **8.1 GB** | | BF16 (original) | 46.2 s | 27.6 GiB | 26.1 GiB | 10.7 GiB | 25.5 GB | Edits in the evaluation runs (one process per set, every edit once, 50 steps, guidance 5): the 6 edits that both runs timed took 52.3 s each (median, 49.5–90.6 s) with W4A8 and 110.9 s (106.8–178.7 s) with BF16; device peaks 12.3 and 29.0 GiB. The loader enables expandable allocator segments when `PYTORCH_CUDA_ALLOC_CONF` is empty. With `--offload`, each component moves to the GPU for its own stage, so the device peak is that of the largest stage. Other GPUs are untested. ## Components | Component | Precision | Size | | --- | --- | ---: | | Transformer, 8.1B parameters (44 blocks, 2 refiner blocks, 2 Perceiver layers) | OrbitQuant 4-bit weights with GPTQ rounding and 8-bit activations for the 364 projections of the self-attention, the cross-attention of the five injection blocks, the Perceiver and the SwiGLU layers. BF16 embedders, time embedding, per-block modulation, norms and output layer. | 4.33 GB | | Text encoder, Qwen3-VL-4B | OrbitQuant 4-bit weights with GPTQ rounding and 8-bit activations for the 252 projections of the language model. BF16 vision tower (it encodes the images to edit and the drawn caption texts), token embeddings and norms. | 3.43 GB | | Autoencoder (FLUX.2) | FP32 file of the original, loaded in BF16 as in the original pipeline | 0.34 GB | The loader drops the LM head and the last decoder layer of the text encoder: the pipeline reads hidden states up to layer 34 and never the logits. The cut does not change the result: with BF16 weights, the cut and the full encoder give bit-identical caption embeddings for all 20 evaluation cases. The 0.8 GB token-embedding table stays in host memory, since a caption reads a few hundred of its 152k rows. GPTQ calibration used 8 text-to-image captions and 4 edits written for calibration, none of them in the evaluation set: 12 of the 50 sampling steps of each BF16 run, both guidance passes (288 transformer calls) for the transformer; the same captions, three random field subsets of each and the two negative captions (26,586 tokens, with the images to edit) for the text encoder. GPTQ keeps the codebook, rotation, bit width and file format of plain OrbitQuant; `transformer/gptq-info.json` and `text_encoder/gptq-info.json` list the error of each projection group before and after it. ## Files - `transformer/`: the packed transformer (`model.safetensors`), its codebooks, rotations, OrbitQuant configuration and manifest, and the original `config.json`. `gptq-info.json` lists the error of each projection group before and after GPTQ rounding. - `text_encoder/`: the quantized Qwen3-VL model with its BF16 vision tower. `gptq-info.json` lists the GPTQ errors. `vae/`, `processor/`, `scheduler/`, `fonts/`, `styles/`: the files of the original model. - `fibo2_orbitquant.py`: the loader. `run_fibo2.py`: the command-line entry point. The fused transformer blocks come from `orbitquant` 0.15. - `manifest.json`: the components and the runtime pins. `SHA256SUMS`: the checksums of all files. - `LICENSE`: CC BY-NC 4.0. `NOTICE.md`: attribution and changes. - `assets/`: the images of this card. ## Limits and license - The fused blocks run one sample at a time, as the pipeline calls the transformer. - Fibo 2.0 expects VGL captions. Plain-text prompts run, but the model was trained on structured captions. - An image runs 100 transformer calls (50 steps, two guidance passes). For fast edits, [Fibo 2.0 Edit Turbo OrbitQuant W4A8](https://huggingface.co/WaveCut/Fibo-2.0-edit-turbo-OrbitQuant-W4A8) needs 4. - The evaluation covers 12 captions and 8 edits at one seed each; rarer scripts, long text and small text were not measured. This release is a quantized derivative of [Fibo 2.0 Dev](https://huggingface.co/briaai/Fibo-2.0-dev) by BRIA AI and is licensed under CC BY-NC 4.0 (see `LICENSE` and `NOTICE.md`), like the original. Use is limited to non-commercial purposes; commercial use of Fibo 2.0 requires an agreement with BRIA AI: . BRIA AI did not create, endorse or validate this release.