Instructions to use ApacheOne/Open-Vision-Banana-4B-Seg-1Step-Lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use ApacheOne/Open-Vision-Banana-4B-Seg-1Step-Lora with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("FunAILab/Open-Vision-Banana", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("ApacheOne/Open-Vision-Banana-4B-Seg-1Step-Lora") prompt = "Turn this cat into a dog" input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png") image = pipe(image=input_image, prompt=prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
import torch
from diffusers import DiffusionPipeline
from diffusers.utils import load_image
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("FunAILab/Open-Vision-Banana", dtype=torch.bfloat16, device_map="cuda")
pipe.load_lora_weights("ApacheOne/Open-Vision-Banana-4B-Seg-1Step-Lora")
prompt = "Turn this cat into a dog"
input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png")
image = pipe(image=input_image, prompt=prompt).images[0]Open Vision Banana 4B Seg 1-Step LoRA
This adapter makes a prompted foreground mask in one denoising step with the Open Vision Banana 4B non-zero-shot transformer. It was trained for 1,000 optimizer updates on 200 ReasonSeg training images, with rank 64, alpha 64, and a 512²-pixel training area cap. Inference uses one conditional transformer pass, an Euler update from sigma 1 to 0, and VAE decoding. Use CFG 1.0.
Benchmarks
The 1,000-update LoRA uses one denoising step and CFG 1.0.
Training used 200 images from ReasonSeg's train split.
| Evaluation | Model | Images | Inference area cap | Steps ↓ | gIoU ↑ | cIoU ↑ | Cached s/image ↓ |
|---|---|---|---|---|---|---|---|
| ReasonSeg held-out validation | OVB 4B non-zero-shot · published baseline | 200 | 1024² pixels | 28 | 74.60 | — | — |
| ReasonSeg held-out validation | One-step LoRA | 200 | 1024² pixels | 1 | 76.89 | 74.79 | 2.46 |
| Open Images test · seed 42 | One-step LoRA | 200 | 512² pixels | 1 | 73.79 | 80.53 | 0.78 |
| Open Images test · seed 42 | One-step LoRA | 200 | 1024² pixels | 1 | 73.96 | 79.35 | 2.31 |
Bold values mark the better result within each benchmark where both rows have a
value. The ReasonSeg runs use the full separate val split and OVB's bundled
Gemma 4 31B query rewrites. The +2.29 gIoU difference uses the published
28-step result as the reference. The paper does not report matching cIoU or runtime.
Both Open Images runs use the same 200 test photos, direct object-name prompts, and starting noise seeds. The area caps preserve aspect ratio without upscaling. Open Images compares the two resolutions with the same adapter. The 28-step baseline in the table applies to ReasonSeg.
gIoU is mean foreground IoU per image. cIoU pools the foreground intersection and union across images. Cached time includes denoising, output VAE decoding, and image conversion. It excludes prompt encoding, input-image encoding, and setup.
OVB paper, Tables 3 and 5 · OVB evaluation protocol · Open Images annotations
Open Images test examples
The same test photo and annotation are shown next to the two one-step masks. IoU labels are per image.
Photo by Anne-Lise Heinrichs · CC BY 2.0.
Photo by Rachel Kramer Bussel · CC BY 2.0.
The panels resize the source photos and add captions, annotations, and predicted masks.
Load with Diffusers
This adapter uses the OVB 4B non-zero-shot transformer together with the FLUX.2 [klein] Base 4B pipeline components. For ReasonSeg-style questions, supply an explicit referring phrase such as “the black billiard ball.” The benchmark used the bundled Gemma query rewrites for that conversion.
import numpy as np
import torch
from PIL import Image
from diffusers import Flux2KleinPipeline, Flux2Transformer2DModel
transformer = Flux2Transformer2DModel.from_pretrained(
"FunAILab/Open-Vision-Banana",
revision="be5da66250cc385ff9c75ebbe4114e13ec305848",
subfolder="final_models/4B/nonzeroshot/checkpoint-avg90000-92500-97500-100000_uniform/transformer",
torch_dtype=torch.bfloat16,
)
pipe = Flux2KleinPipeline.from_pretrained(
"black-forest-labs/FLUX.2-klein-base-4B",
revision="a3b4f4849157f664bdbc776fd7453c2783562f4d",
transformer=transformer,
torch_dtype=torch.bfloat16,
)
pipe.load_lora_weights(
"ApacheOne/Open-Vision-Banana-4B-Seg-1Step-Lora",
weight_name="ovb_4b_reasonseg_one_step.safetensors",
adapter_name="seg_one_step",
)
# Preserve the trained adapter precision used for the benchmark.
for layer in pipe.transformer.modules():
if hasattr(layer, "lora_A") and "seg_one_step" in layer.lora_A:
layer.lora_A["seg_one_step"].to(dtype=torch.float32)
layer.lora_B["seg_one_step"].to(dtype=torch.float32)
pipe.to("cuda")
image = Image.open("input.jpg").convert("RGB")
expression = "the black billiard ball"
prompt = (
f'[TASK: REFERRING_SEGMENTATION] Segment the region that best matches this '
f'description: "{expression}". Paint the matching region white [255, 255, 255] '
'and leave everything else black [0, 0, 0]. Output only the segmentation mask image.'
)
raw = pipe(
image=image, prompt=prompt, num_inference_steps=1, guidance_scale=1.0,
sigmas=[1.0], generator=torch.Generator(device="cuda").manual_seed(0),
).images[0]
foreground = np.asarray(raw.convert("RGB"), dtype=np.float32).mean(axis=-1) > 127.5
mask = Image.fromarray(foreground.astype(np.uint8) * 255)
mask.resize(image.size, Image.Resampling.NEAREST).save("mask.png")
The example uses white for a simple preview. Benchmark inference used OVB's seeded named-color prompts and nearest-color decoding.
License and attribution
This is an independent modification of Open Vision Banana. See LICENSE.md for the FLUX Non-Commercial License and NOTICE.txt for the attribution notice. Rights to use the model and this derivative are granted directly by Black Forest Labs under that license. Cite the Open Vision Banana paper and ReasonSeg / LISA when using these results.
- Downloads last month
- -
Model tree for ApacheOne/Open-Vision-Banana-4B-Seg-1Step-Lora
Base model
black-forest-labs/FLUX.2-klein-base-9B

