File size: 4,257 Bytes
f91cae8 3a6b18a dc65fcc 3a6b18a f91cae8 7bd7cb5 7db07bb | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 | ---
library_name: diffusers
pipeline_tag: text-to-image
base_model: krea/Krea-2-Turbo
tags:
- text-to-image
- image-to-image
- reference-image
- modular-diffusers
- diffusion
- krea
license: apache-2.0
---
# Krea 2 reference-image modular blocks
Custom [Modular Diffusers](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/overview) blocks that
let Krea 2 take reference images. Its text encoder is the full Qwen3-VL, vision tower included, so a reference
can condition the generation through the encoder's vision path without any extra model. Two community edit
LoRAs go further and feed the reference into the transformer as clean VAE latents.
`reference_mode` picks the path:
```
reference_mode="off" Qwen3-VL vision path only. No LoRA, works on the stock checkpoint
reference_mode="append" vision + clean reference tokens after the target. Ostris style-reference LoRA
reference_mode="prepend" vision + clean source before the target. Identity-Edit LoRA
```
With `reference_mode="off"` and no images, the blocks are stock text-to-image. No mode patches the
transformer.
## Loading & running
```python
import sdnq # needed to load the quantized text encoder
import torch
from diffusers import ModularPipeline
from diffusers.utils import load_image
pipe = ModularPipeline.from_pretrained(
"OzzyGT/krea2_reference_blocks", trust_remote_code=True
)
pipe.load_components(dtype=torch.bfloat16)
pipe.to("cuda")
image = pipe(
prompt="the same cat, sitting on a stone wall at sunset",
reference_images=load_image("cat.png"),
height=1024,
width=1024,
output="images",
)[0]
image.save("reference.png")
```
The LoRA modes need their adapter loaded first:
```python
pipe.load_lora_weights("ostris/krea2_turbo_style_reference", weight_name="krea2_style_reference.safetensors")
image = pipe(
prompt="the same subject in a snowy forest",
reference_images=load_image("subject.png"),
reference_mode="append",
output="images",
)[0]
```
For loading a different checkpoint or swapping components, see
[Modular pipeline](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/modular_pipeline) in the
diffusers docs.
## Reference inputs
```
reference_images one image or a list
reference_subject_strength scales content and subject. 1.0 default, 0 drops it
reference_style_strength scales texture and style. 1.0 default, 0 drops it
grounding_px longest side of the view the VLM sees, 768 default
```
Either strength takes one value for every reference, or a list matching them one to one, so a reference can
give its look without its content or the other way round. `grounding_px` is the identity-versus-adherence
dial: lower follows the prompt, higher holds the reference, `0` is native resolution.
## Masks
`reference_masks` restricts a reference to part of the picture, painted being the part to use. One mask per
image, or `None` for a slot without one.
`reference_mask_mode` decides what happens to the rest:
```
"exclude_blank" default. Mask the vision attention and blank the masked-out tokens
"exclude" mask the attention only, leaving the masked-out tokens in the sequence
"deemphasize" blank the tokens only, which attenuates the region instead of removing it
```
Masks always apply to the vision path. In the LoRA modes the clean reference latents are unmasked by default,
so the edit still sees the whole image and a masked edit can show hints of the unpainted parts. Set
`mask_reference_latents=True` to drop those patches from the sequence too.
These blocks only work with Krea 2, Turbo or raw.
## Credits
- [`ostris/krea2_turbo_style_reference`](https://huggingface.co/ostris/krea2_turbo_style_reference) -- the
style-reference LoRA behind `append`
- [`conradlocke/krea2-identity-edit`](https://huggingface.co/conradlocke/krea2-identity-edit) -- the
Identity-Edit LoRA behind `prepend`
- `append` follows the built-in Krea 2 edit arch in [ai-toolkit](https://github.com/ostris/ai-toolkit);
`prepend` follows `predict_velocity_edit` in
[krea2edit-trainer](https://github.com/lbouaraba/krea2edit-trainer), the third-party ai-toolkit extension
by Conrad Locke the Identity-Edit LoRA was trained with
|