Support averaged style references

API.md

@@ -26,7 +26,10 @@ Content-Type: application/json
 
 {
   "prompt": "an orange cat",
-  "style_image": "BASE64_ENCODED_PNG_OR_JPEG",
+  "style_images": [
+    "BASE64_ENCODED_IMAGE_1",
+    "BASE64_ENCODED_IMAGE_2"
+  ],
   "style_scope": "generation",
   "style_weight": 1,
   "style_end": 1,
@@ -75,7 +78,7 @@ Content-Type: application/json
 }
 ```
 
-Returns an encrypted `.epng` with ComfyUI workflow and parameter metadata, scaled after generation by the decimal `return_scale` from `0.01` through `1`, with content type `application/octet-stream`. `batch_size` generates from 1 through 8 images at once and defaults to `1`. Multiple landscape images are stacked top to bottom; portrait or square images are joined left to right. Regional prompts and their mask nodes are included in the metadata. Use `1` for full resolution. `regions` accepts up to three spatial prompts on a 5 by 5 grid. Set `regional_mode` to `attention` for Attention Couple (PPM), or omit it for soft conditioning. Regional prompts establish subjects from the first sampling step. The reduced-strength global prompt starts after the first fifth to complete the environment while regions remain active. Each internal region edge feathers by half a grid cell, so overlap adjacent regions by one cell for a smooth crossfade. `area` accepts `auto`, `full`, `tl`, `tc`, `tr`, `ml`, `mc`, `mr`, `bl`, `bc`, `br`, `th`, `mh`, `bh`, `lh`, `ch`, or `rh`. It also accepts any single cell from `a1` through `e5`, or a rectangular inclusive range such as `a1:c5`. `auto` places one prompt in the center, two left and right, or three left, center, and right. Columns run left to right and rows run top to bottom. Region strength defaults to `1`. Use the global `prompt` for environment, lighting, shared style, and composition outside regions. Omitting `regions` keeps whole-image generation unchanged. Set either scheduler to `AlignYourSteps` to use `AlignYourStepsScheduler` with fixed model type `SDXL` and the corresponding steps and denoise values. Only `prompt` is required for the built-in site defaults. `style_image` enables InstantStyle for SDXL and Illustrious checkpoints and accepts raw base64 or an image data URL up to 20 MiB. `style_weight` ranges from `0` through `5`, and `style_end` controls the fraction of sampling during which style guidance remains active. `style_scope` accepts `first`, `generation`, or `all`. `first` only styles the initial pass. `generation`, the default, styles the first pass and the upscale second pass while leaving ADetailer unconditioned to prioritize facial anatomy. `all` also styles every ADetailer pass and is useful when detailed regions visibly lose the reference style. A one-pass request treats `generation` as `first`. The exported ComfyUI workflow names the external reference `style-reference.png`. LoRAs and detailers run in array order. A blank detailer `prompt` or `negative` inherits the matching main prompt. Each request creates fresh sampler seeds. When `upscale` is true, the selected upscaler is normalized to the requested `upscale_scale` resolution before the second sampler. Leave `second_model` empty to reuse the first model and LoRA chain; otherwise `second_loras` applies to the selected second model. Width and height must be multiples of 8 from 64 through 2048; steps must be from 1 through 100.
+Returns an encrypted `.epng` with ComfyUI workflow and parameter metadata, scaled after generation by the decimal `return_scale` from `0.01` through `1`, with content type `application/octet-stream`. `batch_size` generates from 1 through 8 images at once and defaults to `1`. Multiple landscape images are stacked top to bottom; portrait or square images are joined left to right. Regional prompts and their mask nodes are included in the metadata. Use `1` for full resolution. `regions` accepts up to three spatial prompts on a 5 by 5 grid. Set `regional_mode` to `attention` for Attention Couple (PPM), or omit it for soft conditioning. Regional prompts establish subjects from the first sampling step. The reduced-strength global prompt starts after the first fifth to complete the environment while regions remain active. Each internal region edge feathers by half a grid cell, so overlap adjacent regions by one cell for a smooth crossfade. `area` accepts `auto`, `full`, `tl`, `tc`, `tr`, `ml`, `mc`, `mr`, `bl`, `bc`, `br`, `th`, `mh`, `bh`, `lh`, `ch`, or `rh`. It also accepts any single cell from `a1` through `e5`, or a rectangular inclusive range such as `a1:c5`. `auto` places one prompt in the center, two left and right, or three left, center, and right. Columns run left to right and rows run top to bottom. Region strength defaults to `1`. Use the global `prompt` for environment, lighting, shared style, and composition outside regions. Omitting `regions` keeps whole-image generation unchanged. Set either scheduler to `AlignYourSteps` to use `AlignYourStepsScheduler` with fixed model type `SDXL` and the corresponding steps and denoise values. Only `prompt` is required for the built-in site defaults. `style_images` enables InstantStyle for SDXL and Illustrious checkpoints and accepts one through four raw base64 images or image data URLs up to 20 MiB each. References are center-cropped and their embeddings are averaged, allowing varied subjects and palettes to contribute their shared style. `style_weight` ranges from `0` through `5`, and `style_end` controls the fraction of sampling during which style guidance remains active. `style_scope` accepts `first`, `generation`, or `all`. `first` only styles the initial pass. `generation`, the default, styles the first pass and the upscale second pass while leaving ADetailer unconditioned to prioritize facial anatomy. `all` also styles every ADetailer pass and is useful when detailed regions visibly lose the reference style. A one-pass request treats `generation` as `first`. The exported ComfyUI workflow names the external references `style-reference-1.png` through `style-reference-4.png`. LoRAs and detailers run in array order. A blank detailer `prompt` or `negative` inherits the matching main prompt. Each request creates fresh sampler seeds. When `upscale` is true, the selected upscaler is normalized to the requested `upscale_scale` resolution before the second sampler. Leave `second_model` empty to reuse the first model and LoRA chain; otherwise `second_loras` applies to the selected second model. Width and height must be multiples of 8 from 64 through 2048; steps must be from 1 through 100.
 
 ## Archive
 

app.py

@@ -51,7 +51,7 @@ from huggingface_hub import (
     download_bucket_files,
     list_bucket_tree,
 )
-from PIL import Image, ImageDraw, ImageFont
+from PIL import Image, ImageDraw, ImageFont, ImageOps
 from PIL.PngImagePlugin import PngInfo
 from pydantic import BaseModel, ConfigDict, Field
 from starlette.background import BackgroundTask
@@ -208,6 +208,8 @@ STYLE_SCOPES = ("first", "generation", "all")
 DEFAULT_STYLE_SCOPE = "generation"
 DEFAULT_STYLE_WEIGHT = 1
 DEFAULT_STYLE_END = 1
+MAX_STYLE_IMAGES = 4
+STYLE_IMAGE_SIZE = 1024
 MAX_STYLE_IMAGE_SIZE = 20 * MIB
 BUILTIN_ASSETS = {
     ("ipadapter", STYLE_IPADAPTER): (
@@ -469,7 +471,7 @@ class DirectRequest(BaseModel):
     regions: list[RegionRequest] = Field(default_factory=list, max_length=3)
     regional_mode: str = REGIONAL_MODES[0]
     detailers: list[DetailerRequest] = Field(default_factory=list)
-    style_image: str = ""
+    style_images: list[str] = Field(default_factory=list, max_length=MAX_STYLE_IMAGES)
     style_scope: str = DEFAULT_STYLE_SCOPE
     style_weight: float = Field(DEFAULT_STYLE_WEIGHT, ge=0, le=5)
     style_end: float = Field(DEFAULT_STYLE_END, gt=0, le=1)
@@ -944,7 +946,7 @@ def encode_positive(
 
 
 def has_style(request):
-    return bool(getattr(request, "style_image", ""))
+    return bool(getattr(request, "style_images", []))
 
 
 def wants_style(request):
@@ -961,22 +963,29 @@ def style_stage_enabled(request, stage):
     return request.style_scope == "all"
 
 
-def decode_style_image(value):
-    encoded = value.partition(",")[2] if value.startswith("data:") else value
-    if len(encoded) > (MAX_STYLE_IMAGE_SIZE + 2) // 3 * 4:
-        raise ValueError("InstantStyle image exceeds 20 MiB")
-    try:
-        data = base64.b64decode(encoded, validate=True)
-    except (ValueError, binascii.Error) as error:
-        raise ValueError("Invalid InstantStyle image") from error
-    if len(data) > MAX_STYLE_IMAGE_SIZE:
-        raise ValueError("InstantStyle image exceeds 20 MiB")
-    try:
-        with Image.open(BytesIO(data)) as image:
-            pixels = np.asarray(image.convert("RGB"), dtype=np.float32) / 255
-    except Exception as error:
-        raise ValueError("Invalid InstantStyle image") from error
-    return torch.from_numpy(pixels).unsqueeze(0)
+def decode_style_images(values):
+    images = []
+    for value in values:
+        encoded = value.partition(",")[2] if value.startswith("data:") else value
+        if len(encoded) > (MAX_STYLE_IMAGE_SIZE + 2) // 3 * 4:
+            raise ValueError("InstantStyle image exceeds 20 MiB")
+        try:
+            data = base64.b64decode(encoded, validate=True)
+        except (ValueError, binascii.Error) as error:
+            raise ValueError("Invalid InstantStyle image") from error
+        if len(data) > MAX_STYLE_IMAGE_SIZE:
+            raise ValueError("InstantStyle image exceeds 20 MiB")
+        try:
+            with Image.open(BytesIO(data)) as image:
+                image = ImageOps.fit(
+                    image.convert("RGB"),
+                    (STYLE_IMAGE_SIZE, STYLE_IMAGE_SIZE),
+                    Image.Resampling.LANCZOS,
+                )
+                images.append(np.asarray(image, dtype=np.float32) / 255)
+        except Exception as error:
+            raise ValueError("Invalid InstantStyle image") from error
+    return torch.from_numpy(np.stack(images))
 
 
 def load_style_pipeline(model):
@@ -1662,7 +1671,7 @@ def generate_images(request, from_api=True):
 
     regions = prepare_regions(request.regions)
     style_image = (
-        decode_style_image(request.style_image)
+        decode_style_images(request.style_images)
         if has_style(request)
         else None
     )
@@ -1700,7 +1709,10 @@ def generate_images(request, from_api=True):
 
         metadata_config = request.model_dump()
         if has_style(request):
-            metadata_config["style_image"] = "style-reference.png"
+            metadata_config["style_images"] = [
+                f"style-reference-{index}.png"
+                for index in range(1, len(request.style_images) + 1)
+            ]
         metadata = image_metadata(
             metadata_config,
             [first_seed, second_seed],
@@ -2176,14 +2188,17 @@ def add_detailer(
     ]
 
 
-def encode_style_image(image):
-    if image is None:
-        return ""
-    data = BytesIO()
-    image.convert("RGB").save(data, format="PNG")
-    if data.tell() > MAX_STYLE_IMAGE_SIZE:
-        raise gr.Error("InstantStyle image exceeds 20 MiB")
-    return base64.b64encode(data.getvalue()).decode()
+def encode_style_images(files):
+    files = files or []
+    if len(files) > MAX_STYLE_IMAGES:
+        raise gr.Error("InstantStyle accepts up to 4 images")
+    images = []
+    for file in files:
+        data = Path(file).read_bytes()
+        if len(data) > MAX_STYLE_IMAGE_SIZE:
+            raise gr.Error("InstantStyle image exceeds 20 MiB")
+        images.append(base64.b64encode(data).decode())
+    return images
 
 
 def generate(
@@ -2192,7 +2207,7 @@ def generate(
     regions,
     regional_mode,
     model,
-    style_image,
+    style_images,
     style_scope,
     style_weight,
     style_end,
@@ -2227,7 +2242,7 @@ def generate(
         negative=negative,
         model=model,
         loras=[],
-        style_image=encode_style_image(style_image),
+        style_images=encode_style_images(style_images),
         style_scope=style_scope,
         style_weight=style_weight,
         style_end=style_end,
@@ -3190,15 +3205,17 @@ with gr.Blocks(title="Image generation") as demo:
                 with gr.Accordion("InstantStyle", open=False):
                     gr.HTML(
                         "<model-note>"
-                        "SDXL and Illustrious only. The reference is center-cropped. "
-                        "The default styles both generation passes but leaves "
+                        "SDXL and Illustrious only. Add up to four references; "
+                        "their center crops are averaged. Use varied subjects and "
+                        "palettes. The default styles both generation passes but leaves "
                         "ADetailer focused on anatomy."
                         "</model-note>"
                     )
-                    style_image_input = gr.Image(
-                        label="Style reference",
-                        type="pil",
-                        image_mode="RGB",
+                    style_images_input = gr.File(
+                        label="Style references",
+                        file_count="multiple",
+                        file_types=["image"],
+                        type="filepath",
                     )
                     style_scope_input = gr.Dropdown(
                         [
@@ -3430,7 +3447,7 @@ with gr.Blocks(title="Image generation") as demo:
             regions_input,
             regional_mode_input,
             model_input,
-            style_image_input,
+            style_images_input,
             style_scope_input,
             style_weight_input,
             style_end_input,

workflow_api.py

@@ -21,6 +21,7 @@ DETAILER_DROP_SIZE = 10
 STYLE_PRESET = "PLUS (high strength)"
 STYLE_WEIGHT_TYPE = "style transfer"
 STYLE_EMBEDS_SCALING = "V only"
+STYLE_IMAGE_SIZE = 1024
 
 
 def upscale_size(width, height, scale):
@@ -668,7 +669,7 @@ def image_metadata(
     style_pipeline = None
 
     def style_enabled(stage):
-        if not config.get("style_image"):
+        if not config.get("style_images"):
             return False
         scope = config.get("style_scope", "generation")
         if stage == "first":
@@ -682,9 +683,22 @@ def image_metadata(
         if not style_enabled(stage):
             return model
         if style_image is None:
-            style_image = [add("LoadImage", {
-                "image": config["style_image"],
-            }), 0]
+            for name in config["style_images"]:
+                image = [add("LoadImage", {"image": name}), 0]
+                image = [add("ImageScale", {
+                    "image": image,
+                    "upscale_method": "lanczos",
+                    "width": STYLE_IMAGE_SIZE,
+                    "height": STYLE_IMAGE_SIZE,
+                    "crop": "center",
+                }), 0]
+                if style_image is None:
+                    style_image = image
+                else:
+                    style_image = [add("ImageBatch", {
+                        "image1": style_image,
+                        "image2": image,
+                    }), 0]
         if style_pipeline is None:
             loader = add("IPAdapterUnifiedLoader", {
                 "model": model,