Keep the source's other files, unchanged, for later

README.md

@@ -1,33 +1,33 @@
----
-license: mit
-library_name: transformers.js
-base_model: ZhengPeng7/BiRefNet_lite
-pipeline_tag: image-segmentation
-tags:
-  - onnx
-  - birefnet
-  - background-removal
-  - dichotomous-image-segmentation
-  - webgpu
-  - transformers.js
-  - onnxruntime-web
----
-
-# BiRefNet-lite 1024 (ONNX, WebGPU-runnable)
-
-BiRefNet-lite at 1024 × 1024 as an fp16 ONNX export whose graph is patched to run on WebGPU in the browser; the weights are unchanged. [Card Anvil](https://github.com/Card-Anvil) uses it with [Transformers.js](https://github.com/huggingface/transformers.js) to select the main subject of card art.
-
-This is an unchanged copy of the files Card Anvil loads from [`jiabins0303/birefnet-lite-1024-webgpu`](https://huggingface.co/jiabins0303/birefnet-lite-1024-webgpu) at commit [`1ad01ce`](https://huggingface.co/jiabins0303/birefnet-lite-1024-webgpu/tree/1ad01cef0f4101a285c5a3e0bd7f15597d93403f), kept here so the app loads files that can't change under it. See the source repo for how the graph was patched and verified.
-
-| File | Source path |
-| --- | --- |
-| `config.json` | `config.json` |
-| `preprocessor_config.json` | `preprocessor_config.json` |
-| `onnx/model_fp16.onnx` | `onnx/model_fp16.onnx` |
-
-## Credits and licence
-
-- Model: [BiRefNet](https://github.com/ZhengPeng7/BiRefNet) by Peng Zheng et al. ([`ZhengPeng7/BiRefNet_lite`](https://huggingface.co/ZhengPeng7/BiRefNet_lite)), from *Bilateral Reference for High-Resolution Dichotomous Image Segmentation* (CAAI AIR 2024).
-- WebGPU-runnable export: [`jiabins0303/birefnet-lite-1024-webgpu`](https://huggingface.co/jiabins0303/birefnet-lite-1024-webgpu).
-
-Licensed under the MIT License, as the original model is. See [`LICENSE`](LICENSE).
+---
+license: mit
+library_name: transformers.js
+base_model: ZhengPeng7/BiRefNet_lite
+pipeline_tag: image-segmentation
+tags:
+  - onnx
+  - birefnet
+  - background-removal
+  - dichotomous-image-segmentation
+  - webgpu
+  - transformers.js
+  - onnxruntime-web
+---
+
+# BiRefNet-lite 1024 (ONNX, WebGPU-runnable)
+
+BiRefNet-lite at 1024 × 1024 as an fp16 ONNX export whose graph is patched to run on WebGPU in the browser; the weights are unchanged. [Card Anvil](https://github.com/Card-Anvil) uses it with [Transformers.js](https://github.com/huggingface/transformers.js) to select the main subject of card art.
+
+These are unchanged copies of the files in [`jiabins0303/birefnet-lite-1024-webgpu`](https://huggingface.co/jiabins0303/birefnet-lite-1024-webgpu) at commit [`1ad01ce`](https://huggingface.co/jiabins0303/birefnet-lite-1024-webgpu/tree/1ad01cef0f4101a285c5a3e0bd7f15597d93403f), at the same paths, kept here so the app loads files that can't change under it.
+
+| Files | What they are | Loaded by Card Anvil |
+| --- | --- | --- |
+| `config.json`, `preprocessor_config.json`, `onnx/model_fp16.onnx` | The model | Yes |
+| `scripts/patch_split.py`, `scripts/patch_deform.py`, `scripts/verify_patch.py` | The scripts that rewrote the graph to run on WebGPU, and the check that the result matches the original | No, kept for later |
+| `SOURCE_README.md` | The source repo's README: how and why the graph was patched, and how it was verified | No |
+
+## Credits and licence
+
+- Model: [BiRefNet](https://github.com/ZhengPeng7/BiRefNet) by Peng Zheng et al. ([`ZhengPeng7/BiRefNet_lite`](https://huggingface.co/ZhengPeng7/BiRefNet_lite)), from *Bilateral Reference for High-Resolution Dichotomous Image Segmentation* (CAAI AIR 2024).
+- WebGPU-runnable export: [`jiabins0303/birefnet-lite-1024-webgpu`](https://huggingface.co/jiabins0303/birefnet-lite-1024-webgpu).
+
+Licensed under the MIT License, as the original model is. See [`LICENSE`](LICENSE).

SOURCE_README.md

@@ -0,0 +1,184 @@
+---
+license: mit
+library_name: transformers.js
+pipeline_tag: image-segmentation
+base_model: ZhengPeng7/BiRefNet_lite
+tags:
+  - onnx
+  - background-removal
+  - matting
+  - alpha-matting
+  - image-matting
+  - webgpu
+  - client-side
+  - in-browser
+  - transformers.js
+  - fp16
+---
+
+# BiRefNet-lite 1024 (ONNX, WebGPU-runnable)
+
+A structurally patched ONNX export of **BiRefNet-lite at 1024x1024** that runs on
+**onnxruntime-web's WebGPU execution provider**. The upstream export does not: it
+falls back to the CPU EP on most of its graph and dies of `std::bad_alloc`.
+
+**If you have tried to run BiRefNet at 1024 in a browser and hit an
+out-of-memory error, this repo is the fix, and the section below explains what
+was actually wrong.** The usual workaround is to drop to a 512 export. That
+works and it is a reasonable trade — but the OOM is not caused by the
+resolution, and you do not have to pay it.
+
+The weights are unchanged. Only the graph topology was rewritten, and both
+rewrites are proved **bit-identical** to the original on the CPU EP
+(`max|diff| = 0.000e+00` over the test set).
+
+## Production use
+
+This graph is the `best` quality tier in **[Backgroundless](https://backgroundless.io)**,
+a background remover where the model runs entirely in the visitor's browser —
+no upload, no server round-trip, no account. Both tiers are free to use, so the
+1024 graph is served to anyone whose adapter can run it.
+
+## Quick start (transformers.js)
+
+```js
+import { AutoModel } from '@huggingface/transformers';
+
+const model = await AutoModel.from_pretrained('jiabins0303/birefnet-lite-1024-webgpu', {
+  device: 'webgpu',
+  dtype: 'fp32',                  // the file is ALREADY fp16; this stops the
+  model_file_name: 'model_fp16',  // library appending a dtype suffix
+});
+```
+
+Requires an adapter reporting `maxStorageBuffersPerShaderStage >= 8`. The graph
+emits **logits**, not an image — apply sigmoid, do not read it as 0-255.
+
+## Why the upstream export cannot run on WebGPU
+
+Two independent blockers, which have to be fixed in this order because the first
+one masks the second:
+
+1. **59 `Split` nodes with 32 outputs each.** WebGPU's
+   `maxStorageBuffersPerShaderStage` is 8 by spec (10 on many adapters), so these
+   never compile. Rewritten into trees of <=6 outputs.
+2. **80 `GatherND` nodes.** `deform_conv2d` has no ONNX operator, so the exporter
+   emulates it with `GatherND`, and in the onnxruntime-web build we tested
+   (August 2026) the WebGPU EP did not take those nodes: all 80 ran on the CPU
+   EP (the verbose log adds a `MemcpyFromHost` after each), which meant materialising a
+   `[1,1,64,49,256,256]` fp16 im2col buffer — **392MB**, several live at once — on
+   ORT Web's **32-bit wasm heap**. That is the `std::bad_alloc`; the resolution
+   itself was never the problem. Rewritten to `Gather` plus int32 index
+   arithmetic, and variadic `Sum` to binary `Add`.
+
+Host<->device copies drop from 100 to 80 and the graph runs end to end on WebGPU.
+
+## Reproduce it: the patch scripts
+
+The three scripts that produced this graph are in [`scripts/`](./scripts) (MIT, like
+the rest of this repo). They are topology-only rewrites and should work on other
+BiRefNet exports that hit the same two blockers.
+
+```bash
+pip install onnx onnxruntime numpy pillow
+python scripts/patch_split.py  model.onnx step1.onnx       # Split trees, <= 6 outputs
+python scripts/patch_deform.py step1.onnx patched.onnx     # GatherND -> Gather, Sum -> Add chain
+python scripts/verify_patch.py model.onnx patched.onnx a.jpg b.jpg   # must report EXACT
+```
+
+`patch_deform.py` only rewrites the exact deform-conv `GatherND` pattern and refuses
+to rewrite one unless it can see a clamp upstream, so an index cannot silently wrap.
+`verify_patch.py` runs both graphs on the CPU EP; a structural rewrite should be
+bit-identical, and it says so if it is not.
+
+Two things that are easy to get wrong if you redo this:
+
+- **Cast to int32 LATE.** Doing the index arithmetic in int32 doubles the
+  CPU<->GPU crossings (180 copies / 9.1s vs 80 / 3.4s). Do the maths in int64 and
+  cast once, immediately before the `Gather`.
+- **Use a sequential `Add` chain, not a balanced tree.** fp16 addition is not
+  associative; a balanced tree changes the summation order and breaks
+  bit-identity with the original graph.
+
+## Measurements
+
+Against BiRefNet-lite-512 on the same 30-image set, same compositor, same metric
+(`edgeSharpness().gradient`):
+
+| | all-30 gradient | products | interiorSoft | browser time (WebGPU) |
+|---|---|---|---|---|
+| BiRefNet-lite-512 | 0.0989 | 0.1095 | 0.00296 | 0.85s |
+| **this (1024)** | **0.1436** | **0.1754** | **0.00043** | **3.4s** |
+
++45% edge gradient for ~4x the time. It also recovers subjects 512 misses
+entirely — one test image returns an empty matte at 512 and 18.4% foreground here.
+
+⚠️ **Read the provenance before quoting these.** The **quality** columns and the
+**timing** column come from different runs. The 1024 gradient figures were measured
+in Node on the **CPU EP**, not on the WebGPU path the timings describe; graph
+bit-identity was proven CPU-vs-CPU for the *rewrite*, which is not the same as
+proving CPU and WebGPU produce identical output. And one flag is still open: this
+graph's product `softEdgeRatio` is 0.01030 against a 0.01574 control — 35% lower —
+which under an anti-aliasing guard means part of the gradient win **could be a
+harder, less anti-aliased edge rather than a better one**. Band width (9.15) and the
+crops both support a genuinely tighter edge, but it is not fully cleared. Treat +45%
+as a strong indication, not a settled result, and re-measure on your own path.
+
+## Which one should you use
+
+Not always this one.
+
+- **512** is the better default for interactive, one-image-at-a-time UIs, for
+  batch jobs where latency multiplies by the file count, and for wide device
+  support. 0.85s is a different product feel from 3.4s.
+- **1024 (this)** is worth it when edge quality is the point — hair, fur, fabric,
+  fine product edges — or when 512 misses the subject entirely.
+
+Shipping both and letting the user choose is a reasonable design; that is what
+Backgroundless does. Only one graph should be resident at a time — switching
+disposes and recompiles.
+
+## Licensing
+
+MIT — and unusually for a model, that covers the **weights**, not just the surrounding
+code. No non-commercial clause, no separate model agreement to accept, and nothing in
+it places any condition on the images you produce with it.
+
+Worth checking on whatever you end up picking: licensing on browser matting models
+varies a lot, and the gap between a project's *code* licence and its *weights* licence
+is where people get caught. BRIA's RMBG-1.4 card, for example, states the model is
+"available as a source-available model for non-commercial use" — and an MIT-licensed
+wrapper repo around it does not change that, because the wrapper's licence covers the
+wrapper. Read the weights licence, not the repo badge.
+
+⚠️ **One upstream caveat, in fairness.** BiRefNet_lite lists **DIS5K** among its
+training sets, and DIS5K's own terms restrict commercial use of the dataset. That is an
+open question for every model trained on it, this one included, and it is not something
+an MIT grant downstream can resolve. What MIT does give you here is a clean, explicit
+grant from the model's author covering the weights themselves — which is more than
+several alternatives offer, but it is not a warranty about the training data.
+
+## Credits and licence
+
+- Original model: **[ZhengPeng7/BiRefNet](https://github.com/ZhengPeng7/BiRefNet)** (MIT) — Peng Zheng et al.
+- ONNX export this was derived from: **onnx-community/BiRefNet_lite-ONNX**
+- **[studioludens/birefnet-lite-512](https://huggingface.co/studioludens/birefnet-lite-512)** —
+  Studio Ludens' 512 re-export, and the export method (PyTorch 2.0.1 + Kazuhito00's patched
+  deform-conv exporter) that made a working browser BiRefNet possible at all. Their model card is
+  also where the "1024-way concatenations" problem is documented, which is what sent us looking at
+  the decoder in the first place. **Backgroundless ships their 512 export as its default tier** —
+  this 1024 graph is the opt-in one.
+- Kazuhito00's **deform-conv2d-onnx-exporter**, via the above.
+- Graph surgery: this repo. MIT, same as upstream.
+
+If you use this, cite the BiRefNet authors — the model is theirs. The only thing
+contributed here is the graph rewrite.
+
+## Provenance
+
+`onnx/model_fp16.onnx`
+sha256 `4059896039dfccb0f15b9080ff06d11d90e499449bb045e797055eb8901cf5f4`
+
+Reproduce with `patch_split.py` then `patch_deform.py`, and verify with
+`verify_patch.py` (correlation gate plus max-abs-diff against the unpatched
+graph on the CPU EP).

scripts/patch_deform.py

@@ -0,0 +1,258 @@
+"""
+Move BiRefNet's deformable-convolution nodes off the CPU execution provider.
+
+WHY. `patch_split.py` fixed the FIRST ceiling (WebGPU's storage-buffer limit).
+The graph then still died, and ORT's own verbose log named the second one
+exactly:
+
+    transformer_memcpy.cc:340 AddCopyNode] Add MemcpyFromHost after
+      /decoder/decoder_block1/dec_att/aspp_deforms.2/atrous_conv/GatherND_output_0
+      for WebGpuExecutionProvider
+
+100 such copies at 1024, and NOT ONE of them outside `atrous_conv`: 80 GatherND
+plus 20 Sum, across the 20 deformable-conv blocks (decoder_block1-4 and
+squeeze_module, four ASPP branches each). `deform_conv2d` has no ONNX op, so the
+exporter decomposes it into bilinear sampling built from GatherND - and in the
+onnxruntime-web build we tested (August 2026) the WebGPU EP did not take those
+nodes, so they ran on the CPU EP. CPU tensors
+live on the 32-bit wasm heap, and at 1024 each GatherND output is
+
+    [1, 1, 64, 49, 256, 256] fp16 = 392MB
+
+with five live at once. That is the `std::bad_alloc`. At 512 the same tensors
+are 98MB and it fits, which is the entire reason 512 ships and 1024 does not.
+
+WHAT THIS DOES. Rewrites each GatherND into ops the WebGPU EP demonstrably has -
+plain `Gather` is used 72 times elsewhere in this graph and never appears in the
+memcpy list, so it is known-supported here rather than assumed:
+
+    GatherND(data[1,1,H,W,C], idx[1,1,P,2], batch_dims=2) -> [1,1,P,C]
+      becomes
+    flat  = Cast(Reshape(idx,[P,2]), int32)          # WebGPU has no int64 maths
+    y,x   = Slice(flat,0:1,axis1), Slice(flat,1:2,axis1)
+    lin   = Reshape(Add(Mul(y, W), x), [P])
+    out   = Reshape(Gather(Reshape(data,[H*W,C]), lin, axis=0), [1,1,P,C])
+
+and each variadic `Sum` into a left-to-right chain of binary `Add`, for the
+same reason (see --sum-mode for why a chain and not a tree).
+
+`Mul(y, W) + x` is only equal to the original gather if the indices are already
+in range, so this refuses to rewrite unless it can SEE a clamp upstream. A
+silent wrap on a negative index would corrupt one pixel in a way no aggregate
+metric would ever show.
+
+Verify with `verify_patch.py` - this is a structural rewrite and must be
+bit-identical, not merely close.
+
+    python patch_deform.py in.onnx out.onnx
+"""
+
+import argparse
+import collections
+import sys
+
+import onnx
+from onnx import TensorProto, helper, numpy_helper
+
+# Ops that establish an index is already in range. If none of these appears
+# between the index arithmetic and the gather, the rewrite is refused.
+CLAMP_OPS = {"Clip", "Min", "Max"}
+
+
+def shapes_of(model):
+    inferred = onnx.shape_inference.infer_shapes(model, strict_mode=False, data_prop=True)
+    out = {}
+    for vi in list(inferred.graph.value_info) + list(inferred.graph.output) + list(inferred.graph.input):
+        dims = []
+        ok = True
+        for d in vi.type.tensor_type.shape.dim:
+            if d.HasField("dim_value"):
+                dims.append(d.dim_value)
+            else:
+                ok = False
+                break
+        if ok:
+            out[vi.name] = dims
+    return out
+
+
+def clamped_upstream(name, producer, depth=12):
+    """Walk back from `name` looking for a clamp, stopping at the first fan-in."""
+    seen = set()
+    stack = [(name, 0)]
+    while stack:
+        n, d = stack.pop()
+        if d > depth or n in seen:
+            continue
+        seen.add(n)
+        node = producer.get(n)
+        if node is None:
+            continue
+        if node.op_type in CLAMP_OPS:
+            return True
+        for inp in node.input:
+            if inp:
+                stack.append((inp, d + 1))
+    return False
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("src")
+    ap.add_argument("dst")
+    ap.add_argument("--allow-unclamped", action="store_true",
+                    help="rewrite even when no clamp is visible upstream (unsafe; prints which)")
+    ap.add_argument("--sum-mode", choices=["chain", "tree", "none"], default="chain",
+                    help="how to decompose variadic Sum. 'chain' is left-to-right and is the "
+                         "DEFAULT because floating-point addition is not associative: a balanced "
+                         "'tree' computes (a+b)+(c+d) where ORT's Sum accumulates ((a+b)+c)+d, "
+                         "and in fp16 that alone moved the output by 4.9e-03 - measured, and the "
+                         "sole reason a structural rewrite failed its bit-identity gate. "
+                         "'none' leaves Sum alone, to isolate the GatherND rewrite.")
+    args = ap.parse_args()
+
+    model = onnx.load(args.src)
+    graph = model.graph
+    shapes = shapes_of(model)
+    producer = {o: n for n in graph.node for o in n.output}
+
+    new_nodes = []
+    consts = []
+    stats = collections.Counter()
+    skipped = []
+    uid = 0
+
+    def const_i64(vals, tag):
+        nonlocal uid
+        uid += 1
+        name = f"deformpatch_{tag}_{uid}"
+        consts.append(numpy_helper.from_array(
+            __import__("numpy").array(vals, dtype="int64"), name))
+        return name
+
+    def const_i32(vals, tag):
+        nonlocal uid
+        uid += 1
+        name = f"deformpatch_{tag}_{uid}"
+        consts.append(numpy_helper.from_array(
+            __import__("numpy").array(vals, dtype="int32"), name))
+        return name
+
+    for node in graph.node:
+        if node.op_type == "Sum" and len(node.input) > 2 and args.sum_mode != "none":
+            # Variadic Sum has no WebGPU kernel; Add does.
+            if args.sum_mode == "chain":
+                # Left-to-right, matching ORT's own accumulation order so the
+                # rewrite stays bit-identical in fp16. See --sum-mode.
+                acc = node.input[0]
+                for i in range(1, len(node.input)):
+                    last = i == len(node.input) - 1
+                    out = node.output[0] if last else f"{node.name}_acc{i}"
+                    new_nodes.append(helper.make_node(
+                        "Add", [acc, node.input[i]], [out], name=f"{node.name}_acc{i}"))
+                    acc = out
+                stats["Sum->Add chain"] += 1
+            else:
+                level = list(node.input)
+                tier = 0
+                while len(level) > 1:
+                    nxt = []
+                    for i in range(0, len(level), 2):
+                        if i + 1 == len(level):
+                            nxt.append(level[i])
+                            continue
+                        last = len(level) <= 2
+                        out = node.output[0] if last else f"{node.name}_add{tier}_{i}"
+                        new_nodes.append(helper.make_node(
+                            "Add", [level[i], level[i + 1]], [out],
+                            name=f"{node.name}_add{tier}_{i}"))
+                        nxt.append(out)
+                    level = nxt
+                    tier += 1
+                stats["Sum->Add tree"] += 1
+            continue
+
+        if node.op_type != "GatherND":
+            new_nodes.append(node)
+            continue
+
+        attrs = {a.name: helper.get_attribute_value(a) for a in node.attribute}
+        batch_dims = attrs.get("batch_dims", 0)
+        data_s = shapes.get(node.input[0])
+        idx_s = shapes.get(node.input[1])
+
+        # Only the exact deform-conv pattern is rewritten. Anything else keeps
+        # its GatherND rather than being guessed at.
+        if not (batch_dims == 2 and data_s and idx_s
+                and len(data_s) == 5 and len(idx_s) == 4
+                and data_s[0] == 1 and data_s[1] == 1
+                and idx_s[0] == 1 and idx_s[1] == 1 and idx_s[3] == 2):
+            skipped.append((node.name, f"batch_dims={batch_dims} data={data_s} idx={idx_s}"))
+            new_nodes.append(node)
+            stats["GatherND kept (pattern)"] += 1
+            continue
+
+        if not clamped_upstream(node.input[1], producer) and not args.allow_unclamped:
+            skipped.append((node.name, "no clamp visible upstream"))
+            new_nodes.append(node)
+            stats["GatherND kept (unclamped)"] += 1
+            continue
+
+        _, _, H, W, C = data_s
+        P = idx_s[2]
+        b = node.name.replace("/", "_")
+
+        n_idx2 = f"{b}__idx2"
+        n_i32 = f"{b}__i32"
+        n_y = f"{b}__y"
+        n_x = f"{b}__x"
+        n_yw = f"{b}__yw"
+        n_lin2 = f"{b}__lin2"
+        n_lin = f"{b}__lin"
+        n_data2 = f"{b}__data2"
+        n_gat = f"{b}__gat"
+
+        # ORDER MATTERS, and it is measured. The index chain arrives as int64 and
+        # WebGPU has no int64, so everything up to the Cast necessarily runs on
+        # the CPU EP. Casting FIRST (the obvious reading) then doing the
+        # arithmetic on the GPU means both [P,1] halves cross the boundary:
+        # 160 copies of 12.8MB each at 1024. Doing the arithmetic in int64 while
+        # it is already on the CPU and casting ONCE means a single [P] tensor
+        # crosses per node - half the transfers and half the bytes.
+        new_nodes += [
+            helper.make_node("Reshape", [node.input[1], const_i64([P, 2], "s")], [n_idx2], name=n_idx2),
+            helper.make_node("Slice", [n_idx2, const_i64([0], "st"), const_i64([1], "en"),
+                                       const_i64([1], "ax")], [n_y], name=n_y),
+            helper.make_node("Slice", [n_idx2, const_i64([1], "st"), const_i64([2], "en"),
+                                       const_i64([1], "ax")], [n_x], name=n_x),
+            helper.make_node("Mul", [n_y, const_i64([W], "w")], [n_yw], name=n_yw),
+            helper.make_node("Add", [n_yw, n_x], [n_lin2], name=n_lin2),
+            helper.make_node("Reshape", [n_lin2, const_i64([P], "s")], [n_i32], name=n_i32),
+            helper.make_node("Cast", [n_i32], [n_lin], to=TensorProto.INT32, name=n_lin),
+            helper.make_node("Reshape", [node.input[0], const_i64([H * W, C], "s")], [n_data2], name=n_data2),
+            helper.make_node("Gather", [n_data2, n_lin], [n_gat], axis=0, name=n_gat),
+            helper.make_node("Reshape", [n_gat, const_i64([1, 1, P, C], "s")], [node.output[0]], name=f"{b}__out"),
+        ]
+        stats["GatherND->Gather"] += 1
+
+    del graph.node[:]
+    graph.node.extend(new_nodes)
+    graph.initializer.extend(consts)
+
+    onnx.checker.check_model(model, full_check=False)
+    onnx.save(model, args.dst)
+
+    print(f"{args.src} -> {args.dst}")
+    for k, v in stats.most_common():
+        print(f"  {k}: {v}")
+    if skipped:
+        print(f"\n  {len(skipped)} node(s) left as-is:")
+        for name, why in skipped[:10]:
+            print(f"    {name}: {why}")
+    print(f"\n  nodes: {len(new_nodes)} (+{len(consts)} new initializers)")
+    print("\nNow run verify_patch.py - a structural rewrite must be bit-identical.")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

scripts/patch_split.py

@@ -0,0 +1,159 @@
+"""
+Rewrite wide `Split` nodes into trees of narrow ones, so BiRefNet runs on WebGPU.
+
+THE PROBLEM. ORT's WebGPU backend binds one storage buffer per shader variable
+and throws above `maxStorageBuffersPerShaderStage` (8 by WebGPU spec default;
+some adapters report 10). It chunks `Concat` inputs to stay under that limit but does NOT
+chunk `Split` outputs. BiRefNet_lite@1024's decoder contains 59 Splits with 32
+outputs each - 33 buffers apiece - so the graph cannot execute at all.
+Full diagnosis in the model card (README.md).
+
+WHY THIS IS SAFE. Splitting [7,7,6,6,6] and then splitting each group is exactly
+equivalent to splitting all 32 at once: same axis, same sizes, same order, same
+output names. It is a structural rewrite with no numerical content, which is why
+the acceptance gate is bit-level correlation against the original rather than a
+quality metric.
+
+WHY NOT PATCH `Concat` TOO. The same graph has a 1024-input Concat, but ORT
+already chunks Concat, and the 512 export proves that path works. Leave it.
+
+    python patch_split.py in.onnx out.onnx [--max-outputs 6]
+"""
+
+import argparse
+import sys
+
+import numpy as np
+import onnx
+from onnx import helper, numpy_helper
+
+# 6, not 7, and the extra margin is deliberate.
+#
+# A Split shader almost certainly binds one buffer for the data input plus one
+# per output - the `split` sizes tensor has to be read on the host to compute
+# output shapes, so it should not consume a binding. That would make 7 outputs
+# (8 buffers) fit the WebGPU spec default of 8.
+#
+# But "almost certainly" is the wrong confidence level for the thing that decides
+# whether the model runs on a stranger's GPU, and being wrong costs one extra
+# node per group. 6 outputs is 8 buffers even under the pessimistic accounting
+# where the sizes input IS bound.
+DEFAULT_MAX_OUTPUTS = 6
+
+
+def group_sizes(sizes, max_outputs):
+    """Partition `sizes` into as few near-equal runs as possible, each <= max_outputs."""
+    n = len(sizes)
+    chunks = -(-n // max_outputs)  # ceil
+    base, rem = divmod(n, chunks)
+    groups, i = [], 0
+    for g in range(chunks):
+        take = base + (1 if g < rem else 0)
+        groups.append(sizes[i : i + take])
+        i += take
+    return groups
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("src")
+    ap.add_argument("dst")
+    ap.add_argument("--max-outputs", type=int, default=DEFAULT_MAX_OUTPUTS)
+    args = ap.parse_args()
+
+    model = onnx.load(args.src)
+    graph = model.graph
+
+    # Split sizes can arrive as an initializer or as a Constant node's output.
+    consts = {init.name: numpy_helper.to_array(init) for init in graph.initializer}
+    for node in graph.node:
+        if node.op_type == "Constant" and node.output:
+            for attr in node.attribute:
+                if attr.name == "value":
+                    consts[node.output[0]] = numpy_helper.to_array(attr.t)
+
+    new_nodes, added_inits, patched, skipped = [], [], 0, []
+
+    for node in graph.node:
+        if node.op_type != "Split" or len(node.output) <= args.max_outputs:
+            new_nodes.append(node)
+            continue
+
+        axis = next((a.i for a in node.attribute if a.name == "axis"), 0)
+
+        sizes = None
+        if len(node.input) > 1 and node.input[1] in consts:
+            sizes = [int(v) for v in consts[node.input[1]]]
+        else:
+            attr = next((a for a in node.attribute if a.name == "split"), None)
+            if attr is not None:
+                sizes = [int(v) for v in attr.ints]
+
+        # Without explicit sizes the split is equal-division by output count, and
+        # rewriting it would require the input shape - which is not reliably
+        # known here. Leave it and report it as unfixed rather than emit a
+        # graph that is subtly wrong.
+        if sizes is None or len(sizes) != len(node.output):
+            skipped.append(node.name or "(unnamed)")
+            new_nodes.append(node)
+            continue
+
+        groups = group_sizes(sizes, args.max_outputs)
+        stem = (node.name or f"split_{patched}").replace("/", "_")
+
+        # Level 1: one output per group, each carrying that group's total width.
+        l1_sizes = np.array([sum(g) for g in groups], dtype=np.int64)
+        l1_name = f"{stem}__l1_sizes"
+        added_inits.append(numpy_helper.from_array(l1_sizes, l1_name))
+        l1_outputs = [f"{stem}__g{i}" for i in range(len(groups))]
+        new_nodes.append(
+            helper.make_node(
+                "Split",
+                inputs=[node.input[0], l1_name],
+                outputs=l1_outputs,
+                name=f"{stem}__l1",
+                axis=axis,
+            )
+        )
+
+        # Level 2: each group back to the ORIGINAL output names, in order, so
+        # every downstream consumer is untouched.
+        cursor = 0
+        for gi, group in enumerate(groups):
+            outs = list(node.output[cursor : cursor + len(group)])
+            cursor += len(group)
+            if len(group) == 1:
+                # A one-way Split is illegal; alias instead.
+                new_nodes.append(
+                    helper.make_node("Identity", [l1_outputs[gi]], outs, name=f"{stem}__g{gi}_id")
+                )
+                continue
+            g_name = f"{stem}__g{gi}_sizes"
+            added_inits.append(numpy_helper.from_array(np.array(group, dtype=np.int64), g_name))
+            new_nodes.append(
+                helper.make_node(
+                    "Split",
+                    inputs=[l1_outputs[gi], g_name],
+                    outputs=outs,
+                    name=f"{stem}__g{gi}",
+                    axis=axis,
+                )
+            )
+        patched += 1
+
+    del graph.node[:]
+    graph.node.extend(new_nodes)
+    graph.initializer.extend(added_inits)
+
+    onnx.checker.check_model(model, full_check=False)
+    onnx.save(model, args.dst, save_as_external_data=False)
+
+    print(f"patched {patched} Split nodes -> max {args.max_outputs} outputs each")
+    if skipped:
+        print(f"SKIPPED {len(skipped)} (no explicit sizes): {skipped[:5]}")
+    print(f"nodes {len(model.graph.node)}, wrote {args.dst}")
+    return 1 if skipped else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

scripts/verify_patch.py

@@ -0,0 +1,84 @@
+"""
+Prove the patched graph is numerically identical to the original.
+
+This is the ONLY thing standing between a subtly wrong graph and production: the
+patched ONNX is our own artifact, no upstream validates it, and a Split tree that
+reorders or mis-sizes a group would still produce a plausible-looking matte.
+A quality metric cannot catch that. Bit-level agreement can.
+
+`patch_split.py` and `patch_deform.py` perform structural rewrites with no numerical content, so the
+correct expectation is EXACT equality, not "close enough". The 0.9999 gate exists
+only to absorb non-determinism in CPU kernel scheduling.
+
+    python verify_patch.py orig.onnx patched.onnx img1.jpg img2.jpg ...
+"""
+
+import argparse
+import sys
+
+import numpy as np
+import onnxruntime as ort
+from PIL import Image
+
+SIZE = 1024
+MEAN = np.array([0.485, 0.456, 0.406], dtype=np.float32)
+STD = np.array([0.229, 0.224, 0.225], dtype=np.float32)
+
+
+def preprocess(path):
+    # fit='fill' - the aspect ratio is squashed, matching what the browser does
+    # by drawing into a square canvas. A different framing is a different input.
+    img = Image.open(path).convert("RGB").resize((SIZE, SIZE), Image.BILINEAR)
+    x = np.asarray(img, dtype=np.float32) / 255.0
+    x = (x - MEAN) / STD
+    return x.transpose(2, 0, 1)[None].astype(np.float32)
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("orig")
+    ap.add_argument("patched")
+    ap.add_argument("images", nargs="+")
+    ap.add_argument("--min-correlation", type=float, default=0.9999)
+    args = ap.parse_args()
+
+    opts = ort.SessionOptions()
+    opts.graph_optimization_level = ort.GraphOptimizationLevel.ORT_DISABLE_ALL
+    a = ort.InferenceSession(args.orig, opts, providers=["CPUExecutionProvider"])
+    b = ort.InferenceSession(args.patched, opts, providers=["CPUExecutionProvider"])
+
+    name_a = a.get_inputs()[0].name
+    name_b = b.get_inputs()[0].name
+    print(f"inputs: {name_a} / {name_b}")
+
+    worst_corr, worst_absdiff, failed = 1.0, 0.0, False
+    for path in args.images:
+        x = preprocess(path)
+        oa = np.asarray(a.run(None, {name_a: x})[0], dtype=np.float64).ravel()
+        ob = np.asarray(b.run(None, {name_b: x})[0], dtype=np.float64).ravel()
+
+        if oa.shape != ob.shape:
+            print(f"FAIL {path}: shape {oa.shape} vs {ob.shape}")
+            failed = True
+            continue
+
+        absdiff = float(np.max(np.abs(oa - ob)))
+        corr = 1.0 if absdiff == 0.0 else float(np.corrcoef(oa, ob)[0, 1])
+        worst_corr = min(worst_corr, corr)
+        worst_absdiff = max(worst_absdiff, absdiff)
+        exact = "EXACT" if absdiff == 0.0 else f"max|diff| {absdiff:.3e}"
+        print(f"  {path.split('/')[-1]:50s} corr {corr:.8f}  {exact}")
+
+    print(f"\nworst correlation {worst_corr:.8f}, worst max|diff| {worst_absdiff:.3e}")
+    if failed or worst_corr < args.min_correlation:
+        print("REJECTED — do not ship this graph.")
+        return 1
+    if worst_absdiff == 0.0:
+        print("ACCEPTED — bit-identical, as a structural rewrite should be.")
+    else:
+        print("ACCEPTED — within tolerance, but NOT bit-identical; investigate before shipping.")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())