--- license: mit library_name: transformers.js pipeline_tag: image-segmentation base_model: ZhengPeng7/BiRefNet_lite tags: - onnx - background-removal - matting - alpha-matting - image-matting - webgpu - client-side - in-browser - transformers.js - fp16 --- # BiRefNet-lite 1024 (ONNX, WebGPU-runnable) A structurally patched ONNX export of **BiRefNet-lite at 1024x1024** that runs on **onnxruntime-web's WebGPU execution provider**. The upstream export does not: it falls back to the CPU EP on most of its graph and dies of `std::bad_alloc`. **If you have tried to run BiRefNet at 1024 in a browser and hit an out-of-memory error, this repo is the fix, and the section below explains what was actually wrong.** The usual workaround is to drop to a 512 export. That works and it is a reasonable trade — but the OOM is not caused by the resolution, and you do not have to pay it. The weights are unchanged. Only the graph topology was rewritten, and both rewrites are proved **bit-identical** to the original on the CPU EP (`max|diff| = 0.000e+00` over the test set). ## Production use This graph is the `best` quality tier in **[Backgroundless](https://backgroundless.io)**, a background remover where the model runs entirely in the visitor's browser — no upload, no server round-trip, no account. Both tiers are free to use, so the 1024 graph is served to anyone whose adapter can run it. ## Quick start (transformers.js) ```js import { AutoModel } from '@huggingface/transformers'; const model = await AutoModel.from_pretrained('jiabins0303/birefnet-lite-1024-webgpu', { device: 'webgpu', dtype: 'fp32', // the file is ALREADY fp16; this stops the model_file_name: 'model_fp16', // library appending a dtype suffix }); ``` Requires an adapter reporting `maxStorageBuffersPerShaderStage >= 8`. The graph emits **logits**, not an image — apply sigmoid, do not read it as 0-255. ## Why the upstream export cannot run on WebGPU Two independent blockers, which have to be fixed in this order because the first one masks the second: 1. **59 `Split` nodes with 32 outputs each.** WebGPU's `maxStorageBuffersPerShaderStage` is 8 by spec (10 on many adapters), so these never compile. Rewritten into trees of <=6 outputs. 2. **80 `GatherND` nodes.** `deform_conv2d` has no ONNX operator, so the exporter emulates it with `GatherND`, and in the onnxruntime-web build we tested (August 2026) the WebGPU EP did not take those nodes: all 80 ran on the CPU EP (the verbose log adds a `MemcpyFromHost` after each), which meant materialising a `[1,1,64,49,256,256]` fp16 im2col buffer — **392MB**, several live at once — on ORT Web's **32-bit wasm heap**. That is the `std::bad_alloc`; the resolution itself was never the problem. Rewritten to `Gather` plus int32 index arithmetic, and variadic `Sum` to binary `Add`. Host<->device copies drop from 100 to 80 and the graph runs end to end on WebGPU. ## Reproduce it: the patch scripts The three scripts that produced this graph are in [`scripts/`](./scripts) (MIT, like the rest of this repo). They are topology-only rewrites and should work on other BiRefNet exports that hit the same two blockers. ```bash pip install onnx onnxruntime numpy pillow python scripts/patch_split.py model.onnx step1.onnx # Split trees, <= 6 outputs python scripts/patch_deform.py step1.onnx patched.onnx # GatherND -> Gather, Sum -> Add chain python scripts/verify_patch.py model.onnx patched.onnx a.jpg b.jpg # must report EXACT ``` `patch_deform.py` only rewrites the exact deform-conv `GatherND` pattern and refuses to rewrite one unless it can see a clamp upstream, so an index cannot silently wrap. `verify_patch.py` runs both graphs on the CPU EP; a structural rewrite should be bit-identical, and it says so if it is not. Two things that are easy to get wrong if you redo this: - **Cast to int32 LATE.** Doing the index arithmetic in int32 doubles the CPU<->GPU crossings (180 copies / 9.1s vs 80 / 3.4s). Do the maths in int64 and cast once, immediately before the `Gather`. - **Use a sequential `Add` chain, not a balanced tree.** fp16 addition is not associative; a balanced tree changes the summation order and breaks bit-identity with the original graph. ## Measurements Against BiRefNet-lite-512 on the same 30-image set, same compositor, same metric (`edgeSharpness().gradient`): | | all-30 gradient | products | interiorSoft | browser time (WebGPU) | |---|---|---|---|---| | BiRefNet-lite-512 | 0.0989 | 0.1095 | 0.00296 | 0.85s | | **this (1024)** | **0.1436** | **0.1754** | **0.00043** | **3.4s** | +45% edge gradient for ~4x the time. It also recovers subjects 512 misses entirely — one test image returns an empty matte at 512 and 18.4% foreground here. ⚠️ **Read the provenance before quoting these.** The **quality** columns and the **timing** column come from different runs. The 1024 gradient figures were measured in Node on the **CPU EP**, not on the WebGPU path the timings describe; graph bit-identity was proven CPU-vs-CPU for the *rewrite*, which is not the same as proving CPU and WebGPU produce identical output. And one flag is still open: this graph's product `softEdgeRatio` is 0.01030 against a 0.01574 control — 35% lower — which under an anti-aliasing guard means part of the gradient win **could be a harder, less anti-aliased edge rather than a better one**. Band width (9.15) and the crops both support a genuinely tighter edge, but it is not fully cleared. Treat +45% as a strong indication, not a settled result, and re-measure on your own path. ## Which one should you use Not always this one. - **512** is the better default for interactive, one-image-at-a-time UIs, for batch jobs where latency multiplies by the file count, and for wide device support. 0.85s is a different product feel from 3.4s. - **1024 (this)** is worth it when edge quality is the point — hair, fur, fabric, fine product edges — or when 512 misses the subject entirely. Shipping both and letting the user choose is a reasonable design; that is what Backgroundless does. Only one graph should be resident at a time — switching disposes and recompiles. ## Licensing MIT — and unusually for a model, that covers the **weights**, not just the surrounding code. No non-commercial clause, no separate model agreement to accept, and nothing in it places any condition on the images you produce with it. Worth checking on whatever you end up picking: licensing on browser matting models varies a lot, and the gap between a project's *code* licence and its *weights* licence is where people get caught. BRIA's RMBG-1.4 card, for example, states the model is "available as a source-available model for non-commercial use" — and an MIT-licensed wrapper repo around it does not change that, because the wrapper's licence covers the wrapper. Read the weights licence, not the repo badge. ⚠️ **One upstream caveat, in fairness.** BiRefNet_lite lists **DIS5K** among its training sets, and DIS5K's own terms restrict commercial use of the dataset. That is an open question for every model trained on it, this one included, and it is not something an MIT grant downstream can resolve. What MIT does give you here is a clean, explicit grant from the model's author covering the weights themselves — which is more than several alternatives offer, but it is not a warranty about the training data. ## Credits and licence - Original model: **[ZhengPeng7/BiRefNet](https://github.com/ZhengPeng7/BiRefNet)** (MIT) — Peng Zheng et al. - ONNX export this was derived from: **onnx-community/BiRefNet_lite-ONNX** - **[studioludens/birefnet-lite-512](https://huggingface.co/studioludens/birefnet-lite-512)** — Studio Ludens' 512 re-export, and the export method (PyTorch 2.0.1 + Kazuhito00's patched deform-conv exporter) that made a working browser BiRefNet possible at all. Their model card is also where the "1024-way concatenations" problem is documented, which is what sent us looking at the decoder in the first place. **Backgroundless ships their 512 export as its default tier** — this 1024 graph is the opt-in one. - Kazuhito00's **deform-conv2d-onnx-exporter**, via the above. - Graph surgery: this repo. MIT, same as upstream. If you use this, cite the BiRefNet authors — the model is theirs. The only thing contributed here is the graph rewrite. ## Provenance `onnx/model_fp16.onnx` sha256 `4059896039dfccb0f15b9080ff06d11d90e499449bb045e797055eb8901cf5f4` Reproduce with `patch_split.py` then `patch_deform.py`, and verify with `verify_patch.py` (correlation gate plus max-abs-diff against the unpatched graph on the CPU EP).