| --- |
| license: mit |
| library_name: transformers.js |
| pipeline_tag: image-segmentation |
| base_model: ZhengPeng7/BiRefNet_lite |
| tags: |
| - onnx |
| - background-removal |
| - matting |
| - alpha-matting |
| - image-matting |
| - webgpu |
| - client-side |
| - in-browser |
| - transformers.js |
| - fp16 |
| --- |
| |
| # BiRefNet-lite 1024 (ONNX, WebGPU-runnable) |
|
|
| A structurally patched ONNX export of **BiRefNet-lite at 1024x1024** that runs on |
| **onnxruntime-web's WebGPU execution provider**. The upstream export does not: it |
| falls back to the CPU EP on most of its graph and dies of `std::bad_alloc`. |
|
|
| **If you have tried to run BiRefNet at 1024 in a browser and hit an |
| out-of-memory error, this repo is the fix, and the section below explains what |
| was actually wrong.** The usual workaround is to drop to a 512 export. That |
| works and it is a reasonable trade — but the OOM is not caused by the |
| resolution, and you do not have to pay it. |
|
|
| The weights are unchanged. Only the graph topology was rewritten, and both |
| rewrites are proved **bit-identical** to the original on the CPU EP |
| (`max|diff| = 0.000e+00` over the test set). |
|
|
| ## Production use |
|
|
| This graph is the `best` quality tier in **[Backgroundless](https://backgroundless.io)**, |
| a background remover where the model runs entirely in the visitor's browser — |
| no upload, no server round-trip, no account. Both tiers are free to use, so the |
| 1024 graph is served to anyone whose adapter can run it. |
|
|
| ## Quick start (transformers.js) |
|
|
| ```js |
| import { AutoModel } from '@huggingface/transformers'; |
| |
| const model = await AutoModel.from_pretrained('jiabins0303/birefnet-lite-1024-webgpu', { |
| device: 'webgpu', |
| dtype: 'fp32', // the file is ALREADY fp16; this stops the |
| model_file_name: 'model_fp16', // library appending a dtype suffix |
| }); |
| ``` |
|
|
| Requires an adapter reporting `maxStorageBuffersPerShaderStage >= 8`. The graph |
| emits **logits**, not an image — apply sigmoid, do not read it as 0-255. |
|
|
| ## Why the upstream export cannot run on WebGPU |
|
|
| Two independent blockers, which have to be fixed in this order because the first |
| one masks the second: |
|
|
| 1. **59 `Split` nodes with 32 outputs each.** WebGPU's |
| `maxStorageBuffersPerShaderStage` is 8 by spec (10 on many adapters), so these |
| never compile. Rewritten into trees of <=6 outputs. |
| 2. **80 `GatherND` nodes.** `deform_conv2d` has no ONNX operator, so the exporter |
| emulates it with `GatherND`, and in the onnxruntime-web build we tested |
| (August 2026) the WebGPU EP did not take those nodes: all 80 ran on the CPU |
| EP (the verbose log adds a `MemcpyFromHost` after each), which meant materialising a |
| `[1,1,64,49,256,256]` fp16 im2col buffer — **392MB**, several live at once — on |
| ORT Web's **32-bit wasm heap**. That is the `std::bad_alloc`; the resolution |
| itself was never the problem. Rewritten to `Gather` plus int32 index |
| arithmetic, and variadic `Sum` to binary `Add`. |
|
|
| Host<->device copies drop from 100 to 80 and the graph runs end to end on WebGPU. |
|
|
| ## Reproduce it: the patch scripts |
|
|
| The three scripts that produced this graph are in [`scripts/`](./scripts) (MIT, like |
| the rest of this repo). They are topology-only rewrites and should work on other |
| BiRefNet exports that hit the same two blockers. |
|
|
| ```bash |
| pip install onnx onnxruntime numpy pillow |
| python scripts/patch_split.py model.onnx step1.onnx # Split trees, <= 6 outputs |
| python scripts/patch_deform.py step1.onnx patched.onnx # GatherND -> Gather, Sum -> Add chain |
| python scripts/verify_patch.py model.onnx patched.onnx a.jpg b.jpg # must report EXACT |
| ``` |
|
|
| `patch_deform.py` only rewrites the exact deform-conv `GatherND` pattern and refuses |
| to rewrite one unless it can see a clamp upstream, so an index cannot silently wrap. |
| `verify_patch.py` runs both graphs on the CPU EP; a structural rewrite should be |
| bit-identical, and it says so if it is not. |
|
|
| Two things that are easy to get wrong if you redo this: |
|
|
| - **Cast to int32 LATE.** Doing the index arithmetic in int32 doubles the |
| CPU<->GPU crossings (180 copies / 9.1s vs 80 / 3.4s). Do the maths in int64 and |
| cast once, immediately before the `Gather`. |
| - **Use a sequential `Add` chain, not a balanced tree.** fp16 addition is not |
| associative; a balanced tree changes the summation order and breaks |
| bit-identity with the original graph. |
|
|
| ## Measurements |
|
|
| Against BiRefNet-lite-512 on the same 30-image set, same compositor, same metric |
| (`edgeSharpness().gradient`): |
|
|
| | | all-30 gradient | products | interiorSoft | browser time (WebGPU) | |
| |---|---|---|---|---| |
| | BiRefNet-lite-512 | 0.0989 | 0.1095 | 0.00296 | 0.85s | |
| | **this (1024)** | **0.1436** | **0.1754** | **0.00043** | **3.4s** | |
|
|
| +45% edge gradient for ~4x the time. It also recovers subjects 512 misses |
| entirely — one test image returns an empty matte at 512 and 18.4% foreground here. |
|
|
| ⚠️ **Read the provenance before quoting these.** The **quality** columns and the |
| **timing** column come from different runs. The 1024 gradient figures were measured |
| in Node on the **CPU EP**, not on the WebGPU path the timings describe; graph |
| bit-identity was proven CPU-vs-CPU for the *rewrite*, which is not the same as |
| proving CPU and WebGPU produce identical output. And one flag is still open: this |
| graph's product `softEdgeRatio` is 0.01030 against a 0.01574 control — 35% lower — |
| which under an anti-aliasing guard means part of the gradient win **could be a |
| harder, less anti-aliased edge rather than a better one**. Band width (9.15) and the |
| crops both support a genuinely tighter edge, but it is not fully cleared. Treat +45% |
| as a strong indication, not a settled result, and re-measure on your own path. |
|
|
| ## Which one should you use |
|
|
| Not always this one. |
|
|
| - **512** is the better default for interactive, one-image-at-a-time UIs, for |
| batch jobs where latency multiplies by the file count, and for wide device |
| support. 0.85s is a different product feel from 3.4s. |
| - **1024 (this)** is worth it when edge quality is the point — hair, fur, fabric, |
| fine product edges — or when 512 misses the subject entirely. |
|
|
| Shipping both and letting the user choose is a reasonable design; that is what |
| Backgroundless does. Only one graph should be resident at a time — switching |
| disposes and recompiles. |
|
|
| ## Licensing |
|
|
| MIT — and unusually for a model, that covers the **weights**, not just the surrounding |
| code. No non-commercial clause, no separate model agreement to accept, and nothing in |
| it places any condition on the images you produce with it. |
|
|
| Worth checking on whatever you end up picking: licensing on browser matting models |
| varies a lot, and the gap between a project's *code* licence and its *weights* licence |
| is where people get caught. BRIA's RMBG-1.4 card, for example, states the model is |
| "available as a source-available model for non-commercial use" — and an MIT-licensed |
| wrapper repo around it does not change that, because the wrapper's licence covers the |
| wrapper. Read the weights licence, not the repo badge. |
|
|
| ⚠️ **One upstream caveat, in fairness.** BiRefNet_lite lists **DIS5K** among its |
| training sets, and DIS5K's own terms restrict commercial use of the dataset. That is an |
| open question for every model trained on it, this one included, and it is not something |
| an MIT grant downstream can resolve. What MIT does give you here is a clean, explicit |
| grant from the model's author covering the weights themselves — which is more than |
| several alternatives offer, but it is not a warranty about the training data. |
| |
| ## Credits and licence |
| |
| - Original model: **[ZhengPeng7/BiRefNet](https://github.com/ZhengPeng7/BiRefNet)** (MIT) — Peng Zheng et al. |
| - ONNX export this was derived from: **onnx-community/BiRefNet_lite-ONNX** |
| - **[studioludens/birefnet-lite-512](https://huggingface.co/studioludens/birefnet-lite-512)** — |
| Studio Ludens' 512 re-export, and the export method (PyTorch 2.0.1 + Kazuhito00's patched |
| deform-conv exporter) that made a working browser BiRefNet possible at all. Their model card is |
| also where the "1024-way concatenations" problem is documented, which is what sent us looking at |
| the decoder in the first place. **Backgroundless ships their 512 export as its default tier** — |
| this 1024 graph is the opt-in one. |
| - Kazuhito00's **deform-conv2d-onnx-exporter**, via the above. |
| - Graph surgery: this repo. MIT, same as upstream. |
| |
| If you use this, cite the BiRefNet authors — the model is theirs. The only thing |
| contributed here is the graph rewrite. |
| |
| ## Provenance |
| |
| `onnx/model_fp16.onnx` |
| sha256 `4059896039dfccb0f15b9080ff06d11d90e499449bb045e797055eb8901cf5f4` |
|
|
| Reproduce with `patch_split.py` then `patch_deform.py`, and verify with |
| `verify_patch.py` (correlation gate plus max-abs-diff against the unpatched |
| graph on the CPU EP). |
|
|
