File size: 2,067 Bytes
1088358 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 | ---
license: mit
library_name: transformers.js
base_model: onnx-community/BiRefNet_lite-ONNX
pipeline_tag: image-segmentation
tags:
- background-removal
- webgpu
---
# BiRefNet_lite (fp16), patched to run fully on WebGPU
This is [onnx-community/BiRefNet_lite-ONNX](https://huggingface.co/onnx-community/BiRefNet_lite-ONNX) (`onnx/model_fp16.onnx`), an ONNX export of [ZhengPeng7/BiRefNet_lite](https://github.com/ZhengPeng7/BiRefNet). The graph is rewritten so it runs entirely on WebGPU in the browser. The weights are unchanged.
## Why
On some GPUs the original fails in ONNX Runtime Web's WebGPU backend. It was measured on Intel Iris Xe in Chrome, where three problems showed up:
1. Concat/Split nodes with up to 1024 inputs or 32 outputs exceed the per-shader storage-buffer limit: "Too many storage buffers in shader. Current: 17, Max is 16".
2. The deformable-convolution index math is int64. WebGPU has no int64, so those nodes run on the CPU.
3. There is no WebGPU `Sum` kernel. Large fp16 tensors were therefore summed on the CPU, which went past the browser's 4 GB memory limit (`std::bad_alloc`).
## What changed
- Each oversized Concat or Split is now a tree of the same op, each using at most 8 buffers. 8 is WebGPU's guaranteed minimum.
- The int64 index chains are int32, with a cast back to int64 right before each `GatherND`.
- Each `Sum` is now a chain of `Add`s.
The output is bit-identical to the original: all 1,048,576 output values matched on a test image (onnxruntime-node, CPU).
## Usage (Transformers.js)
```js
import { AutoModel, AutoProcessor, RawImage } from "@huggingface/transformers";
const model = await AutoModel.from_pretrained("<this repo>", { dtype: "fp16", device: "webgpu" });
const processor = await AutoProcessor.from_pretrained("<this repo>");
const { pixel_values } = await processor(await RawImage.fromURL(url));
const { output_image } = await model({ input_image: pixel_values });
```
## Licence
MIT, the same as the original. See `LICENSE`. The original copyright notice is kept as the licence requires.
|