birefnet-lite-web

BiRefNet_lite exported to ONNX so that it runs inside a web browser with onnxruntime-web, on WebGPU or on WebAssembly. It powers the background remover on clevr.tools. The weights are BiRefNet_lite's, untouched: nothing was retrained or fine-tuned.

File	Input	Intended backend	Size
birefnet_lite_768_fp16w.onnx	1 × 3 × 768 × 768	WebGPU	104,329,641 bytes
birefnet_lite_512_fp16w.onnx	1 × 3 × 512 × 512	WASM fallback	101,066,931 bytes

SHA-256 checksums are in lib/bgremove/manifest.json of the clevr.tools repository, which is how the site verifies them before use.

What we changed, and why

The stock ONNX export of BiRefNet does not run in a browser. We changed four things:

1. Deformable convolution, computed in chunks. The stock export turns each deformable convolution into GatherND operations over tensors of about 822 MB each, which exceed browser memory. We recompute it one kernel position at a time: bilinear-sample the input at the learned offsets (four flat Gathers with validity masks), multiply by the modulation mask, apply a 1 × 1 convolution with that position's weight, and accumulate. It matches torchvision's deform_conv2d to about 1e-5. Batch size is 1.
2. Narrower Split operators. WebGPU in Chrome on a Mac allows 10 storage buffers per shader stage; 66 decoder Splits with 8 to 32 outputs exceeded it. They are regrouped as a coarse split followed by 8-way splits (bit-identical output).
3. Static input sizes. The patch-size constants are baked in, so 768 and 512 are separate exports.
4. Half-precision weight storage. Float32 weights are stored as float16 and cast back to float32 when the model loads, which halves the download. The arithmetic is still float32. (A full float16 conversion did not load in onnxruntime-web, so it was not used.) The graph was also optimised once with ONNX Runtime at the basic level before this conversion, which cuts the node count from about 62,800 to 27,500 and shortens session creation.

Validation

Everything below was measured with ONNX Runtime on CPU on an Apple M4, on a fixed 96-image sample: 8 photos from each of the 6 object categories of AIM-500 plus 20 portraits from each of the two P3M-500 sets. The images are used for evaluation only and are not redistributed here. Score is mean IoU against the reference alpha matte (threshold 0.5). "Refined" means the model's matte was upsampled to the photo's size and passed through a guided filter on the photo; that step lives in the clevr.tools app, not in these files.

	Mean IoU, raw	Mean IoU, refined	Photos with IoU < 0.5
Stock BiRefNet_lite export, 1024	0.9606	not measured	not measured
Chunked deformable conv, 1024 (fp32)	0.9605	0.9699	0 of 96
768 (shipped for WebGPU)	0.9556	0.9651	0 of 96
512 (shipped for WASM)	0.9438	0.9540	2 raw, 1 refined of 96

Checks that the other changes are neutral, both at 768: float16 weight storage gave 0.95561 raw / 0.96506 refined, the same as fp32 to five decimal places; the basic-optimised graph gave 0.96506 refined, again the same, with 0 of 96 failing. The 512 model was checked the same way (0.9540 refined, 1 of 96).

Where it is weak. Glass and other see-through objects: the 8 "transparent" images in the sample scored 0.754 (768, refined) and 0.692 (768, raw). Fine hair on a busy background and objects close in color to their background are also harder. These are properties of the base model on these sets, not of the conversion.

Browser timings (Chrome on a Mac, M4): see the clevr.tools benchmark notes. Numbers on other devices are not claimed here.

Using it

import * as ort from "onnxruntime-web/webgpu";
const session = await ort.InferenceSession.create(bytes, { executionProviders: ["webgpu"] });
// input_image: float32 [1, 3, S, S], pixels / 255 normalised with mean [0.485, 0.456, 0.406] and std [0.229, 0.224, 0.225]
const out = await session.run({ input_image: new ort.Tensor("float32", chw, [1, 3, S, S]) });
// output_image: logits [1, 1, S, S]; apply a sigmoid to get the matte in 0..1

The photo is resized to S × S (no aspect-ratio padding) and the matte resized back.

Reproducing

The export, the chunked deformable convolution, the Split rewrite, the weight conversion and the benchmark harness are in bench/background-removal of the clevr.tools repository.

Credit and licence

BiRefNet: Peng Zheng, Dehong Gao, Deng-Ping Fan, Li Liu, Jorma Laaksonen, Wanli Ouyang, Nicu Sebe, Bilateral Reference for High-Resolution Dichotomous Image Segmentation, CAAI Artificial Intelligence Research, 2024, arXiv:2401.03407. Original code and weights: ZhengPeng7/BiRefNet, MIT. These files are released under the same MIT licence (see LICENSE and NOTICE). Please cite the original paper if you use them.