| --- |
| license: mit |
| base_model: ZhengPeng7/BiRefNet_lite |
| tags: |
| - background-removal |
| - image-segmentation |
| - dichotomous-image-segmentation |
| - onnx |
| - coreml |
| pipeline_tag: image-segmentation |
| --- |
| |
| # BiRefNet_lite — ONNX re-export for the CoreML execution provider |
| |
| This is **not** a new model. It is [BiRefNet_lite](https://huggingface.co/ZhengPeng7/BiRefNet_lite) |
| by **Peng Zheng et al.** ([BiRefNet](https://github.com/ZhengPeng7/BiRefNet), MIT), re-exported to |
| ONNX so that ONNX Runtime's **CoreML execution provider** (Apple Silicon GPU) runs the whole graph |
| in a single partition. All credit for the model goes to the BiRefNet authors. |
| |
| ## Why a re-export |
| |
| The published ONNX exports don't work well with the CoreML EP: the deformable convolution exports |
| as huge `GatherND` intermediates (>4 GB on CPU), and CoreML either fails to compile it or splits the |
| graph into ~100 partitions. This export (scripts in `scripts/`) changes only how operations are |
| expressed, not the weights: |
| |
| 1. `deform_conv2d` → one bilinear `GridSample` per kernel tap, accumulated with 1×1 convolutions |
| (numerically equivalent, max logit difference ~5e-5). |
| 2. Swin window partition/reverse and `image2patches` reshaped to rank ≤ 5 (CoreML's limit). |
| 3. `qkv[0..2]` indexing → `unbind`. |
| 4. Shapes fixed to 1024×1024, ORT basic constant folding, and Gemm weights rewritten to |
| `transB=1` (`fix_gemm.py`) so the CoreML compile stays small and fast. |
|
|
| Result: 6924/6924 nodes on CoreML, ~0.4 s per image on an M3 Pro (CPU+GPU, MLProgram), output |
| matching the CPU EP to within 2e-4 in alpha. |
|
|
| ## Usage |
|
|
| - File: `model.onnx` (fp32) |
| - Input `input_image`: `[1, 3, 1024, 1024]` float32 — RGB resized to 1024×1024 (bilinear, no crop), |
| `(v/255 − mean) / std` with mean `[0.485, 0.456, 0.406]`, std `[0.229, 0.224, 0.225]`. |
| - Output `output_image`: `[1, 1, 1024, 1024]` **logits** — apply a sigmoid, resize to the image size. |
| - CoreML EP: MLProgram format, static shapes, compute units CPU+GPU. |
|
|
| Used by Mixer, a personal D&D session app, for in-app background removal of character art. |
|
|
| ## License |
|
|
| MIT, same as the original BiRefNet. Please cite the original work: |
|
|
| ``` |
| @article{zheng2024birefnet, |
| title={Bilateral Reference for High-Resolution Dichotomous Image Segmentation}, |
| author={Zheng, Peng and Gao, Dehong and Fan, Deng-Ping and Liu, Li and Laaksonen, Jorma and Ouyang, Wanli and Sebe, Nicu}, |
| journal={CAAI Artificial Intelligence Research}, |
| year={2024} |
| } |
| ``` |
|
|
