README.md

---
license: mit
base_model: ZhengPeng7/BiRefNet
tags:
  - background-removal
  - image-matting
  - dichotomous-image-segmentation
  - birefnet
  - core-ai
  - apple-silicon
  - macos
library_name: swift-object-cutout
pipeline_tag: image-segmentation
---

# cutout-birefnet: BiRefNet general for Core AI (macOS 27)

[BiRefNet](https://github.com/ZhengPeng7/BiRefNet) general — Zheng, Gao, Fan, Liu, Laaksonen,
Ouyang and Sebe, *Bilateral Reference for High-Resolution Dichotomous Image Segmentation*, CAAI
Artificial Intelligence Research, 2024; code and weights MIT — exported to an Apple Core AI
`.aimodel`. Give it a photo and it returns a soft alpha matte of the main subject: a product, a
plant, a glass, an animal, a person — the background-removal job, on device.

It is what [`cutout`](https://github.com/arraypress/swift-cutout-cli) uses for `--subject` and
`--person`, through [swift-object-cutout](https://github.com/arraypress/swift-object-cutout), and
what `design` uses to lift a subject off its photo. Both download this repository by themselves the
first time it is needed, pinned to one commit, every file checked against its published hash.
Apple's built-in Vision matte stays as the no-download fallback.

## Files

| Path | What |
|---|---|
| `cutout-birefnet-float16.aimodel/` | One entry point, `main`: `image` `[1,3,1024,1024]` float32, RGB in 0…1 → `alpha` `[1,1,1024,1024]` float32 in 0…1. Float16 weights and arithmetic, 463 MB. |
| `export_birefnet.py` | The export script (`uv run export_birefnet.py --dtype float16`). |
| `LICENSE`, `NOTICE` | The authors' MIT licence, and what the export changed. |

The host resizes the picture to 1024×1024 without keeping its aspect ratio (as the network was
trained) and stretches the alpha back to the picture's size with bilinear sampling. ImageNet
normalisation and the sigmoid are inside the asset.

## What changed

Core AI has no deformable convolution, which BiRefNet's decoder uses. The export computes it from
what it is: for each kernel tap, a bilinear sample of the feature map (four gathers from a
zero-padded copy), scaled by the modulator, then a matrix product with that tap's weights, summed
over the taps. Sample positions stay float32 in the float16 export. Against the authors' code in
float32 on the CPU, the rewrite differs by at most 4.6 × 10⁻⁶ in alpha.

## Fidelity

Run through Core AI on the GPU, against the authors' PyTorch model (float32, CPU) on the same 30
inputs at 1024²:

| Export | Mean abs. alpha diff. | 99.9th percentile | Worst pixel | Mask IoU (>0.5), worst image |
|---|---|---|---|---|
| float32 | 1.9 × 10⁻⁷ | 2.4 × 10⁻⁵ | 1.3 × 10⁻⁴ | 0.99996 |
| float16 (this repo) | 1.9 × 10⁻⁵ | 1.5 × 10⁻³ | 0.068 | 0.9985 |

End to end (Swift, Core Graphics resizing) the float16 asset scores the same as PyTorch on the
true-alpha sets below: SAD 23.40 against 23.39 on objects and 6.12 against 6.10 on animals.

## Quality

Measured with the standard matting metrics against true alpha on AIM-500 (Li et al., 2021), which
this checkpoint was not trained on (its training sets are DIS5K, DUTS, HRSOD, UHRSD, HRS10K, P3M-10k
and a humans set). 100 objects (plants, furniture, toys, fruit, glass), 100 animals and the 100
portraits, at their own size (at most 1920 px). Lower is better except IoU; the edge band is the
uncertain strip round the subject where hair and fur live.

| Set | Matte | SAD | MSE ×10³ | Grad | Conn | IoU | Edge-band MSE ×10³ |
|---|---|---|---|---|---|---|---|
| Objects | Apple Vision subject mask, cutout's crisp edge | 81.39 | 33.09 | 0.690 | 82.82 | 0.802 | 92.5 |
| Objects | **this model** | **23.40** | **3.29** | **0.211** | **22.87** | **0.919** | **12.6** |
| Animals | Apple Vision subject mask, crisp edge | 17.17 | 5.79 | 0.309 | 17.15 | 0.976 | 47.7 |
| Animals | **this model** | **6.12** | **0.99** | **0.074** | **5.50** | **0.991** | **9.4** |
| Portraits | Apple Vision person segmentation, crisp edge | 24.91 | 10.63 | 0.224 | 24.45 | 0.962 | 36.0 |
| Portraits | **this model** | **4.86** | **0.62** | **0.058** | **4.19** | **0.992** | **6.7** |

It beat Vision by SAD on 99 of the 100 objects, all 100 animals and 99 of the 100 portraits. Vision found no subject at all in 4 of the objects (counted
as empty mattes); glass is the hardest category for both (SAD 211 for Vision, 51 here). Re-solving this
model's alpha from colour (the crisp edge that helps Vision's softer mask) made every set worse, so
cutout passes this model's alpha through.

## Speed and memory

Apple M3 Max, macOS 27, GPU: the first load compiles the model for the Mac in about 15 s and keeps a
553 MB compiled copy in `~/Library/Caches/coreai-cache`; later loads are quick. About 0.9 s per
1024² inference from Python, and 0.9–1.3 s per photo through `cutout` end to end (decode, model, edge,
PNG). A folder run of 100 photos peaked at 3.2–3.8 GB of memory, against 5.6 GB for the float32 export
and up to 10 GB for PyTorch on MPS.

## Use

```sh
cutout photo.jpg                 # fetches this model on first use
cutout ./shots --out ./cut       # a folder, the model loaded once
cutout model remove subject      # the model and its compiled copy
```

Or by hand: `hf download arraypress/cutout-birefnet --local-dir models`, then point
`$CUTOUT_SUBJECT_MODEL` at the `.aimodel`. Requires macOS 27 (Core AI) on Apple silicon. Converted
with coreai-torch 0.4.3, coreai-core 1.0.0b3, torch 2.9.0 and transformers 4.57.1.

## Licence

MIT, as the original: see `LICENSE` (the authors' copyright) and `NOTICE` (what the export changed).