cutout-birefnet: BiRefNet general for Core AI (macOS 27)

BiRefNet general — Zheng, Gao, Fan, Liu, Laaksonen, Ouyang and Sebe, Bilateral Reference for High-Resolution Dichotomous Image Segmentation, CAAI Artificial Intelligence Research, 2024; code and weights MIT — exported to an Apple Core AI .aimodel. Give it a photo and it returns a soft alpha matte of the main subject: a product, a plant, a glass, an animal, a person — the background-removal job, on device.

It is what cutout uses for --subject and --person, through swift-object-cutout, and what design uses to lift a subject off its photo. Both download this repository by themselves the first time it is needed, pinned to one commit, every file checked against its published hash. Apple's built-in Vision matte stays as the no-download fallback.

Files

Path	What
cutout-birefnet-float16.aimodel/	One entry point, main: image [1,3,1024,1024] float32, RGB in 0…1 → alpha [1,1,1024,1024] float32 in 0…1. Float16 weights and arithmetic, 463 MB.
export_birefnet.py	The export script (uv run export_birefnet.py --dtype float16).
LICENSE, NOTICE	The authors' MIT licence, and what the export changed.

The host resizes the picture to 1024×1024 without keeping its aspect ratio (as the network was trained) and stretches the alpha back to the picture's size with bilinear sampling. ImageNet normalisation and the sigmoid are inside the asset.

What changed

Core AI has no deformable convolution, which BiRefNet's decoder uses. The export computes it from what it is: for each kernel tap, a bilinear sample of the feature map (four gathers from a zero-padded copy), scaled by the modulator, then a matrix product with that tap's weights, summed over the taps. Sample positions stay float32 in the float16 export. Against the authors' code in float32 on the CPU, the rewrite differs by at most 4.6 × 10⁻⁶ in alpha.

Fidelity

Run through Core AI on the GPU, against the authors' PyTorch model (float32, CPU) on the same 30 inputs at 1024²:

Export	Mean abs. alpha diff.	99.9th percentile	Worst pixel	Mask IoU (>0.5), worst image
float32	1.9 × 10⁻⁷	2.4 × 10⁻⁵	1.3 × 10⁻⁴	0.99996
float16 (this repo)	1.9 × 10⁻⁵	1.5 × 10⁻³	0.068	0.9985

End to end (Swift, Core Graphics resizing) the float16 asset scores the same as PyTorch on the true-alpha sets below: SAD 23.40 against 23.39 on objects and 6.12 against 6.10 on animals.

Quality

Measured with the standard matting metrics against true alpha on AIM-500 (Li et al., 2021), which this checkpoint was not trained on (its training sets are DIS5K, DUTS, HRSOD, UHRSD, HRS10K, P3M-10k and a humans set). 100 objects (plants, furniture, toys, fruit, glass), 100 animals and the 100 portraits, at their own size (at most 1920 px). Lower is better except IoU; the edge band is the uncertain strip round the subject where hair and fur live.

Set	Matte	SAD	MSE ×10³	Grad	Conn	IoU	Edge-band MSE ×10³
Objects	Apple Vision subject mask, cutout's crisp edge	81.39	33.09	0.690	82.82	0.802	92.5
Objects	this model	23.40	3.29	0.211	22.87	0.919	12.6
Animals	Apple Vision subject mask, crisp edge	17.17	5.79	0.309	17.15	0.976	47.7
Animals	this model	6.12	0.99	0.074	5.50	0.991	9.4
Portraits	Apple Vision person segmentation, crisp edge	24.91	10.63	0.224	24.45	0.962	36.0
Portraits	this model	4.86	0.62	0.058	4.19	0.992	6.7

It beat Vision by SAD on 99 of the 100 objects, all 100 animals and 99 of the 100 portraits. Vision found no subject at all in 4 of the objects (counted as empty mattes); glass is the hardest category for both (SAD 211 for Vision, 51 here). Re-solving this model's alpha from colour (the crisp edge that helps Vision's softer mask) made every set worse, so cutout passes this model's alpha through.

Speed and memory

Apple M3 Max, macOS 27, GPU: the first load compiles the model for the Mac in about 15 s and keeps a 553 MB compiled copy in ~/Library/Caches/coreai-cache; later loads are quick. About 0.9 s per 1024² inference from Python, and 0.9–1.3 s per photo through cutout end to end (decode, model, edge, PNG). A folder run of 100 photos peaked at 3.2–3.8 GB of memory, against 5.6 GB for the float32 export and up to 10 GB for PyTorch on MPS.

Use

cutout photo.jpg                 # fetches this model on first use
cutout ./shots --out ./cut       # a folder, the model loaded once
cutout model remove subject      # the model and its compiled copy

Or by hand: hf download arraypress/cutout-birefnet --local-dir models, then point $CUTOUT_SUBJECT_MODEL at the .aimodel. Requires macOS 27 (Core AI) on Apple silicon. Converted with coreai-torch 0.4.3, coreai-core 1.0.0b3, torch 2.9.0 and transformers 4.57.1.

Licence

MIT, as the original: see LICENSE (the authors' copyright) and NOTICE (what the export changed).