InSPyReNet (fastforks)

InSPyReNet (Kim et al., ACCV 2022): salient object detection with an inverse saliency pyramid: a Swin-B encoder, pyramid attention context modules and scale-invariant context attention that reconstruct the saliency map from a coarse map plus Laplacian residuals. The transparent-background models ComfyUI's InSPyReNet nodes run: inspyrenet (base, 1024 x 1024) and inspyrenet-fast (384 x 384), both trained on the authors' extra-data mix (Plus Ultra).

This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: https://github.com/kleinicke/fastforks. Kernels for the kernels backend: https://huggingface.co/fastforks/kernels. Original code: https://github.com/plemeri/transparent-background at f0fa91701a98.

Task: matting. Tier: experimental. Outputs: alpha. Backends: portable.

Input: one RGB image (trimap-guided models also a trimap: background / unknown / foreground). Output: an alpha matte [H, W] in [0, 1] at the input size: 1 = foreground (background removal, dichotomous / salient object segmentation).

Variants

Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in card.toml.

variant weights (author link) SHA-256 weights licence commercial use
inspyrenet@1 ckpt_base.pth 0a6fe2a73ab0 MIT yes
inspyrenet-fast@1 ckpt_fast.pth e61ed7e85763 MIT yes

Licence

Original-code strings, recorded as they are (research-use package; check them before any other use). Code: MIT. Licence files: LICENSE.

Training data: DUTS (salient object detection), HRSOD (high-resolution salient object detection), UHRSD (ultra-high-resolution salient object detection), DIS5K (dichotomous image segmentation) (plemeri/InSPyReNet docs/model_zoo.md, Plus_Ultra(_LR): DUTS-TR/TE, HRSOD-TR/TE, UHRSD-TR/TE, DIS-TR/VD/TE1-4, DUT-OMRON, ECSSD, HKU-IS, PASCAL-S, DAVIS-S, FSS-1000, MSRA-10K (all test sets included, so benchmark numbers on them are not held out). Swin-B backbone from ImageNet-22k -> 1k.).

Licence evidence: https://github.com/plemeri/transparent-background/blob/f0fa91701a98cfc8e955c554e84522f365ec6da3/LICENSE ; https://github.com/plemeri/transparent-background/releases/tag/1.2.12 ; https://github.com/plemeri/InSPyReNet/blob/692ad8a5774cf95ef615a730d5070aad18f6b160/docs/model_zoo.md

Input size

any H x W >= 32, padded to a multiple of 32 Tested up to (portable, fp16, RTX 5060 Ti 16 GB): inspyrenet 3000x4000 (2.0 GiB peak allocated, 2.4 reserved); inspyrenet-fast 3000x4000 (0.9 GiB peak allocated, 1.0 reserved)

Speed and memory

Matting, synthetic, 1080x1920

model time (ms): original portable kernels kernels speed-up: vs original vs portable peak memory (MiB, allocated / reserved): original portable kernels
inspyrenet 241 109 – – – 3240 / 3262 1649 / 1848 –
inspyrenet-fast 71.1 41.7 – – – 810 / 836 432 / 488 –

Startup and first call

model load (s): original portable kernels first call (s): original portable kernels
inspyrenet 6.68 0.78 – 1.49 0.62 –
inspyrenet-fast 5.93 11.07 – 1.33 0.79 –

torch.compile and TensorRT

model input time: portable compile TensorRT fp16 kernels preparation: compile (cold first call) ONNX + engine build peak MiB (alloc / reserved): portable compile kernels TensorRT (engine + torch)
inspyrenet 1080x1920 109 ms 85.8 ms 51.1 ms – 70.8 s 133 s 1649 / 1848 913 / 1018 – 1181 / 1252
inspyrenet-fast 1080x1920 41.7 ms 17.7 ms 6.99 ms – 76.6 s 138 s 432 / 488 345 / 376 – 561 / 600

Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).

Hardware, protocol and caveats: benchmarks (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or (fp32) where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.

Accuracy

On public datasets

Each image at its own size against its binary mask: MAE of the alpha, max F-beta (beta^2 = 0.3) over 256 thresholds (BiRefNet's evaluation). Original = the original code named under Details, same checkpoint and protocol.

variant dataset original fp32 portable note
inspyrenet DIS5K validation (DIS-VD), every 2nd image (235), MAE / max F-beta 0.0129 / 0.9699 0.0129 / 0.9699
inspyrenet-fast DIS5K validation (DIS-VD), every 2nd image (235), MAE / max F-beta 0.0420 / 0.8952 0.0420 / 0.8952

Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 45a5c91f, 2026-11-13.

Parity with the original code

Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.02 and p99 <= 0.25 (absolute error of the output alpha).

Usage

from fastforks import matting

model = matting.load("inspyrenet")  # mode="auto": the preferred backend from recorded measurements
result = model("photo.jpg")
result.alpha  # [H, W] float32 in [0, 1], input resolution
result.backend  # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')

Install with pip install fastforks (see https://github.com/kleinicke/fastforks). Other variants: inspyrenet, inspyrenet-fast.

Kernels backend

No kernels backend: measured slower: cudnn_hacc convolutions 189.9 ms vs 159.4 ms portable (1080x1920, size 1024; upstream views their outputs, so channels-last needs copies); fp16-accumulate GEMMs alone 3 %.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Collection including fastforks/inspyrenet