InSPyReNet (fastforks)
InSPyReNet (Kim et al., ACCV 2022): salient object detection with an inverse saliency pyramid: a Swin-B encoder, pyramid attention context modules and scale-invariant context attention that reconstruct the saliency map from a coarse map plus Laplacian residuals. The transparent-background models ComfyUI's InSPyReNet nodes run: inspyrenet (base, 1024 x 1024) and inspyrenet-fast (384 x 384), both trained on the authors' extra-data mix (Plus Ultra).
This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: https://github.com/kleinicke/fastforks. Kernels for the kernels backend: https://huggingface.co/fastforks/kernels. Original code: https://github.com/plemeri/transparent-background at f0fa91701a98.
Task: matting. Tier: experimental. Outputs: alpha. Backends: portable.
Input: one RGB image (trimap-guided models also a trimap: background / unknown / foreground). Output: an alpha matte [H, W] in [0, 1] at the input size: 1 = foreground (background removal, dichotomous / salient object segmentation).
Variants
Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in card.toml.
| variant | weights (author link) | SHA-256 | weights licence | commercial use |
|---|---|---|---|---|
inspyrenet@1 |
ckpt_base.pth | 0a6fe2a73ab0 |
MIT | yes |
inspyrenet-fast@1 |
ckpt_fast.pth | e61ed7e85763 |
MIT | yes |
Licence
Original-code strings, recorded as they are (research-use package; check them before any other use). Code: MIT. Licence files: LICENSE.
Training data: DUTS (salient object detection), HRSOD (high-resolution salient object detection), UHRSD (ultra-high-resolution salient object detection), DIS5K (dichotomous image segmentation) (plemeri/InSPyReNet docs/model_zoo.md, Plus_Ultra(_LR): DUTS-TR/TE, HRSOD-TR/TE, UHRSD-TR/TE, DIS-TR/VD/TE1-4, DUT-OMRON, ECSSD, HKU-IS, PASCAL-S, DAVIS-S, FSS-1000, MSRA-10K (all test sets included, so benchmark numbers on them are not held out). Swin-B backbone from ImageNet-22k -> 1k.).
Licence evidence: https://github.com/plemeri/transparent-background/blob/f0fa91701a98cfc8e955c554e84522f365ec6da3/LICENSE ; https://github.com/plemeri/transparent-background/releases/tag/1.2.12 ; https://github.com/plemeri/InSPyReNet/blob/692ad8a5774cf95ef615a730d5070aad18f6b160/docs/model_zoo.md
Input size
any H x W >= 32, padded to a multiple of 32
Tested up to (portable, fp16, RTX 5060 Ti 16 GB): inspyrenet 3000x4000 (2.0 GiB peak allocated, 2.4 reserved); inspyrenet-fast 3000x4000 (0.9 GiB peak allocated, 1.0 reserved)
Speed and memory
Matting, synthetic, 1080x1920
| model | time (ms): original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels |
|---|---|---|---|---|---|---|---|---|
inspyrenet |
241 | 109 | β | β | β | 3240 / 3262 | 1649 / 1848 | β |
inspyrenet-fast |
71.1 | 41.7 | β | β | β | 810 / 836 | 432 / 488 | β |
Startup and first call
| model | load (s): original | portable | kernels | first call (s): original | portable | kernels |
|---|---|---|---|---|---|---|
inspyrenet |
6.68 | 0.78 | β | 1.49 | 0.62 | β |
inspyrenet-fast |
5.93 | 11.07 | β | 1.33 | 0.79 | β |
torch.compile and TensorRT
| model | input | time: portable | compile | TensorRT fp16 | kernels | preparation: compile (cold first call) | ONNX + engine build | peak MiB (alloc / reserved): portable | compile | kernels | TensorRT (engine + torch) |
|---|---|---|---|---|---|---|---|---|---|---|---|
inspyrenet |
1080x1920 | 109 ms | 85.8 ms | 51.1 ms | β | 70.8 s | 133 s | 1649 / 1848 | 913 / 1018 | β | 1181 / 1252 |
inspyrenet-fast |
1080x1920 | 41.7 ms | 17.7 ms | 6.99 ms | β | 76.6 s | 138 s | 432 / 488 | 345 / 376 | β | 561 / 600 |
Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).
Hardware, protocol and caveats: benchmarks (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or (fp32) where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.
Accuracy
On public datasets
Each image at its own size against its binary mask: MAE of the alpha, max F-beta (beta^2 = 0.3) over 256 thresholds (BiRefNet's evaluation). Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | portable | note |
|---|---|---|---|---|
inspyrenet |
DIS5K validation (DIS-VD), every 2nd image (235), MAE / max F-beta | 0.0129 / 0.9699 | 0.0129 / 0.9699 | |
inspyrenet-fast |
DIS5K validation (DIS-VD), every 2nd image (235), MAE / max F-beta | 0.0420 / 0.8952 | 0.0420 / 0.8952 |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 45a5c91f, 2026-11-13.
Parity with the original code
Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.02 and p99 <= 0.25 (absolute error of the output alpha).
Usage
from fastforks import matting
model = matting.load("inspyrenet") # mode="auto": the preferred backend from recorded measurements
result = model("photo.jpg")
result.alpha # [H, W] float32 in [0, 1], input resolution
result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')
Install with pip install fastforks (see https://github.com/kleinicke/fastforks). Other variants: inspyrenet, inspyrenet-fast.
Kernels backend
No kernels backend: measured slower: cudnn_hacc convolutions 189.9 ms vs 159.4 ms portable (1080x1920, size 1024; upstream views their outputs, so channels-last needs copies); fp16-accumulate GEMMs alone 3 %.