gumka/sam2.1-small-coreml

SAM 2.1 Small for Core ML

Apple's float16 Core ML conversion of SAM 2.1 Small, in three packages, as used by the Select object tool of Gumka on macOS: a click on an object gives its mask, and more clicks add to it or take parts away.

Package	Input	Output
SAM2_1SmallImageEncoderFLOAT16.mlpackage	image 1024 × 1024 BGRA (the picture stretched)	image_embedding [1, 256, 64, 64], feats_s0 [1, 32, 256, 256], feats_s1 [1, 64, 128, 128]
SAM2_1SmallPromptEncoderFLOAT16.mlpackage	points [1, N, 2] in 1024 × 1024 coordinates, labels [1, N] (1 includes, 0 excludes)	sparse_embeddings, dense_embeddings
SAM2_1SmallMaskDecoderFLOAT16.mlpackage	the five arrays above	low_res_masks [1, 3, 256, 256] (scores; above 0 is selected), scores [1, 3]

• The image encoder runs once per picture; each click runs only the prompt encoder and the mask decoder. Gumka keeps the best of the three masks.
• The mask decoder takes 2 to 15 prompt tokens: the clicks and a padding point, so one selection takes at most 14 clicks.
• SAM2 Studio is the reference app for wiring them together.
• The ONNX version, for Linux, is gumka/sam2.1-small-onnx.

Source and license

• Model: SAM 2, "SAM 2: Segment Anything in Images and Videos", Nikhila Ravi et al., 2024, https://arxiv.org/abs/2408.00714, https://github.com/facebookresearch/sam2 (Apache-2.0). Weights: SAM 2.1 Hiera Small, facebook/sam2.1-hiera-small.
• Core ML conversion: Apple, in float16, with Hugging Face's conversion fork, published at apple/coreml-sam2.1-small (Apache-2.0).
• This repository holds the three packages of apple/coreml-sam2.1-small at revision 883f5787eb0b, unchanged; Gumka's models.toml pins each file's size and SHA-256. License: Apache-2.0, see LICENSE.