See our collection for all versions of SAM.

Run SAM3 with Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/sam3

Paper: SAM 3: Segment Anything with Concepts (arXiv:2511.16719) · HF Papers

SAM3 segments by concept, not location: give it a noun phrase and it finds every matching instance. A ViT-L backbone and FPN feed a DETR-style encoder/decoder with object queries; a CLIP text encoder supplies the open-vocabulary side. Boxes can still be mixed with text.

For more details on the model, please go to Meta's original model card.

Pure-Keras 3 conversion of facebook/sam3 for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a concept-prompted checkpoint (SAM3InstanceSegment / SAM3Detect / SAM3SemanticSegment): pass a text noun phrase (backbone ViT-L/14).

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from zeromodels.models.sam3 import SAM3InstanceSegment

segmenter = SAM3InstanceSegment.from_weights("zeromodels/sam3")
result = segmenter.predict(
    images="your_image.jpg", text="person", threshold=0.3
)[0]
print(len(result["scores"]), result["masks"].shape)

Load any SAM / SAM2 / SAM3 variant the same way with from_weights("zeromodels/<variant>") (use SAM3InstanceSegment for this repo):

Variant	Hub	Family
`sam_vit_base`	`zeromodels/sam_vit_base`	SAM
`sam_vit_large`	`zeromodels/sam_vit_large`	SAM
`sam_vit_huge`	`zeromodels/sam_vit_huge`	SAM
`sam2_hiera_small`	`zeromodels/sam2_hiera_small`	SAM2
`sam2_hiera_base_plus`	`zeromodels/sam2_hiera_base_plus`	SAM2
`sam2_hiera_large`	`zeromodels/sam2_hiera_large`	SAM2
`sam3`	`zeromodels/sam3`	SAM3

Tips

Set KERAS_BACKEND before importing Keras / zeromodels.
SAM / SAM2: point coordinates are in original pixel space; box prompts need enable_boxes=True / include_box_input=True when building the graph.
SAM2 in this port is image-only (no video memory bank).
SAM3: prefer SAM3InstanceSegment.predict(...) for text prompts; upstream facebook/sam3 is gated.
See SAM3 docs and Loading Weights.
Community / upstream safetensors still work via the hf: prefix, e.g. SAM3Model.from_weights("hf:facebook/sam3").