| --- |
| base_model: |
| - facebook/sam3.1 |
| license: other |
| license_name: sam-license |
| license_link: LICENSE |
| language: |
| - en |
| library_name: transformers |
| pipeline_tag: mask-generation |
| tags: |
| - unsloth |
| - sam3 |
| - sam3.1 |
| - segment-anything |
| - segmentation |
| - text-prompted-segmentation |
| - inpainting |
| - bf16 |
| - fp8 |
| - int8 |
| --- |
| |
| <div> |
| <div style="display: flex; gap: 5px; align-items: center; "> |
| <a href="https://github.com/unslothai/unsloth/"> |
| <img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133"> |
| </a> |
| <a href="https://discord.gg/unsloth"> |
| <img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173"> |
| </a> |
| <a href="https://docs.unsloth.ai/"> |
| <img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143"> |
| </a> |
| </div> |
| </div> |
| |
| SAM 3.1 by Meta in the transformers `Sam3Model` format: give it an image and a text prompt such as "sofa" and it returns a mask, a box and a score for every match. It powers "Mask from text" in [Unsloth Studio](https://github.com/unslothai/unsloth/). |
|
|
| | Prompts: sofa, red lamp, pillow, potted plant, rug | |
| |---| |
| | <img src="samples/living_room.png" width="900"> | |
|
|
| | Prompts: person, bicycle, dog, yellow car | |
| |---| |
| | <img src="samples/street.png" width="900"> | |
|
|
| | File | Size | Mask IoU vs BF16 (mean / min) | |
| |---|---|---| |
| | `model.safetensors` (BF16) | 1.68 GB | 1.0000 / 1.0000 | |
| | `sam3.1-INT8-ConvRot.safetensors` | 1.05 GB | 0.9984 / 0.9913 | |
| | `sam3.1-INT8.safetensors` | 0.99 GB | 0.9976 / 0.9829 | |
| | `sam3.1-FP8.safetensors` | 1.05 GB | 0.9971 / 0.9805 | |
|
|
| 53 text prompts on 6 images; 1.0 means identical masks. Unsloth Studio uses INT8-ConvRot on NVIDIA GPUs and BF16 elsewhere. |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| from PIL import Image |
| from transformers import Sam3Model, Sam3Processor |
| |
| model = Sam3Model.from_pretrained("unsloth/sam3.1", dtype=torch.bfloat16).to("cuda").eval() |
| processor = Sam3Processor.from_pretrained("unsloth/sam3.1") |
| |
| image = Image.open("living_room.png").convert("RGB") |
| inputs = processor(images=image, text="sofa", return_tensors="pt").to("cuda", torch.bfloat16) |
| with torch.inference_mode(): |
| outputs = model(**inputs) |
| results = processor.post_process_instance_segmentation( |
| outputs, threshold=0.5, mask_threshold=0.5, target_sizes=inputs.get("original_sizes").tolist() |
| )[0] |
| print(results["scores"], results["boxes"], results["masks"].shape) |
| ``` |
|
|
| --- |
|
|
| # SAM 3.1 |
|
|
| [SAM 3](https://github.com/facebookresearch/sam3) (Segment Anything with Concepts) is a unified foundation model from Meta for promptable segmentation in images and videos. It detects, segments, and tracks objects using text or visual prompts such as points, boxes, and masks. SAM 3 introduces the ability to exhaustively segment all instances of an open-vocabulary concept specified by a short text phrase, handling over 50x more unique concepts than existing benchmarks. SAM 3.1 builds on this with **Object Multiplex**, a shared-memory approach for joint multi-object tracking that delivers ~7x faster inference at 128 objects on a single H100 GPU without sacrificing accuracy, along with improved VOS performance on 6 out of 7 benchmarks. |
|
|
| <p align="center"> |
| <img src="https://github.com/facebookresearch/sam3/blob/main/assets/sam3.1_diagram.png?raw=true" width="720" /> |
| </p> |
|
|
| For the video tracker, installation, code and full documentation, see the [SAM 3 GitHub repository](https://github.com/facebookresearch/sam3) and [facebook/sam3.1](https://huggingface.co/facebook/sam3.1). |
|
|
| ## License |
|
|
| SAM 3.1 is released by Meta under the SAM License. This repository is a derivative of the SAM Materials and is distributed under the same agreement; the full text is in [LICENSE](LICENSE). By using or distributing these weights you agree to the SAM License. If you publish research results that use this model, the license asks you to acknowledge the use of the SAM Materials. Not an official Meta release, and not endorsed by Meta. |
|
|
