license: other
license_name: sam-license
license_link: https://huggingface.co/facebook/sam3/blob/main/LICENSE
base_model: facebook/sam3
pipeline_tag: mask-generation
library_name: coreai
tags:
- core-ai
- ios
- video-segmentation
- sam3
SAM 3 video for Core AI
facebook/sam3 exported to Apple Core AI as a video segmenter with the apple/coreai-models recipe, at 672 px for iPhone:
uv run models/sam3_video/export.py --dtype float16 --image-size 672
| Folder | Contents | Size | Platforms |
|---|---|---|---|
sam3-video/ |
CoreAIVideoSegmentation bundle: sam3_video_672_float16.aimodel (image and text encoders, detector, tracker and memory encoder functions), the CLIP tokenizer/, metadata.json (kind: video_segmenter, tracking defaults) |
1.6 GB | iOS, macOS |
Give it a video and text prompts (cat, the red car). It finds the matching objects and tracks them from frame to frame with stable ids. The recipe also accepts 336 px (faster, less accurate on small objects) and 1008 px (the default, slower).
Use
With @react-native-ai/core-ai, download sam3-video/ into app storage, then:
const { output } = await CoreAI.runTask(
'video-segmentation',
{ id: 'sam3-video', sourceType: 'file', sourceUri: pathToFolder },
{ videoUri, data: { prompts: ['cat'] } },
{ outputVideo: true }
)
In Swift, load the folder with VideoSegmenter(resourcesAt:) from the CoreAIVideoSegmentation product of coreai-models.
License
SAM 3 is licensed by Meta under the SAM License, which applies to this converted copy too: a copy of the agreement is included, as it requires. This repository only converts the model to Core AI format.