File size: 2,179 Bytes
1e06d16 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | ---
license: other
license_name: sam-license
license_link: https://huggingface.co/facebook/sam3/blob/main/LICENSE
base_model: facebook/sam3
pipeline_tag: mask-generation
library_name: coreai
tags:
- core-ai
- ios
- video-segmentation
- sam3
---
# SAM 3 video for Core AI
[`facebook/sam3`](https://huggingface.co/facebook/sam3) exported to Apple Core AI as a video segmenter with the [`apple/coreai-models`](https://github.com/apple/coreai-models/tree/main/models/sam3_video) recipe, at 672 px for iPhone:
```sh
uv run models/sam3_video/export.py --dtype float16 --image-size 672
```
| Folder | Contents | Size | Platforms |
| ------------- | ------------------------------------------------------------------------------------------------------------------------------ | ------ | ---------- |
| `sam3-video/` | `CoreAIVideoSegmentation` bundle: `sam3_video_672_float16.aimodel` (image and text encoders, detector, tracker and memory encoder functions), the CLIP `tokenizer/`, `metadata.json` (`kind: video_segmenter`, tracking defaults) | 1.6 GB | iOS, macOS |
Give it a video and text prompts (`cat`, `the red car`). It finds the matching objects and tracks them from frame to frame with stable ids. The recipe also accepts 336 px (faster, less accurate on small objects) and 1008 px (the default, slower).
## Use
With [`@react-native-ai/core-ai`](https://github.com/callstackincubator/ai/tree/main/packages/core-ai), download `sam3-video/` into app storage, then:
```ts
const { output } = await CoreAI.runTask(
'video-segmentation',
{ id: 'sam3-video', sourceType: 'file', sourceUri: pathToFolder },
{ videoUri, data: { prompts: ['cat'] } },
{ outputVideo: true }
)
```
In Swift, load the folder with `VideoSegmenter(resourcesAt:)` from the `CoreAIVideoSegmentation` product of `coreai-models`.
## License
SAM 3 is licensed by Meta under the [SAM License](./LICENSE), which applies to this converted copy too: a copy of the agreement is included, as it requires. This repository only converts the model to Core AI format.
|