# Use SAM2 with videos
import torch
from sam2.sam2_video_predictor import SAM2VideoPredictor
predictor = SAM2VideoPredictor.from_pretrained("CardAnvilModels/sam2.1-hiera-tiny-ONNX")
with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
state = predictor.init_state(<your_video>)
# add new prompts and instantly get the output on the same frame
frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>)
# propagate the prompts to get masklets throughout the video
for frame_idx, object_ids, masks in predictor.propagate_in_video(state):
...SAM 2.1 Hiera Tiny (ONNX, fp16)
Segment Anything 2.1 (Hiera Tiny) as ONNX for Transformers.js, at every precision the source has. Card Anvil uses it in the browser to select the object under a click or inside a box.
These are unchanged copies of the files in onnx-community/sam2.1-hiera-tiny-ONNX at commit 814a066, at the same paths, kept here so the app loads files that can't change under it.
Card Anvil loads the three config files and the fp16 encoder and decoder (onnx/*_fp16.onnx and their .onnx_data), 78 MB. The other precisions are kept for later:
| Precision | Files in onnx/ |
Encoder + decoder |
|---|---|---|
| fp32 | vision_encoder, prompt_encoder_mask_decoder |
155 MB |
| fp16 (loaded) | *_fp16 |
78 MB |
| int8 / uint8 / quantized | *_int8, *_uint8, *_quantized |
61 MB |
| q4 | *_q4 |
51 MB |
| bnb4 | *_bnb4 |
49 MB |
| q4f16 | *_q4f16 |
33 MB |
Credits and licence
- Model: SAM 2 by Meta AI (
facebook/sam2.1-hiera-tiny), from SAM 2: Segment Anything in Images and Videos (2024). - ONNX conversion: ONNX Community.
Licensed under the Apache License 2.0, as the original model is. See LICENSE.
- Downloads last month
- 90
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for CardAnvilModels/sam2.1-hiera-tiny-ONNX
Base model
facebook/sam2.1-hiera-tiny
# Use SAM2 with images import torch from sam2.sam2_image_predictor import SAM2ImagePredictor predictor = SAM2ImagePredictor.from_pretrained("CardAnvilModels/sam2.1-hiera-tiny-ONNX") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): predictor.set_image(<your_image>) masks, _, _ = predictor.predict(<input_prompts>)