Instructions to use ipsilondev/sam2.1-hiera-tiny-ONNX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sam2
How to use ipsilondev/sam2.1-hiera-tiny-ONNX with sam2:
# Use SAM2 with images import torch from sam2.sam2_image_predictor import SAM2ImagePredictor predictor = SAM2ImagePredictor.from_pretrained("ipsilondev/sam2.1-hiera-tiny-ONNX") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): predictor.set_image(<your_image>) masks, _, _ = predictor.predict(<input_prompts>)# Use SAM2 with videos import torch from sam2.sam2_video_predictor import SAM2VideoPredictor predictor = SAM2VideoPredictor.from_pretrained("ipsilondev/sam2.1-hiera-tiny-ONNX") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): state = predictor.init_state(<your_video>) # add new prompts and instantly get the output on the same frame frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>) # propagate the prompts to get masklets throughout the video for frame_idx, object_ids, masks in predictor.propagate_in_video(state): ... - Notebooks
- Google Colab
- Kaggle
sam2.1-hiera-tiny-ONNX (FP16)
FP16 ONNX models for Meta SAM 2.1 Hiera Tiny at the repository root, for mobile ONNX Runtime inference (used by ipsilondev apps: image encoder once per photo, prompt encoder + mask decoder once per tap).
Provenance and licence
- Upstream model:
facebook/sam2.1-hiera-tiny, Meta Platforms, Inc., Apache-2.0. - ONNX conversion:
onnx-community/sam2.1-hiera-tiny-ONNX, Apache-2.0 (this repository'sbase_model). - This repository: the two FP16 graphs of that conversion as single self-contained ONNX files (no external
onnx_data), unchanged weights. The licence of this repository is therefore Apache-2.0; keep the upstream notices when you redistribute.
Files
| file | bytes | SHA-256 |
|---|---|---|
vision_encoder_fp16.onnx |
67,313,499 | f4ca896cf99816ad0cb7062e9ebef44a211e6f0d0a0656a7eb13349204cb6caa |
prompt_encoder_mask_decoder_fp16.onnx |
10,678,872 | 3b63240a6cb10fa62485b9a672093974dc881268c90b700687bf9c07aab33392 |
preprocessor_config.json, processor_config.json and config.json come from the upstream conversion.
Tensor names (read with the onnx Python package, opset 18)
vision_encoder_fp16.onnx (image encoder)
Inputs
| name | dtype | shape |
|---|---|---|
pixel_values |
float32 | [batch_size, 3, 1024, 1024] |
Outputs
| name | dtype | shape |
|---|---|---|
image_embeddings.0 |
float32 | [batch_size, 32, 256, 256] |
image_embeddings.1 |
float32 | [batch_size, 64, 128, 128] |
image_embeddings.2 |
float32 | [batch_size, ?, ?, ?] |
prompt_encoder_mask_decoder_fp16.onnx (prompt encoder + mask decoder)
Inputs
| name | dtype | shape |
|---|---|---|
input_points |
float32 | [batch_size, 1, num_points_per_image, 2] |
input_labels |
int64 | [batch_size, 1, num_points_per_image] |
input_boxes |
float32 | [batch_size, num_boxes_per_image, 4] |
image_embeddings.0 |
float32 | [batch_size, 32, 256, 256] |
image_embeddings.1 |
float32 | [batch_size, 64, 128, 128] |
image_embeddings.2 |
float32 | [batch_size, 256, 64, 64] |
Outputs
| name | dtype | shape |
|---|---|---|
iou_scores |
float32 | [batch_size, num_boxes_or_points, 3] |
pred_masks |
float32 | [batch_size, num_boxes_or_points, num_masks, ?, ?] |
object_score_logits |
float32 | [batch_size, num_boxes_or_points, 1] |
Usage notes
- Pre-processing: resize the RGB image to 1024x1024 (stretched, no padding), scale to 0..1, normalise with
mean
[0.485, 0.456, 0.406]and std[0.229, 0.224, 0.225], layout NCHW. - Prompts are in the 1024x1024 frame;
input_labels1 = foreground, 0 = background;input_boxesmay be empty ([1, 0, 4]). - The decoder returns three candidate masks (
pred_masks[1, 1, 3, 256, 256]logits, positive = inside) with predicted IoU scores (iou_scores[1, 1, 3]) andobject_score_logits. - Post-processing (sigmoid, upsampling, edge refinement) is up to the application.
- Downloads last month
- 7
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support
Model tree for ipsilondev/sam2.1-hiera-tiny-ONNX
Base model
onnx-community/sam2.1-hiera-tiny-ONNX