Instructions to use CardAnvilModels/sam2.1-hiera-tiny-ONNX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers.js
How to use CardAnvilModels/sam2.1-hiera-tiny-ONNX with Transformers.js:
// npm i @huggingface/transformers import { pipeline } from '@huggingface/transformers'; // Allocate pipeline const pipe = await pipeline('mask-generation', 'CardAnvilModels/sam2.1-hiera-tiny-ONNX'); - sam2
How to use CardAnvilModels/sam2.1-hiera-tiny-ONNX with sam2:
# Use SAM2 with images import torch from sam2.sam2_image_predictor import SAM2ImagePredictor predictor = SAM2ImagePredictor.from_pretrained("CardAnvilModels/sam2.1-hiera-tiny-ONNX") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): predictor.set_image(<your_image>) masks, _, _ = predictor.predict(<input_prompts>)# Use SAM2 with videos import torch from sam2.sam2_video_predictor import SAM2VideoPredictor predictor = SAM2VideoPredictor.from_pretrained("CardAnvilModels/sam2.1-hiera-tiny-ONNX") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): state = predictor.init_state(<your_video>) # add new prompts and instantly get the output on the same frame frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>) # propagate the prompts to get masklets throughout the video for frame_idx, object_ids, masks in predictor.propagate_in_video(state): ... - Notebooks
- Google Colab
- Kaggle
|
Download README.md from CardAnvilModels/sam2.1-hiera-tiny-ONNX: direct link, hf CLI and curl.
- Browser
- Download file 1.82 kB
-
https://huggingface.co/CardAnvilModels/sam2.1-hiera-tiny-ONNX/resolve/main/README.md
- Command line
-
hf download hf://CardAnvilModels/sam2.1-hiera-tiny-ONNX/README.md
-
curl -L -o README.md https://huggingface.co/CardAnvilModels/sam2.1-hiera-tiny-ONNX/resolve/main/README.md
1.82 kB
| license: apache-2.0 | |
| library_name: transformers.js | |
| base_model: facebook/sam2.1-hiera-tiny | |
| pipeline_tag: mask-generation | |
| tags: | |
| - onnx | |
| - sam2 | |
| - segmentation | |
| - transformers.js | |
| - onnxruntime-web | |
| # SAM 2.1 Hiera Tiny (ONNX, fp16) | |
| Segment Anything 2.1 (Hiera Tiny) as ONNX for [Transformers.js](https://github.com/huggingface/transformers.js), at every precision the source has. [Card Anvil](https://github.com/Card-Anvil) uses it in the browser to select the object under a click or inside a box. | |
| These are unchanged copies of the files in [`onnx-community/sam2.1-hiera-tiny-ONNX`](https://huggingface.co/onnx-community/sam2.1-hiera-tiny-ONNX) at commit [`814a066`](https://huggingface.co/onnx-community/sam2.1-hiera-tiny-ONNX/tree/814a066640debee5a91e70aa401fb8e17e030503), at the same paths, kept here so the app loads files that can't change under it. | |
| Card Anvil loads the three config files and the **fp16** encoder and decoder (`onnx/*_fp16.onnx` and their `.onnx_data`), 78 MB. The other precisions are kept for later: | |
| | Precision | Files in `onnx/` | Encoder + decoder | | |
| | --- | --- | --- | | |
| | fp32 | `vision_encoder`, `prompt_encoder_mask_decoder` | 155 MB | | |
| | fp16 (loaded) | `*_fp16` | 78 MB | | |
| | int8 / uint8 / quantized | `*_int8`, `*_uint8`, `*_quantized` | 61 MB | | |
| | q4 | `*_q4` | 51 MB | | |
| | bnb4 | `*_bnb4` | 49 MB | | |
| | q4f16 | `*_q4f16` | 33 MB | | |
| ## Credits and licence | |
| - Model: [SAM 2](https://github.com/facebookresearch/sam2) by Meta AI ([`facebook/sam2.1-hiera-tiny`](https://huggingface.co/facebook/sam2.1-hiera-tiny)), from *SAM 2: Segment Anything in Images and Videos* (2024). | |
| - ONNX conversion: [ONNX Community](https://huggingface.co/onnx-community). | |
| Licensed under the Apache License 2.0, as the original model is. See [`LICENSE`](LICENSE). | |