Instructions to use inference4j/sam2.1-hiera-tiny with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sam2
How to use inference4j/sam2.1-hiera-tiny with sam2:
# Use SAM2 with images import torch from sam2.sam2_image_predictor import SAM2ImagePredictor predictor = SAM2ImagePredictor.from_pretrained("inference4j/sam2.1-hiera-tiny") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): predictor.set_image(<your_image>) masks, _, _ = predictor.predict(<input_prompts>)# Use SAM2 with videos import torch from sam2.sam2_video_predictor import SAM2VideoPredictor predictor = SAM2VideoPredictor.from_pretrained("inference4j/sam2.1-hiera-tiny") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): state = predictor.init_state(<your_video>) # add new prompts and instantly get the output on the same frame frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>) # propagate the prompts to get masklets throughout the video for frame_idx, object_ids, masks in predictor.propagate_in_video(state): ... - Notebooks
- Google Colab
- Kaggle
|
Download README.md from inference4j/sam2.1-hiera-tiny: direct link, hf CLI and curl.
- Browser
- Download file 3.28 kB
-
https://huggingface.co/inference4j/sam2.1-hiera-tiny/resolve/main/README.md
- Command line
-
hf download hf://inference4j/sam2.1-hiera-tiny/README.md
-
curl -L -o README.md https://huggingface.co/inference4j/sam2.1-hiera-tiny/resolve/main/README.md
3.28 kB
| library_name: onnx | |
| tags: | |
| - sam2 | |
| - image-segmentation | |
| - promptable-segmentation | |
| - vision | |
| - onnx | |
| - inference4j | |
| license: apache-2.0 | |
| pipeline_tag: mask-generation | |
| # SAM 2.1 Hiera Tiny — ONNX | |
| FP32 ONNX export of [facebook/sam2.1-hiera-tiny](https://huggingface.co/facebook/sam2.1-hiera-tiny), split into an image encoder and a prompt encoder/mask decoder for point- and box-prompted image segmentation. The ONNX graphs are mirrored unchanged from the [onnx-community export](https://huggingface.co/onnx-community/sam2.1-hiera-tiny-ONNX). | |
| Mirrored for use with [inference4j](https://github.com/inference4j/inference4j), an inference-only AI library for Java. | |
| ## Original Source | |
| - **Repository:** [Meta (ONNX export by onnx-community)](https://huggingface.co/facebook/sam2.1-hiera-tiny) | |
| - **License:** apache-2.0 | |
| ## Usage with inference4j | |
| ```java | |
| try (Sam2Segmenter segmenter = Sam2Segmenter.builder().build()) { | |
| SegmentationMask mask = segmenter.segment( | |
| Path.of("photo.jpg"), | |
| List.of(new PromptPoint(320, 240, true))); | |
| ImageIO.write(mask.toImage(), "png", Path.of("mask.png").toFile()); | |
| } | |
| ``` | |
| ## Model Details | |
| | Property | Value | | |
| |----------|-------| | |
| | Architecture | SAM 2.1 Hiera Tiny | | |
| | Task | Promptable image segmentation | | |
| | Graphs | `vision_encoder.onnx` and `prompt_encoder_mask_decoder.onnx` (opset 18, FP32) | | |
| | External data | Each graph loads its matching `.onnx_data` sidecar from the same directory | | |
| | Image input | `pixel_values`: `[batch, 3, 1024, 1024]` float32, RGB | | |
| | Image preprocessing | Resize to 1024 x 1024 without preserving aspect ratio; rescale by 1/255; ImageNet mean/std normalization | | |
| | Prompts | Point coordinates and boxes use the 1024 x 1024 input coordinate space | | |
| | Mask output | `pred_masks`: `[batch, prompts, 3, 256, 256]` float32 low-resolution logits | | |
| | Mask selection | Choose one of three candidate masks using `iou_scores` | | |
| | Refinement | No previous-mask (`mask_input`) input; iterative mask refinement is not supported by this export | | |
| ## License | |
| The original SAM 2.1 model and weights are licensed under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). Original model by [Meta](https://huggingface.co/facebook/sam2.1-hiera-tiny); ONNX conversion by [onnx-community](https://huggingface.co/onnx-community/sam2.1-hiera-tiny-ONNX). | |
| ## Files | |
| The FP32 model remains split into two ONNX graphs and their external-data sidecars: | |
| - `vision_encoder.onnx` + `vision_encoder.onnx_data` | |
| - `prompt_encoder_mask_decoder.onnx` + `prompt_encoder_mask_decoder.onnx_data` | |
| All four files must remain in the same directory. The sidecars are deliberately not | |
| folded into the graph files. | |
| ## Prompt Encoding | |
| - Foreground point label: `1` | |
| - Background point label: `0` | |
| - Padding point label: `-10` | |
| - Box layout: `(x1, y1, x2, y2)` | |
| Coordinates are expressed in the resized 1024 x 1024 image space. See the source | |
| `processor_config.json` for processor defaults and the inference4j wrapper for conversion | |
| from original-image coordinates. | |
| ## Outputs | |
| The decoder returns three candidate low-resolution mask logits per prompt, their | |
| `iou_scores`, and `object_score_logits`. Resize the selected logits to the original image | |
| size and threshold at zero to obtain a binary mask. | |