Mask Generation
ONNX
sam2
segment-anything
sam-med3d
microscopy
medical-imaging
interactive-segmentation
Instructions to use voxide/voxide-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sam2
How to use voxide/voxide-models with sam2:
# Use SAM2 with images import torch from sam2.sam2_image_predictor import SAM2ImagePredictor predictor = SAM2ImagePredictor.from_pretrained("voxide/voxide-models") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): predictor.set_image(<your_image>) masks, _, _ = predictor.predict(<input_prompts>)# Use SAM2 with videos import torch from sam2.sam2_video_predictor import SAM2VideoPredictor predictor = SAM2VideoPredictor.from_pretrained("voxide/voxide-models") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): state = predictor.init_state(<your_video>) # add new prompts and instantly get the output on the same frame frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>) # propagate the prompts to get masklets throughout the video for frame_idx, object_ids, masks in predictor.propagate_in_video(state): ... - Notebooks
- Google Colab
- Kaggle
File size: 3,068 Bytes
d59dcf8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 | ---
license: apache-2.0
library_name: onnx
pipeline_tag: mask-generation
tags:
- onnx
- segment-anything
- sam2
- sam-med3d
- microscopy
- medical-imaging
- interactive-segmentation
base_model:
- facebook/sam2.1-hiera-tiny
- facebook/sam2.1-hiera-small
- facebook/sam2.1-hiera-base-plus
- facebook/sam2.1-hiera-large
---
# Voxide segmentation models (ONNX)
ONNX exports of **SAM 2.1** and **SAM-Med3D (turbo)** used by
[Voxide](https://huggingface.co/voxide), a GPU volume viewer for microscopy, for interactive
segmentation. Each model is an encoder/decoder pair run on the CPU with ONNX Runtime.
Voxide ships the SAM 2.1 Tiny pair. It downloads the others on request from this repository at a
pinned revision and checks every file against the SHA-256 in [`SHA256SUMS`](SHA256SUMS), so a file
that does not match is never loaded.
## Files
| Model | Encoder | Decoder | Use in Voxide |
|---|---|---|---|
| SAM 2.1 Hiera Tiny | `sam2.1_hiera_tiny_encoder.onnx` (110 MB) | `sam2.1_hiera_tiny_decoder.onnx` (17 MB) | 2D slices (default, fastest) |
| SAM 2.1 Hiera Small | `sam2.1_hiera_small_encoder.onnx` (139 MB) | `sam2.1_hiera_small_decoder.onnx` (17 MB) | 2D slices |
| SAM 2.1 Hiera Base+ | `sam2.1_hiera_base_plus_encoder.onnx` (278 MB) | `sam2.1_hiera_base_plus_decoder.onnx` (17 MB) | 2D slices |
| SAM 2.1 Hiera Large | `sam2.1_hiera_large_encoder.onnx` (853 MB) | `sam2.1_hiera_large_decoder.onnx` (17 MB) | 2D slices (best masks, slowest) |
| SAM-Med3D turbo | `sammed3d_turbo_encoder.onnx` (373 MB) | `sammed3d_turbo_decoder.onnx` (31 MB) | 3D volumes |
## Using them without the in-app download
Download the files you need and either use **Models → Import model…** in Voxide or copy them
into Voxide's model folder. The file names must stay as they are: Voxide finds each model by its
`<name>_encoder.onnx` / `<name>_decoder.onnx` pair.
```bash
pip install -U huggingface_hub
hf download voxide/voxide-models --include "sam2.1_hiera_small_*" --local-dir voxide-models
```
## How they were made
Unmodified inference exports of the upstream checkpoints, produced with PyTorch 2.9.0 (ONNX opset
18, IR version 8). No fine-tuning, quantization or other change to the weights.
## License and attribution
Both upstream model families are released under the **Apache License 2.0** (see [`LICENSE`](LICENSE)),
and so are these exports.
- **SAM 2.1** © Meta Platforms, Inc. — <https://github.com/facebookresearch/sam2>
- **SAM-Med3D** © the SAM-Med3D authors — <https://github.com/uni-medical/SAM-Med3D>
If you use these models in published work, please cite the original papers:
```bibtex
@article{ravi2024sam2,
title = {SAM 2: Segment Anything in Images and Videos},
author = {Ravi, Nikhila and others},
journal = {arXiv preprint arXiv:2408.00714},
year = {2024}
}
@article{wang2023sammed3d,
title = {SAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical Images},
author = {Wang, Haoyu and others},
journal = {arXiv preprint arXiv:2310.15161},
year = {2023}
}
```
|