Instructions to use DotoPototo/paint-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sam2
How to use DotoPototo/paint-models with sam2:
# Use SAM2 with images import torch from sam2.sam2_image_predictor import SAM2ImagePredictor predictor = SAM2ImagePredictor.from_pretrained("DotoPototo/paint-models") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): predictor.set_image(<your_image>) masks, _, _ = predictor.predict(<input_prompts>)# Use SAM2 with videos import torch from sam2.sam2_video_predictor import SAM2VideoPredictor predictor = SAM2VideoPredictor.from_pretrained("DotoPototo/paint-models") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): state = predictor.init_state(<your_video>) # add new prompts and instantly get the output on the same frame frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>) # propagate the prompts to get masklets throughout the video for frame_idx, object_ids, masks in predictor.propagate_in_video(state): ... - Notebooks
- Google Colab
- Kaggle
doto.gg Paint selection models
Tap-to-select models for the doto.gg Paint wall colour visualiser, which runs them in the browser with onnxruntime-web (WebGPU, with a WASM fallback). Every file is full precision (fp32): fp16 and int8 versions were tried and dropped because they made selections worse.
The app pins a commit of this repo, so a file here never changes under it: a new version means a new commit.
| Folder | Model | Encoder | Decoder | Used for |
|---|---|---|---|---|
mobilesam/ |
MobileSAM | 28.2 MB | 16.5 MB | Phones, tablets and slower computers |
sam2.1-small/ |
SAM 2.1 Hiera Small | 138.0 MB | 16.5 MB | Desktops with a capable GPU |
mobilesam
Original MobileSAM weights: Apache-2.0 (LICENSE). ONNX exports: MIT (LICENSE.MIT).
encoder.onnx: https://huggingface.co/comicbox/mobilesam-onnx/resolve/main/mobile_sam_image_encoder.onnx, unchanged. SHA-256580f5fb648ea1062c0aabc26217aed56921985f03f0cbbd852bba81d760cc749. Inputinput_image[H, W, 3] float RGB 0–255 with the longest side 1024; normalisation and padding are inside.decoder.onnx: https://huggingface.co/Acly/MobileSAM/resolve/main/sam_mask_decoder_multi.onnx, unchanged. SHA-2568976b90a87ba50a6a72217a5ff994f7d25ce16f2229fcc1ed259e1294c622ffe. SAM's ONNX decoder interface, returning all four mask candidates.
sam2.1-small
SAM 2.1 (Meta): Apache-2.0 (LICENSE). ONNX export by Viet-Anh Nguyen with
samexporter (MIT), from
https://huggingface.co/vietanhdev/segment-anything-2.1-onnx-models/resolve/main/sam2.1_hiera_small_20260221.zip (Apache-2.0).
encoder.onnx: SHA-2564b71c31308ff1e49f812c10fef910f0a4011991cf4935c0a53284b79fcd1b3c3. Inputimage[1, 3, 1024, 1024], the photo stretched to 1024² and ImageNet-normalised. Changed from the export: its twoIfnodes, whose branches declare different output shapes (which onnxruntime-web rejects), are replaced by the branch a 1024² input always takes. Outputs are bit-identical to the original.decoder.onnx: unchanged from the export. SHA-256d158eb26a43d39eed23ef677d6826d17eb9eda951921f9a252fa5d36d871b57d. Returns three 256² mask candidates on the stretched 1024² frame and their predicted IoU.