pch/rawmakase-models

RAWmakase model files

The models RAWmakase, an open source RAW photo editor, downloads when you first use Select Subject, Select Sky or Select Background in its Masking tool. They run on your computer with ONNX Runtime; no photo is sent anywhere. The app fetches these files at a pinned commit of this repository and checks each against the SHA-256 below before using it.

Nothing here was trained by RAWmakase. The SAM 2 and DETR files are copies of public ONNX exports of Meta's Apache-2.0 models, kept here so the app's downloads do not depend on third-party repositories. The UperNet file is RAWmakase's own ONNX export of an MIT-licensed checkpoint. See NOTICE for exact provenance, LICENSE for the Apache license and upernet-convnext-base/LICENSE for the MIT one.

File	Bytes	SHA-256
sam2.1-hiera-small/vision_encoder.onnx	467440	aacf1f7137bb6fffcf6bf166abcfabe28f57a76059254f3fb611c4a64a208119
sam2.1-hiera-small/vision_encoder.onnx_data	162476288	260fd1f0a34e72a3dc79a739e563b4facc0ba75504818b433a1f808e66637456
sam2.1-hiera-small/prompt_encoder_mask_decoder.onnx	213114	079c59b261f723ff5c6a125e69b0170a957b21c58738c28d2b0394ecd0587d7f
sam2.1-hiera-small/prompt_encoder_mask_decoder.onnx_data	20958208	f9e59a584ab8ced21fa812c211bc01084204db1c9e92a5ef4fb3a49972b4e864
detr-resnet-50-panoptic/detr-panoptic-fp16.onnx	86559030	afd9f02d864302d690356fd4bfcb2feed2397a1190bf46a7306cb430464d734a
upernet-convnext-base/upernet-convnext-base-sky-fp16.onnx	242162020	ed75b2ad81ad209949c9580071ed486a72a423037822a49faa4e98991a43b891

Segment Anything Model 2.1, Hiera small

• What it does in RAWmakase: draws the outline of each person and animal (and of the sky in earlier releases), and answers clicks when you refine a selection.
• Made by: Meta AI (Ravi et al., SAM 2: Segment Anything in Images and Videos, 2024). Weights and code are Apache-2.0: facebookresearch/sam2, facebook/sam2.1-hiera-small. Trained on Meta's SA-1B and SA-V datasets.
• This export: onnx-community/sam2.1-hiera-small-ONNX at commit a7df49d8de14b9d2e4504d1687b0d568f905fd8d (float32). An image encoder (pixel_values [1,3,1024,1024], ImageNet mean/std, stretched) and a prompt encoder with mask decoder (points, labels, boxes and the three image embeddings in; three candidate 256×256 mask logits, IoU scores and an object score out). Each graph's weights are in the .onnx_data file beside it.

DETR ResNet-50 panoptic

• What it does in RAWmakase: finds the people and animals in the photo (and the sky in earlier releases), so the right outlines are chosen.
• Made by: Facebook AI Research (Carion et al., End-to-End Object Detection with Transformers, 2020). Weights and code are Apache-2.0: facebookresearch/detr, facebook/detr-resnet-50-panoptic. Trained on COCO 2017 panoptic.
• This export: Xenova/detr-resnet-50-panoptic at commit ea24b2d4e0bfae31f0a1299ba3fb892a2df064de, onnx/model_fp16.onnx (half precision weights, float32 inputs). pixel_values [1,3,H,W] (ImageNet mean/std; RAWmakase uses a long edge of 800) and pixel_mask [1,64,64]; logits [1,100,251] (COCO category ids, last class is "no object"; person 1, animals 16–25, sky 187) and pred_masks [1,100,H/4,W/4]. ONNX Runtime's full graph optimisation fails on this file; RAWmakase loads it with basic optimisation.

UperNet ConvNeXt base

• What it does in RAWmakase: finds the sky, for Select Sky.
• Made by: OpenMMLab's training of UperNet (Xiao et al., 2018) with ConvNeXt (Liu et al., Meta AI, 2022), openmmlab/upernet-convnext-base, whose card says MIT. Trained on ADE20K (MIT CSAIL), whose images are offered for non-commercial research and education.
• This export: made by export_upernet_sky.py from that repository at commit 13a9c410ff779f25236a8d24802d2742e4694e9c, which reproduces it byte for byte with the same package versions. Half precision weights, cast to float32 as it loads; the auxiliary head dropped; adaptive pooling written as the same averages so any size works. pixel_values [1,3,H,W] (ImageNet mean/std; RAWmakase uses a long edge of 1024); sky [1,H,W], the probability of ADE20K's sky at each pixel.

Citation

@article{ravi2024sam2,
  title={SAM 2: Segment Anything in Images and Videos},
  author={Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and Hu, Ronghang and Ryali, Chaitanya and Ma, Tengyu and Khedr, Haitham and R{\"a}dle, Roman and Rolland, Chloe and Gustafson, Laura and Mintun, Eric and Pan, Junting and Alwala, Kalyan Vasudev and Carion, Nicolas and Wu, Chao-Yuan and Girshick, Ross and Doll{\'a}r, Piotr and Feichtenhofer, Christoph},
  journal={arXiv preprint arXiv:2408.00714},
  year={2024}
}
@inproceedings{xiao2018upernet,
  title={Unified Perceptual Parsing for Scene Understanding},
  author={Xiao, Tete and Liu, Yingcheng and Zhou, Bolei and Jiang, Yuning and Sun, Jian},
  booktitle={ECCV},
  year={2018}
}
@inproceedings{liu2022convnext,
  title={A ConvNet for the 2020s},
  author={Liu, Zhuang and Mao, Hanzi and Wu, Chao-Yuan and Feichtenhofer, Christoph and Darrell, Trevor and Xie, Saining},
  booktitle={CVPR},
  year={2022}
}
@inproceedings{carion2020detr,
  title={End-to-End Object Detection with Transformers},
  author={Carion, Nicolas and Massa, Francisco and Synnaeve, Gabriel and Usunier, Nicolas and Kirillov, Alexander and Zagoruyko, Sergey},
  booktitle={ECCV},
  year={2020}
}