| --- |
| license: other |
| license_name: apache-2.0-mit-and-cc-by-4.0 |
| license_link: https://huggingface.co/pch/rawmakase-models/blob/main/NOTICE |
| library_name: onnx |
| pipeline_tag: image-segmentation |
| tags: |
| - onnx |
| - sam2 |
| - detr |
| - upernet |
| - rawnind |
| - denoising |
| - image-segmentation |
| - rawmakase |
| --- |
| |
| # RAWmakase model files |
|
|
| The models [RAWmakase](https://github.com/pch/rawmakase), an open source RAW photo |
| editor, downloads when you first use **Select Subject**, **Select Sky** or **Select |
| Background** in its Masking tool, or **AI Denoise** in its Detail panel. They run on your computer with ONNX Runtime; no |
| photo is sent anywhere. The app fetches these files at a pinned commit of this |
| repository and checks each against the SHA-256 below before using it. |
|
|
| Nothing here was trained by RAWmakase. The SAM 2 and DETR files are copies of public ONNX |
| exports of Meta's Apache-2.0 models, kept here so the app's downloads do not depend on |
| third-party repositories. The UperNet file is RAWmakase's own ONNX export of an MIT-licensed |
| checkpoint. The RawNIND file is RAWmakase's ONNX conversion of Benoit Brummer's checkpoint, |
| weights unchanged, under CC BY 4.0. See [NOTICE](NOTICE) for exact provenance, [LICENSE](LICENSE) |
| for the Apache license, [upernet-convnext-base/LICENSE](upernet-convnext-base/LICENSE) for the |
| MIT one and [rawnind-utnet2-bayer/LICENSE](rawnind-utnet2-bayer/LICENSE) for CC BY 4.0. |
|
|
| | File | Bytes | SHA-256 | |
| | --- | --- | --- | |
| | `sam2.1-hiera-small/vision_encoder.onnx` | 467440 | `aacf1f7137bb6fffcf6bf166abcfabe28f57a76059254f3fb611c4a64a208119` | |
| | `sam2.1-hiera-small/vision_encoder.onnx_data` | 162476288 | `260fd1f0a34e72a3dc79a739e563b4facc0ba75504818b433a1f808e66637456` | |
| | `sam2.1-hiera-small/prompt_encoder_mask_decoder.onnx` | 213114 | `079c59b261f723ff5c6a125e69b0170a957b21c58738c28d2b0394ecd0587d7f` | |
| | `sam2.1-hiera-small/prompt_encoder_mask_decoder.onnx_data` | 20958208 | `f9e59a584ab8ced21fa812c211bc01084204db1c9e92a5ef4fb3a49972b4e864` | |
| | `detr-resnet-50-panoptic/detr-panoptic-fp16.onnx` | 86559030 | `afd9f02d864302d690356fd4bfcb2feed2397a1190bf46a7306cb430464d734a` | |
| | `upernet-convnext-base/upernet-convnext-base-sky-fp16.onnx` | 242162020 | `ed75b2ad81ad209949c9580071ed486a72a423037822a49faa4e98991a43b891` | |
| | `rawnind-utnet2-bayer/model_bayer.onnx` | 31057086 | `5111f19a66a9377002e2ff4ca87de8e3303085b33e6dc11746e42ce811bf6cc1` | |
| | `rawnind-utnet2-bayer/LICENSE` | 18657 | `9ba9550ad48438d0836ddab3da480b3b69ffa0aac7b7878b5a0039e7ab429411` | |
| | `rawnind-utnet2-bayer/NOTICE` | 777 | `809b7f8aac9b4bf7134e7543de867b031f194895b280efebe0206e1ddca868e6` | |
| | `rawnind-utnet2-bayer/MODEL_CARD.md` | 3670 | `3c2d3d55e8bb0d6b4d0b1e5e357ce2cb4aa0e5f0dd6383cd6bf46c44f755ddf6` | |
|
|
| ## Segment Anything Model 2.1, Hiera small |
|
|
| - **What it does in RAWmakase:** draws the outline of each person and animal (and of the |
| sky in earlier releases), and answers clicks when you refine a selection. |
| - **Made by:** Meta AI (Ravi et al., *SAM 2: Segment Anything in Images and Videos*, |
| 2024). Weights and code are Apache-2.0: |
| [facebookresearch/sam2](https://github.com/facebookresearch/sam2), |
| [facebook/sam2.1-hiera-small](https://huggingface.co/facebook/sam2.1-hiera-small). |
| Trained on Meta's SA-1B and SA-V datasets. |
| - **This export:** [onnx-community/sam2.1-hiera-small-ONNX](https://huggingface.co/onnx-community/sam2.1-hiera-small-ONNX) |
| at commit `a7df49d8de14b9d2e4504d1687b0d568f905fd8d` (float32). An image encoder |
| (`pixel_values` [1,3,1024,1024], ImageNet mean/std, stretched) and a prompt encoder |
| with mask decoder (points, labels, boxes and the three image embeddings in; three |
| candidate 256×256 mask logits, IoU scores and an object score out). Each graph's |
| weights are in the `.onnx_data` file beside it. |
|
|
| ## DETR ResNet-50 panoptic |
|
|
| - **What it does in RAWmakase:** finds the people and animals in the photo (and the |
| sky in earlier releases), so the right outlines are chosen. |
| - **Made by:** Facebook AI Research (Carion et al., *End-to-End Object Detection with |
| Transformers*, 2020). Weights and code are Apache-2.0: |
| [facebookresearch/detr](https://github.com/facebookresearch/detr), |
| [facebook/detr-resnet-50-panoptic](https://huggingface.co/facebook/detr-resnet-50-panoptic). |
| Trained on COCO 2017 panoptic. |
| - **This export:** [Xenova/detr-resnet-50-panoptic](https://huggingface.co/Xenova/detr-resnet-50-panoptic) |
| at commit `ea24b2d4e0bfae31f0a1299ba3fb892a2df064de`, `onnx/model_fp16.onnx` (half |
| precision weights, float32 inputs). `pixel_values` [1,3,H,W] (ImageNet mean/std; |
| RAWmakase uses a long edge of 800) and `pixel_mask` [1,64,64]; `logits` [1,100,251] |
| (COCO category ids, last class is "no object"; person 1, animals 16–25, sky 187) and |
| `pred_masks` [1,100,H/4,W/4]. ONNX Runtime's full graph optimisation fails on this |
| file; RAWmakase loads it with basic optimisation. |
|
|
| ## UperNet ConvNeXt base |
|
|
| - **What it does in RAWmakase:** finds the sky, for **Select Sky**. |
| - **Made by:** OpenMMLab's training of UperNet (Xiao et al., 2018) with ConvNeXt (Liu et al., |
| Meta AI, 2022), [openmmlab/upernet-convnext-base](https://huggingface.co/openmmlab/upernet-convnext-base), |
| whose card says MIT. Trained on [ADE20K](https://groups.csail.mit.edu/vision/datasets/ADE20K/) |
| (MIT CSAIL), whose images are offered for non-commercial research and education. |
| - **This export:** made by [export_upernet_sky.py](upernet-convnext-base/export_upernet_sky.py) |
| from that repository at commit `13a9c410ff779f25236a8d24802d2742e4694e9c`, which reproduces |
| it byte for byte with the same package versions. Half precision weights, cast to float32 as it loads; the |
| auxiliary head dropped; adaptive pooling written as the same averages so any size works. |
| `pixel_values` [1,3,H,W] (ImageNet mean/std; RAWmakase uses a long edge of 1024); `sky` |
| [1,H,W], the probability of ADE20K's sky at each pixel. |
|
|
| ## RawNIND UtNet2 (Bayer) |
|
|
| - **What it does in RAWmakase:** **AI Denoise** denoises and demosaics a RAW photo's mosaic |
| in one pass, on your computer. |
| - **Made by:** Benoit Brummer (UCLouvain), [rawnind_jddc](https://github.com/trougnouf/rawnind_jddc) |
| at commit `e80fb7b14dc440bba38c01f312fd3cc02deb566c`; Brummer and De Vleeschouwer, |
| [arXiv:2501.08924](https://arxiv.org/abs/2501.08924). Trained on the RawNIND dataset |
| ([doi:10.14428/DVN/DEQCIM](https://doi.org/10.14428/DVN/DEQCIM)). The author licenses the |
| weights under GPL-3.0 or CC BY 4.0; RAWmakase takes and redistributes them under |
| **CC BY 4.0** ([LICENSE](rawnind-utnet2-bayer/LICENSE), credit in |
| [NOTICE](rawnind-utnet2-bayer/NOTICE)). RAWmakase is not endorsed by the authors. |
| - **This export:** RAWmakase's conversion of checkpoint |
| `DenoiserTrainingBayerToProfiledRGB_4ch_2024-02-21-bayer_ms-ssim_mgout_notrans_valeither_-4` |
| (SHA-256 `c1f7de3de24cb12f4803ee65e11cfb467e9a5b80c3cb7126746ef026106c15e0`) to ONNX |
| with darktable-ai `76513a1e379b4265007bae768825b9a581944ff8`; the weights are unchanged. |
| `input` [1,4,512,512] (packed RGGB planes, 0–1, not white balanced) and `output` |
| [1,3,1024,1024] (camera RGB at an arbitrary scale). Details and measurements in its |
| [model card](rawnind-utnet2-bayer/MODEL_CARD.md). |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{ravi2024sam2, |
| title={SAM 2: Segment Anything in Images and Videos}, |
| author={Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and Hu, Ronghang and Ryali, Chaitanya and Ma, Tengyu and Khedr, Haitham and R{\"a}dle, Roman and Rolland, Chloe and Gustafson, Laura and Mintun, Eric and Pan, Junting and Alwala, Kalyan Vasudev and Carion, Nicolas and Wu, Chao-Yuan and Girshick, Ross and Doll{\'a}r, Piotr and Feichtenhofer, Christoph}, |
| journal={arXiv preprint arXiv:2408.00714}, |
| year={2024} |
| } |
| @inproceedings{xiao2018upernet, |
| title={Unified Perceptual Parsing for Scene Understanding}, |
| author={Xiao, Tete and Liu, Yingcheng and Zhou, Bolei and Jiang, Yuning and Sun, Jian}, |
| booktitle={ECCV}, |
| year={2018} |
| } |
| @inproceedings{liu2022convnext, |
| title={A ConvNet for the 2020s}, |
| author={Liu, Zhuang and Mao, Hanzi and Wu, Chao-Yuan and Feichtenhofer, Christoph and Darrell, Trevor and Xie, Saining}, |
| booktitle={CVPR}, |
| year={2022} |
| } |
| @article{brummer2025rawnind, |
| title={Learning Joint Denoising, Demosaicing, and Compression from the Raw Natural Image Noise Dataset}, |
| author={Brummer, Benoit and De Vleeschouwer, Christophe}, |
| journal={arXiv preprint arXiv:2501.08924}, |
| year={2025} |
| } |
| @inproceedings{carion2020detr, |
| title={End-to-End Object Detection with Transformers}, |
| author={Carion, Nicolas and Massa, Francisco and Synnaeve, Gabriel and Usunier, Nicolas and Kirillov, Alexander and Zagoruyko, Sergey}, |
| booktitle={ECCV}, |
| year={2020} |
| } |
| ``` |
|
|
