MaskRCNN / README.md

v0.59.0

6fcd54f verified 13 days ago

6.67 kB

	---
	library_name: pytorch
	license: other
	tags:
	- android
	pipeline_tag: image-segmentation

	---

	![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/maskrcnn/web-assets/model_demo.png)

	# MaskRCNN: Optimized for Qualcomm Devices

	Mask R-CNN is a machine learning model that extends Faster R-CNN to perform instance segmentation by detecting objects in an image while simultaneously generating a high-quality segmentation mask for each instance. It adds a branch for predicting segmentation masks in parallel with the existing branch for bounding box recognition.

	This is based on the implementation of MaskRCNN found [here](https://github.com/pytorch/vision).
	This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.59.0/src/qai_hub_models/models/maskrcnn) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).

	Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.

	## Getting Started
	There are two ways to deploy this model on your device:

	### Option 1: Download Pre-Exported Models

	Below are pre-exported model assets ready for deployment.

	\| Runtime \| Precision \| Chipset \| SDK Versions \| Download \|
	\|---\|---\|---\|---\|---\|
	\| QNN_DLC \| float \| Universal \| QAIRT 2.45 \| [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/maskrcnn/releases/v0.59.0/maskrcnn-qnn_dlc-float.zip)

	For more device-specific assets and performance metrics, visit [MaskRCNN on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/maskrcnn).


	### Option 2: Export with Custom Configurations

	Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.59.0/src/qai_hub_models/models/maskrcnn) Python library to compile and export the model with your own:
	- Custom weights (e.g., fine-tuned checkpoints)
	- Custom input shapes
	- Target device and runtime configurations

	This option is ideal if you need to customize the model beyond the default configuration provided here.

	See our repository for [MaskRCNN on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.59.0/src/qai_hub_models/models/maskrcnn) for usage instructions.

	## Model Details

	Model Type: Model_use_case.semantic_segmentation

	Model Stats:
	- Model checkpoint: Mask R-CNN ResNet-50 FPN V2
	- Input resolution: 800x800
	- Number of output classes: 91
	- Number of parameters: 46.4M
	- Model size (float): 177 MB

	## Performance Summary
	\| Model \| Runtime \| Precision \| Chipset \| Inference Time (ms) \| Peak Memory Range (MB) \| Primary Compute Unit
	\|---\|---\|---\|---\|---\|---\|---
	\| proposal_generator \| QNN_DLC \| float \| Snapdragon® X2 Elite \| 42.554 ms \| 7 - 7 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Snapdragon® X Elite \| 91.369 ms \| 7 - 7 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Snapdragon® 8 Gen 3 Mobile \| 70.212 ms \| 7 - 2326 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Snapdragon® 8 Gen 1 Mobile \| 151.831 ms \| 7 - 2736 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® Dragonwing™ QCS8275 \| 354.445 ms \| 2 - 1759 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® Dragonwing™ QCS8550 (Proxy) \| 94.473 ms \| 7 - 24 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® SA8775P \| 122.283 ms \| 2 - 1759 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® SA8650P \| 122.283 ms \| 2 - 1759 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® SA8255P \| 122.283 ms \| 2 - 1759 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® QCS8450 \| 151.831 ms \| 7 - 2736 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® Dragonwing™ IQ-9075 \| 122.578 ms \| 7 - 71 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® Dragonwing™ IQ-X7181 \| 91.369 ms \| 7 - 7 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® Dragonwing™ Q-8750 \| 53.816 ms \| 12 - 1523 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® SA7255P \| 354.445 ms \| 2 - 1759 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® SA8295P \| 125.987 ms \| 0 - 1430 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Snapdragon® 8 Elite Mobile \| 53.816 ms \| 12 - 1523 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Snapdragon® 8 Elite Gen 5 Mobile \| 42.27 ms \| 7 - 1789 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Snapdragon® X2 Elite \| 98.312 ms \| 52 - 52 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Snapdragon® X Elite \| 257.611 ms \| 52 - 52 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Snapdragon® 8 Gen 3 Mobile \| 177.662 ms \| 49 - 897 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Snapdragon® 8 Gen 1 Mobile \| 300.661 ms \| 0 - 918 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® Dragonwing™ QCS8275 \| 573.44 ms \| 30 - 725 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® Dragonwing™ QCS8550 (Proxy) \| 243.427 ms \| 52 - 679 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® SA8775P \| 270.285 ms \| 49 - 925 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® SA8650P \| 270.285 ms \| 49 - 925 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® SA8255P \| 270.285 ms \| 49 - 925 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® QCS8450 \| 300.661 ms \| 0 - 918 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® Dragonwing™ IQ-9075 \| 279.963 ms \| 52 - 106 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® Dragonwing™ IQ-X7181 \| 257.611 ms \| 52 - 52 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® Dragonwing™ Q-8750 \| 125.217 ms \| 34 - 726 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® SA7255P \| 573.44 ms \| 30 - 725 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® SA8295P \| 306.965 ms \| 49 - 847 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Snapdragon® 8 Elite Mobile \| 125.217 ms \| 34 - 726 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Snapdragon® 8 Elite Gen 5 Mobile \| 92.646 ms \| 13 - 719 MB \| NPU

	## License
	* The license for the original implementation of MaskRCNN can be found
	[here](https://github.com/pytorch/vision/blob/main/LICENSE).

	## References
	* [Mask R-CNN](https://arxiv.org/abs/1703.06870)
	* [Source Model Implementation](https://github.com/pytorch/vision)

	## Community
	* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
	* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).

	---
	library_name: pytorch
	license: other
	tags:
	- android
	pipeline_tag: image-segmentation

	---

	![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/maskrcnn/web-assets/model_demo.png)

	# MaskRCNN: Optimized for Qualcomm Devices

	Mask R-CNN is a machine learning model that extends Faster R-CNN to perform instance segmentation by detecting objects in an image while simultaneously generating a high-quality segmentation mask for each instance. It adds a branch for predicting segmentation masks in parallel with the existing branch for bounding box recognition.

	This is based on the implementation of MaskRCNN found [here](https://github.com/pytorch/vision).
	This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.59.0/src/qai_hub_models/models/maskrcnn) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).

	Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.

	## Getting Started
	There are two ways to deploy this model on your device:

	### Option 1: Download Pre-Exported Models

	Below are pre-exported model assets ready for deployment.

	\| Runtime \| Precision \| Chipset \| SDK Versions \| Download \|
	\|---\|---\|---\|---\|---\|
	\| QNN_DLC \| float \| Universal \| QAIRT 2.45 \| [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/maskrcnn/releases/v0.59.0/maskrcnn-qnn_dlc-float.zip)

	For more device-specific assets and performance metrics, visit [MaskRCNN on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/maskrcnn).


	### Option 2: Export with Custom Configurations

	Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.59.0/src/qai_hub_models/models/maskrcnn) Python library to compile and export the model with your own:
	- Custom weights (e.g., fine-tuned checkpoints)
	- Custom input shapes
	- Target device and runtime configurations

	This option is ideal if you need to customize the model beyond the default configuration provided here.

	See our repository for [MaskRCNN on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.59.0/src/qai_hub_models/models/maskrcnn) for usage instructions.

	## Model Details

	Model Type: Model_use_case.semantic_segmentation

	Model Stats:
	- Model checkpoint: Mask R-CNN ResNet-50 FPN V2
	- Input resolution: 800x800
	- Number of output classes: 91
	- Number of parameters: 46.4M
	- Model size (float): 177 MB

	## Performance Summary
	\| Model \| Runtime \| Precision \| Chipset \| Inference Time (ms) \| Peak Memory Range (MB) \| Primary Compute Unit
	\|---\|---\|---\|---\|---\|---\|---
	\| proposal_generator \| QNN_DLC \| float \| Snapdragon® X2 Elite \| 42.554 ms \| 7 - 7 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Snapdragon® X Elite \| 91.369 ms \| 7 - 7 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Snapdragon® 8 Gen 3 Mobile \| 70.212 ms \| 7 - 2326 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Snapdragon® 8 Gen 1 Mobile \| 151.831 ms \| 7 - 2736 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® Dragonwing™ QCS8275 \| 354.445 ms \| 2 - 1759 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® Dragonwing™ QCS8550 (Proxy) \| 94.473 ms \| 7 - 24 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® SA8775P \| 122.283 ms \| 2 - 1759 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® SA8650P \| 122.283 ms \| 2 - 1759 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® SA8255P \| 122.283 ms \| 2 - 1759 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® QCS8450 \| 151.831 ms \| 7 - 2736 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® Dragonwing™ IQ-9075 \| 122.578 ms \| 7 - 71 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® Dragonwing™ IQ-X7181 \| 91.369 ms \| 7 - 7 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® Dragonwing™ Q-8750 \| 53.816 ms \| 12 - 1523 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® SA7255P \| 354.445 ms \| 2 - 1759 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Qualcomm® SA8295P \| 125.987 ms \| 0 - 1430 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Snapdragon® 8 Elite Mobile \| 53.816 ms \| 12 - 1523 MB \| NPU
	\| proposal_generator \| QNN_DLC \| float \| Snapdragon® 8 Elite Gen 5 Mobile \| 42.27 ms \| 7 - 1789 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Snapdragon® X2 Elite \| 98.312 ms \| 52 - 52 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Snapdragon® X Elite \| 257.611 ms \| 52 - 52 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Snapdragon® 8 Gen 3 Mobile \| 177.662 ms \| 49 - 897 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Snapdragon® 8 Gen 1 Mobile \| 300.661 ms \| 0 - 918 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® Dragonwing™ QCS8275 \| 573.44 ms \| 30 - 725 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® Dragonwing™ QCS8550 (Proxy) \| 243.427 ms \| 52 - 679 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® SA8775P \| 270.285 ms \| 49 - 925 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® SA8650P \| 270.285 ms \| 49 - 925 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® SA8255P \| 270.285 ms \| 49 - 925 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® QCS8450 \| 300.661 ms \| 0 - 918 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® Dragonwing™ IQ-9075 \| 279.963 ms \| 52 - 106 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® Dragonwing™ IQ-X7181 \| 257.611 ms \| 52 - 52 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® Dragonwing™ Q-8750 \| 125.217 ms \| 34 - 726 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® SA7255P \| 573.44 ms \| 30 - 725 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Qualcomm® SA8295P \| 306.965 ms \| 49 - 847 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Snapdragon® 8 Elite Mobile \| 125.217 ms \| 34 - 726 MB \| NPU
	\| roi_head \| QNN_DLC \| float \| Snapdragon® 8 Elite Gen 5 Mobile \| 92.646 ms \| 13 - 719 MB \| NPU

	## License
	* The license for the original implementation of MaskRCNN can be found
	[here](https://github.com/pytorch/vision/blob/main/LICENSE).

	## References
	* [Mask R-CNN](https://arxiv.org/abs/1703.06870)
	* [Source Model Implementation](https://github.com/pytorch/vision)

	## Community
	* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
	* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).