nxp/ssdlite-mobilenetv2-imx

SSDLite MobileNetV2

Introduction

SSDLite MobileNetV2 is a lightweight real-time object detection model designed to run efficiently on mobile and embedded devices. It combines the SSD (Single Shot MultiBox Detector) architecture with a MobileNetV2 backbone, replacing the standard SSD convolutions with depthwise-separable convolutions (hence "SSDLite") to reduce computation.

The model regresses bounding boxes (4 coordinates) and a confidence score for each of 1917 prior boxes. The bounding box decoding and non-maximum suppression (NMS) steps are NOT included in the model. See example.py for a complete post-processing implementation.

Model Description

This repository contains pre-compiled model files optimized for NXP i.MX 8M Plus, i.MX 93, i.MX 95, and i.MX 952 processors.

• Base Model: SSDLite MobileNetV2 COCO (2018-05-09)
• Original Model Authors: Google / TensorFlow Object Detection API team
• Original License: Apache-2.0
• Modified by: NXP

Modifications

This model is a derivative work with the following changes from the original:

• Pruning: Pre- and post-processing subgraphs (image decoder, box decoding, NMS) are removed; only the backbone and SSD prediction heads are retained.
• Conversion: Converted from TensorFlow 1 SavedModel to TensorFlow Lite with full int8 quantization.
• Compilation: Compiled for i.MX 93 with Ethos-U65 NPU using the Vela tool.
• Compilation: Compiled for i.MX 95 and i.MX 952 with eIQ Neutron NPU using eIQ Neutron SDK.

Model Information

Information	Value
Input shape	RGB image (300, 300, 3)
Input example	(Image source, Public domain)
Output shape	Tensor of size (1917, 1, 4) containing bounding box coordinates (not decoded). Tensor of size (1917, 91) containing class scores for each box.
Output example
FLOPS	1.5G
Number of parameters	4.3M
File size (int8)	5.4M
Source framework	TensorFlow 1
Target platform	MPUs

Version and changelog

Initial release of quantized int8 model for i.MX 8M Plus, i.MX 93, i.MX 95, and i.MX 952. The model uses float32 input and float32 output tensors (internal operations are int8 quantized).

Tested configurations

The int8 model has been tested on i.MX 8M Plus, i.MX 93, i.MX 95, and i.MX 952 using benchmark-model (see i.MX Machine Learning User Guide).

Training and evaluation

The model has been trained and evaluated on the COCO dataset [1], which features 80 object classes. The floating-point model achieved 22 mAP on the test set, according to the source of the model.

The int8 quantized model was evaluated on COCO val2017 using the evaluate.py script and obtained 16.5 mAP (COCO metric, IoU=0.50:0.95; 29.2 mAP@IoU=0.50).

Instructions for re-training the model can be found in the TensorFlow Detection Model Zoo.

Conversion/Quantization

The original model is converted from TensorFlow 1 SavedModel format to TensorFlow Lite. Pre- and post-processing operations are removed before quantization so that the model only contains the feature extraction backbone and prediction heads.

Post-training quantization is performed with TensorFlow 2.2.0. 100 images randomly sampled from COCO val2017 are used as a calibration dataset; images are resized to 300x300 and normalized with image / 127.5 - 1 to match the model's expected input range.

Use case and limitations

This model can be used for fast object detection on 300x300 pixel images. It detects 80 COCO object categories. The model is not the most accurate but is suited for latency-constrained edge applications.

Download and run

To create the TensorFlow Lite model fully quantized in int8, run:

bash recipe.sh

The TFLite model file for i.MX 8M Plus is original_model/ssdlite_mobilenet_v2_quant.tflite. The files for i.MX 93 are in the imx93/ directory. The files for i.MX 95 are in the imx95/ directory. The files for i.MX 952 are in the imx952/ directory.

An example of how to use the model is in example.py:

python3 example.py -m original_model/ssdlite_mobilenet_v2_quant.tflite \
                   -i example_input.jpg -o example_output.jpg

To evaluate accuracy on COCO val2017:

python3 evaluate.py -m original_model/ssdlite_mobilenet_v2_quant.tflite

Please refer to i.MX Machine Learning User Guide for details on how to run the models on each platform.

Full on-device examples leveraging GStreamer/NNStreamer and this model can be found in nxp-nnstreamer-examples on GitHub

Origin

Model implementation: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf1_detection_zoo.md

[1] Lin, Tsung-Yi, et al. "Microsoft COCO: Common Objects in Context." European Conference on Computer Vision. Springer, Cham, 2014.

[2] Sandler, Mark, et al. "MobileNetV2: Inverted Residuals and Linear Bottlenecks." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.

[3] Liu, Wei, et al. "SSD: Single Shot MultiBox Detector." Computer Vision - ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016. Springer International Publishing, 2016.