DeepLabV3

Overview

DeepLabV3-MobileNetV2 is a lightweight deep learning model for semantic image segmentation, based on the DeepLabV3 architecture with a MobileNetV2 backbone (depth multiplier 0.5). Given an RGB image, it classifies every pixel into one of 20 PASCAL VOC object classes plus a background class.

This repository contains the pre-quantized int8 TFLite model, downloaded as-is from TensorFlow Hub and compiled for NXP platforms:

i.MX MPUs (i.MX 8M Plus, i.MX 93, i.MX 95, i.MX 952) as LiteRT (.tflite) models, at a single input resolution of 513x513.

It also contains the script used to regenerate these assets (recipe.sh) and an inference example (example.py).

Key Features

Fully quantized int8 weights, ready for edge deployment.
Lightweight MobileNetV2 backbone (depth multiplier 0.5) for fast inference.
Outputs per-pixel class scores for 21 classes (20 PASCAL VOC classes + background).
Compiled for i.MX 8M Plus (CPU/VX NPU), i.MX 93 (Ethos-U65 NPU) and i.MX 95 / i.MX 952 (eIQ Neutron NPU).

Model Description

This is a quantized and compiled version of the DeepLabV3-MobileNetV2 (depth multiplier 0.5) model published on TensorFlow Hub, optimized for NXP i.MX MPUs.

Base Model: DeepLabV3-MobileNetV2 dm0.5 int8 (TensorFlow Hub)
Original Model Authors: TensorFlow / DeepLab authors (Google), model packaged by Sayak Paul
Original License: Apache License 2.0 (see MODEL_LICENSE)
Modified by: NXP

Modifications

This model is a derivative work with the following changes from the original:

Quantization: none performed by NXP. The model is downloaded as an already int8-quantized TFLite file from TensorFlow Hub; its inputs and outputs remain float32 (only the weights and activations are int8).
Format: no conversion needed; the TFLite file from TensorFlow Hub is used directly as the hardware-independent reference model.
Compilation (i.MX 93): compiled with Vela for the Ethos-U65 NPU (--accelerator-config=ethos-u65-256 --optimise Performance), tags lf-6.18.20_2.0.0 and lf-6.18.37_2.1.0.
Compilation (i.MX 95 / i.MX 952): converted with the eIQ Neutron SDK versions 3.1.3, 3.2.1 and 3.2.3.

Original model available at: https://tfhub.dev/sayakpaul/lite-model/deeplabv3-mobilenetv2_dm05-int8/1/default/2.

Model Information

Information	Value
Input shape	RGB image (1, 513, 513, 3), float32, normalized as `(pixel - 127.5) / 127.5`
Input example
Output shape	(1, 513, 513, 21) float32 - per-pixel scores for 21 classes
Output example
FLOPS	1.76B OPS
File size (int8)	983 KB (`assets/quantized_tflite_model/deeplabv3_quant.tflite`)
Source framework	TensorFlow / TensorFlow Lite

Supported Hardware and Software Configurations

All models run with the eIQ LiteRT (TensorFlow Lite) runtime and the corresponding NPU delegate (VX for i.MX 8M Plus, Ethos-U for i.MX 93, Neutron for i.MX 95 / i.MX 952). The same information is captured in release_manifest.json.

Asset	Runtime	Precision	HW target	BSP & SDK versions	Link to asset
i.MX 8M Plus Quantized	LiteRT	INT8	i.MX 8M Plus	-	Download
i.MX 93 Precompiled	LiteRT	INT8	i.MX 93	BSP lf-6.18.20_2.0.0, VELA_TAG = lf-6.18.20_2.0.0	Download
i.MX 93 Precompiled	LiteRT	INT8	i.MX 93	BSP lf-6.18.37_2.1.0, VELA_TAG = lf-6.18.37_2.1.0	Download
i.MX 95 Precompiled	LiteRT	INT8	i.MX 95	BSP lf-6.18.20_2.0.0, Neutron SDK v3.1.3	Download
i.MX 95 Precompiled	LiteRT	INT8	i.MX 95	BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.1	Download
i.MX 95 Precompiled	LiteRT	INT8	i.MX 95	BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.3	Download
i.MX 952 Precompiled	LiteRT	INT8	i.MX 952	BSP lf-6.18.20_2.0.0, Neutron SDK v3.1.3	Download
i.MX 952 Precompiled	LiteRT	INT8	i.MX 952	BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.1	Download
i.MX 952 Precompiled	LiteRT	INT8	i.MX 952	BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.3	Download

The assets/imx8mplus/deeplabv3_quant.tflite file is identical to the hardware-independent reference model in assets/quantized_tflite_model/deeplabv3_quant.tflite (same sha256); it is also provided for host (x86) inference and as the input to the i.MX 93 / i.MX 95 / i.MX 952 compilation steps.

Limitations and Biases

This model inherits all limitations from the original DeepLabV3-MobileNetV2 model. Additional limitations:

Task scope: Semantic segmentation into the 20 PASCAL VOC classes plus background only; it does not perform instance segmentation or detect classes outside this label set.
Dataset bias: the model is trained on PASCAL VOC, which has limited object categories and scene diversity compared to real-world deployment conditions; segmentation quality may vary across lighting conditions, occlusions or scene types that are under-represented in that dataset.

Training

The original DeepLabV3-MobileNetV2 model was trained and evaluated by its authors on the PASCAL VOC dataset, following the recipe described in the TensorFlow Model Zoo. NXP did not retrain this model; the int8 weights are used as published on TensorFlow Hub.

Training code: TensorFlow Models - DeepLab train.py

Download and Run

Running the example

An example of running inference on a host PC is in example.py:

python3 example.py -m assets/quantized_tflite_model/deeplabv3_quant.tflite -i example_input.jpg -o example_output.jpg

example.py accepts -m/--model (default assets/quantized_tflite_model/deeplabv3_quant.tflite), -i/--input (default example_input.jpg) and -o/--output (optional; shows the result interactively if omitted).

Full on-device examples leveraging GStreamer/NNStreamer and this model can be found in nxp-nnstreamer-examples on GitHub. To run the model on an i.MX MPU target with the NPU delegates, see the i.MX Machine Learning User Guide.

Regenerating the assets with recipe.sh

The recipe installs the eIQ Neutron SDK from the NXP eIQ PyPI Distribution Portal. Configure pip once to use it:

python -m pip config --user set global.index https://eiq.nxp.com/repository
python -m pip config --user set global.index-url https://eiq.nxp.com/repository
python -m pip config --user set global.trusted-host eiq.nxp.com
python -m pip config --user set global.extra-index-url https://pypi.org/simple
python -m pip config --user set global.find-links https://eiq.nxp.com/repository/

To download and compile the model for all supported platforms, run:

bash recipe.sh

This will:

Download the pre-quantized int8 TFLite model from TensorFlow Hub into assets/quantized_tflite_model/deeplabv3_quant.tflite.
Compile it with Vela for i.MX 93 (Ethos-U65 NPU).
Convert it with the eIQ Neutron SDK for i.MX 95 and i.MX 952.

The recipe defaults to eIQ Neutron SDK 3.2.3 and Vela tag lf-6.18.20_2.0.0. Override the SDK version with the EIQ_NEUTRON_SDK_VERSION environment variable if a different release is needed. The generated files are:

assets/quantized_tflite_model/deeplabv3_quant.tflite - hardware-independent int8 model (also used for i.MX 8M Plus)
assets/imx93/<VELA_TAG>/deeplabv3_quant_vela.tflite - Vela-compiled for i.MX 93 Ethos-U65
assets/imx95/<EIQ_NEUTRON_SDK_VERSION>/deeplabv3_quant_converted.tflite - Neutron-converted for i.MX 95
assets/imx952/<EIQ_NEUTRON_SDK_VERSION>/deeplabv3_quant_converted.tflite - Neutron-converted for i.MX 952

License

This repository's scripts and recipe, as well as the compiled model assets, are released under the MIT License (see LICENSE). The original DeepLabV3-MobileNetV2 model downloaded from TensorFlow Hub is released under the Apache License 2.0 (see MODEL_LICENSE).

Citation

If you use this model, please cite both this work and the original model:

M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn and A. Zisserman, "The PASCAL Visual Object Classes (VOC) Challenge," International Journal of Computer Vision, 88(2):303-338, 2010.
L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff and H. Adam, "Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation," Proceedings of the European Conference on Computer Vision (ECCV), 2018.
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov and L.-C. Chen, "MobileNetV2: Inverted Residuals and Linear Bottlenecks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.

@article{everingham2010pascal,
  title={The pascal visual object classes (voc) challenge},
  author={Everingham, Mark and Van Gool, Luc and Williams, Christopher KI and Winn, John and Zisserman, Andrew},
  journal={International journal of computer vision},
  volume={88},
  number={2},
  pages={303--338},
  year={2010},
  publisher={Springer}
}

@inproceedings{chen2018encoder,
  title={Encoder-decoder with atrous separable convolution for semantic image segmentation},
  author={Chen, Liang-Chieh and Zhu, Yukun and Papandreou, George and Schroff, Florian and Adam, Hartwig},
  booktitle={European conference on computer vision},
  pages={833--851},
  year={2018},
  organization={Springer}
}

@inproceedings{sandler2018mobilenetv2,
  title={Mobilenetv2: Inverted residuals and linear bottlenecks},
  author={Sandler, Mark and Howard, Andrew and Zhu, Menglong and Zhmoginov, Andrey and Chen, Liang-Chieh},
  booktitle={2018 IEEE/CVF conference on computer vision and pattern recognition},
  pages={4510--4520},
  year={2018},
  organization={Ieee}
}

Downloads last month: 111

Collections including nxp/deeplabv3-imx