FastestDet Person Detection (persondetect)

Overview

FastestDet is a lightweight anchor-free object detector built from a ShuffleNetV2 backbone and a fully convolutional detection head. Given an RGB image, the network outputs a per-cell grid tensor that encodes an objectness score, a class score and a bounding box for the detected object.

This repository contains the person-detection ("persondetect") variant, trained on the PASCAL-VOC dataset, quantized to int8, and prepared for deployment on NXP MCUs with TensorFlow Lite Micro. The shippable artifacts here are C header files (model byte arrays) that are compiled directly into a TFLite Micro firmware image, since the target MCUs have no file system to load a .tflite file from. Two input resolutions are shipped: 160x128 and 220x220.

It also contains the script used to regenerate these assets (recipe.sh) and a host-side (x86) inference example (example.py) that runs the base .tflite model.

Ready-to-run application examples that use these models are provided in the eIQ Media Processing Pipeline (MPP):

Key Features

  • Very small (~153K parameter) anchor-free person detector.
  • Two input resolutions: 160x128 and 220x220.
  • Fully quantized int8 model (int8 input, int8 output).
  • Shipped as C byte-array headers for TensorFlow Lite Micro.
  • Prepared for three NXP MCUs, with and without Neutron NPU acceleration.

Model Description

This is a quantized and compiled version of FastestDet, retrained for person detection and optimized for NXP MCUs.

  • Base Model: FastestDet (dog-qiuqiu)
  • Original Model Authors: xuehao.ma (dog-qiuqiu)
  • Original License: BSD-3-Clause
  • Modified by: NXP

Modifications

This model is a derivative work with the following changes from the original:

  • Retraining: trained for person detection only, on the PASCAL-VOC dataset, person class.
  • Quantization: post-training int8 quantization, int8 input / int8 output. Two base models are provided, one per input resolution, in assets/quantized_tflite_model_mcu/.
  • Compilation (i.MX RT1170): no NPU compilation; the base int8 model runs as-is on the Cortex-M7 CPU.
  • Compilation (MCX-N947): compiled with the eIQ Neutron Compiler for the mcxn94x target (Neutron-16 NPU), 160x128 resolution only, with eIQ Neutron SDK 3.2.2 and 3.2.3.
  • Compilation (i.MX RT700): compiled with the eIQ Neutron Compiler for the imxrt700 target (Neutron-64 NPU), both resolutions, with eIQ Neutron SDK 3.2.2 and 3.2.3.
  • Format: every compiled .tflite is converted into a C byte-array header (xxd -i) and prefixed with the NXP banner plus the alignment / linker-placement macros expected by the eIQ MPP example projects.
  • Post-processing: grid decoding and non-maximum suppression are performed outside the model graph (see dperson.py).

Original model available at: https://github.com/dog-qiuqiu/FastestDet.

Model Information

Information Value
Input shape RGB image (1, H, W, 3), int8 (H x W = 160x128 or 220x220)
Input example (Image source, CC BY-SA 4.0)
Output shape Grid tensor (GRID_H, GRID_W, 6) - per cell: score, x, y, w, h, class. 160x128 -> (8, 10, 6); 220x220 -> (14, 14, 6). See example.py / dperson.py for decoding.
Output example
FLOPS Not reported
Number of parameters 152.9K
File size (int8) 461 KB (160x128), 309 KB (220x220) (assets/quantized_tflite_model_mcu/*.tflite)

Each MCU model directory also ships a matching *_tflite_info.h metadata header. It exposes the pre/post-processing constants the MPP example needs (input mean/std, output zero-point/scale, grid size, channel count, NMS and score thresholds), for example PERSONDETECT_INPUT_MEAN, PERSONDETECT_OUTPUT_SCALE, PERSONDETECT_GRID_HEIGHT/WIDTH, PERSONDETECT_NMS_THRESHOLD, PERSONDETECT_SCORE_THRESHOLD.

Supported Hardware and Software Configurations

All assets run with the TensorFlow Lite Micro runtime through eIQ Media Processing Pipeline (MPP). Each link points to a directory containing one asset per listed resolution, except for MCX-N947 which links directly to its single file. The same information is captured in release_manifest.json.

Asset Runtime Precision HW target BSP & SDK versions Resolutions Link to asset
i.MX RT1170 C header TensorFlow Lite Micro INT8 i.MX RT1170 MCUXpresso SDK 26.09.00 160x128, 220x220 Download
i.MX RT700 C header TensorFlow Lite Micro INT8 i.MX RT700 MCUXpresso SDK 26.09.00, Neutron SDK v3.2.2 160x128, 220x220 Download
i.MX RT700 C header TensorFlow Lite Micro INT8 i.MX RT700 MCUXpresso SDK 26.09.00, Neutron SDK v3.2.3 160x128, 220x220 Download
MCX-N947 C header TensorFlow Lite Micro INT8 MCX-N947 MCUXpresso SDK 26.09.00, Neutron SDK v3.2.2 160x128 Download
MCX-N947 C header TensorFlow Lite Micro INT8 MCX-N947 MCUXpresso SDK 26.09.00, Neutron SDK v3.2.3 160x128 Download

The hardware-independent int8 base models are also provided for host (x86) inference and as inputs to the Neutron Compiler: assets/quantized_tflite_model_mcu/persondetect_160_128.tflite and assets/quantized_tflite_model_mcu/persondetect_220_220.tflite.

Limitations and Biases

This model inherits all limitations from the original FastestDet model. Additional limitations:

  • Task scope: Person detection only (single class). The model does not perform re-identification, pose estimation or attribute recognition.
  • Dataset bias: the model is trained on PASCAL-VOC (person class only). Detection quality may vary for poses, lighting conditions, occlusions or demographic groups that are under-represented in that dataset.

Training

The model was trained and evaluated on the PASCAL-VOC 2012 dataset, person class only.

Download and Run

Running the example

A host-side (x86) reference of the pre/post-processing is provided in example.py. It runs the base .tflite model with tflite-runtime and draws the detected persons. Install the dependencies first:

python3 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

Then run, for example with the 220x220 model (matching example.py's defaults):

python example.py -m assets/quantized_tflite_model_mcu/persondetect_220_220.tflite -i example_input.jpg -o example_output.jpg

or with the 160x128 model:

python example.py -m assets/quantized_tflite_model_mcu/persondetect_160_128.tflite -i example_input.jpg -o example_output.jpg

The --thresh flag (default 0.6) sets the detection score threshold.

To run the models on an MCU target with TensorFlow Lite Micro, see the i.MX Machine Learning User Guide and the eIQ MPP camera_persondetect_view example linked above.

Regenerating the assets with recipe.sh

The recipe installs the eIQ Neutron SDK from the NXP eIQ PyPI Distribution Portal. Configure pip once to use it:

python -m pip config --user set global.index https://eiq.nxp.com/repository
python -m pip config --user set global.index-url https://eiq.nxp.com/repository
python -m pip config --user set global.trusted-host eiq.nxp.com
python -m pip config --user set global.extra-index-url https://pypi.org/simple
python -m pip config --user set global.find-links https://eiq.nxp.com/repository/

To regenerate all the shippable header files (Neutron compilation of the NPU variants and the xxd-based header conversion), run:

bash recipe.sh [EIQ_NEUTRON_SDK_VERSION]

The default eIQ Neutron SDK version is 3.2.3. The script:

  1. Creates a virtualenv and installs eiq-neutron-sdk.
  2. Compiles the two base models with the Neutron Compiler for mcxn94x (160x128 only) and imxrt700 (both resolutions).
  3. Converts every .tflite into a C header with the NXP banner and placement macros.

The generated header files are:

  • assets/rt1170/persondetect_160_128_tflite.h - i.MX RT1170 (CPU), 160x128
  • assets/rt1170/persondetect_220_220_tflite.h - i.MX RT1170 (CPU), 220x220
  • assets/mcxn947/<neutron_ver>/persondetect_160_128_npu16_tflite.h - MCX-N947 (Neutron-16), 160x128
  • assets/rt700/<neutron_ver>/persondetect_160_128_npu64_tflite.h - i.MX RT700 (Neutron-64), 160x128
  • assets/rt700/<neutron_ver>/persondetect_220_220_npu64_tflite.h - i.MX RT700 (Neutron-64), 220x220

(where <neutron_ver> is the eIQ Neutron SDK version, e.g. 3.2.2 or 3.2.3). Each *_tflite.h is accompanied by a matching *_tflite_info.h metadata header.

MCU targets (TensorFlow Lite Micro)

On MCUs the shippable artifacts are C header files containing the model as a byte array, to be compiled directly into the firmware image - not .tflite files. The i.MX RT1170 headers use the plain int8 model (no NPU) and live directly under assets/rt1170/. The Neutron (NPU) headers are versioned by the eIQ Neutron SDK used to compile them, so they go into a version-named subdirectory (assets/mcxn947/<version>/, assets/rt700/<version>/).

MCU Core / Accelerator Neutron target Resolutions shipped Directory
i.MX RT1170 Cortex-M7, CPU only - 160x128, 220x220 assets/rt1170/
MCX-N947 Neutron-16 (small NPU) mcxn94x 160x128 only assets/mcxn947/<version>/
i.MX RT700 Neutron-64 (large NPU) imxrt700 160x128, 220x220 assets/rt700/<version>/

License

This model, including the quantized and compiled assets and the scripts in this repository, is released under the BSD 3-Clause License (see LICENSE). The original FastestDet source code is also released under the BSD 3-Clause License (see MODEL_LICENSE).

Citation

If you use this model, please cite both this work and the original model:

  • FastestDet: https://github.com/dog-qiuqiu/FastestDet
  • N. Ma, X. Zhang, H.-T. Zheng and J. Sun, "ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design," Proceedings of the European Conference on Computer Vision (ECCV), 2018.
  • M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn and A. Zisserman, "The PASCAL Visual Object Classes (VOC) Challenge," International Journal of Computer Vision, vol. 88, pp. 303-338, 2010.
@misc{fastestdet,
  title = {FastestDet: Ultra lightweight anchor-free object detection model},
  author = {xuehao.ma (dog-qiuqiu)},
  howpublished = {\url{https://github.com/dog-qiuqiu/FastestDet}},
  year = {2022}
}

@InProceedings{Ma_2018_ECCV,
author = {Ma, Ningning and Zhang, Xiangyu and Zheng, Hai-Tao and Sun, Jian},
title = {ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
month = {September},
year = {2018}
} 

@article{EveringhamGWWZ10,
  author    = {Mark Everingham and
               Luc Van Gool and
               Christopher K. I. Williams and
               John M. Winn and
               Andrew Zisserman},
  title     = {The Pascal Visual Object Classes (VOC) Challenge},
  journal   = {International Journal of Computer Vision},
  volume    = {88},
  number    = {2},
  pages     = {303--338},
  year      = {2010},
  url       = {https://doi.org},
  doi       = {10.1007/s11263-009-0275-4}
}
Downloads last month
28
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collections including nxp/fastestdet-imx

Paper for nxp/fastestdet-imx