FastestDet Person Detection (persondetect)
Overview
FastestDet is a lightweight anchor-free object detector built from a ShuffleNetV2 backbone and a fully convolutional detection head. Given an RGB image, the network outputs a per-cell grid tensor that encodes an objectness score, a class score and a bounding box for the detected object.
This repository contains the person-detection ("persondetect") variant, trained on the
PASCAL-VOC dataset, quantized to int8, and prepared for deployment on NXP MCUs with
TensorFlow Lite Micro. The shippable artifacts
here are C header files (model byte arrays) that are compiled directly into a TFLite Micro
firmware image, since the target MCUs have no file system to load a .tflite file from. Two input
resolutions are shipped: 160x128 and 220x220.
It also contains the script used to regenerate these assets (recipe.sh) and a host-side (x86)
inference example (example.py) that runs the base .tflite model.
Ready-to-run application examples that use these models are provided in the eIQ Media Processing Pipeline (MPP):
- eIQ Media Processing Pipeline User's Guide: https://mcuxpresso.nxp.com/mcuxsdk/latest/html/middleware/eiq/mpp/docs/eIQ_Media_Processing_Pipeline_Users_Guide.html
- The
camera_persondetect_viewexample (uses these models): https://mcuxpresso.nxp.com/mcuxsdk/latest/html/examples/eiq_examples/mpp/camera_persondetect_view/readme.html
Key Features
- Very small (~153K parameter) anchor-free person detector.
- Two input resolutions: 160x128 and 220x220.
- Fully quantized int8 model (int8 input, int8 output).
- Shipped as C byte-array headers for TensorFlow Lite Micro.
- Prepared for three NXP MCUs, with and without Neutron NPU acceleration.
Model Description
This is a quantized and compiled version of FastestDet, retrained for person detection and optimized for NXP MCUs.
- Base Model: FastestDet (dog-qiuqiu)
- Original Model Authors: xuehao.ma (dog-qiuqiu)
- Original License: BSD-3-Clause
- Modified by: NXP
Modifications
This model is a derivative work with the following changes from the original:
- Retraining: trained for person detection only, on the PASCAL-VOC dataset, person class.
- Quantization: post-training int8 quantization, int8 input / int8 output. Two base models are
provided, one per input resolution, in
assets/quantized_tflite_model_mcu/. - Compilation (i.MX RT1170): no NPU compilation; the base int8 model runs as-is on the Cortex-M7 CPU.
- Compilation (MCX-N947): compiled with the eIQ Neutron Compiler for the
mcxn94xtarget (Neutron-16 NPU), 160x128 resolution only, with eIQ Neutron SDK3.2.2and3.2.3. - Compilation (i.MX RT700): compiled with the eIQ Neutron Compiler for the
imxrt700target (Neutron-64 NPU), both resolutions, with eIQ Neutron SDK3.2.2and3.2.3. - Format: every compiled
.tfliteis converted into a C byte-array header (xxd -i) and prefixed with the NXP banner plus the alignment / linker-placement macros expected by the eIQ MPP example projects. - Post-processing: grid decoding and non-maximum suppression are performed outside the model
graph (see
dperson.py).
Original model available at: https://github.com/dog-qiuqiu/FastestDet.
Model Information
| Information | Value |
|---|---|
| Input shape | RGB image (1, H, W, 3), int8 (H x W = 160x128 or 220x220) |
| Input example | (Image source, CC BY-SA 4.0) |
| Output shape | Grid tensor (GRID_H, GRID_W, 6) - per cell: score, x, y, w, h, class. 160x128 -> (8, 10, 6); 220x220 -> (14, 14, 6). See example.py / dperson.py for decoding. |
| Output example | ![]() |
| FLOPS | Not reported |
| Number of parameters | 152.9K |
| File size (int8) | 461 KB (160x128), 309 KB (220x220) (assets/quantized_tflite_model_mcu/*.tflite) |
Each MCU model directory also ships a matching *_tflite_info.h metadata header. It exposes the
pre/post-processing constants the MPP example needs (input mean/std, output zero-point/scale, grid
size, channel count, NMS and score thresholds), for example PERSONDETECT_INPUT_MEAN,
PERSONDETECT_OUTPUT_SCALE, PERSONDETECT_GRID_HEIGHT/WIDTH, PERSONDETECT_NMS_THRESHOLD,
PERSONDETECT_SCORE_THRESHOLD.
Supported Hardware and Software Configurations
All assets run with the TensorFlow Lite Micro runtime through eIQ Media Processing Pipeline (MPP). Each link points to a directory containing one asset per listed resolution, except for MCX-N947 which links directly to its single file. The same information is captured in release_manifest.json.
| Asset | Runtime | Precision | HW target | BSP & SDK versions | Resolutions | Link to asset |
|---|---|---|---|---|---|---|
| i.MX RT1170 C header | TensorFlow Lite Micro | INT8 | i.MX RT1170 | MCUXpresso SDK 26.09.00 | 160x128, 220x220 | Download |
| i.MX RT700 C header | TensorFlow Lite Micro | INT8 | i.MX RT700 | MCUXpresso SDK 26.09.00, Neutron SDK v3.2.2 | 160x128, 220x220 | Download |
| i.MX RT700 C header | TensorFlow Lite Micro | INT8 | i.MX RT700 | MCUXpresso SDK 26.09.00, Neutron SDK v3.2.3 | 160x128, 220x220 | Download |
| MCX-N947 C header | TensorFlow Lite Micro | INT8 | MCX-N947 | MCUXpresso SDK 26.09.00, Neutron SDK v3.2.2 | 160x128 | Download |
| MCX-N947 C header | TensorFlow Lite Micro | INT8 | MCX-N947 | MCUXpresso SDK 26.09.00, Neutron SDK v3.2.3 | 160x128 | Download |
The hardware-independent int8 base models are also provided for host (x86) inference and as inputs
to the Neutron Compiler: assets/quantized_tflite_model_mcu/persondetect_160_128.tflite and
assets/quantized_tflite_model_mcu/persondetect_220_220.tflite.
Limitations and Biases
This model inherits all limitations from the original FastestDet model. Additional limitations:
- Task scope: Person detection only (single class). The model does not perform re-identification, pose estimation or attribute recognition.
- Dataset bias: the model is trained on PASCAL-VOC (person class only). Detection quality may vary for poses, lighting conditions, occlusions or demographic groups that are under-represented in that dataset.
Training
The model was trained and evaluated on the PASCAL-VOC 2012 dataset, person class only.
Download and Run
Running the example
A host-side (x86) reference of the pre/post-processing is provided in example.py. It runs the base
.tflite model with tflite-runtime and draws the detected persons. Install the dependencies first:
python3 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
Then run, for example with the 220x220 model (matching example.py's defaults):
python example.py -m assets/quantized_tflite_model_mcu/persondetect_220_220.tflite -i example_input.jpg -o example_output.jpg
or with the 160x128 model:
python example.py -m assets/quantized_tflite_model_mcu/persondetect_160_128.tflite -i example_input.jpg -o example_output.jpg
The --thresh flag (default 0.6) sets the detection score threshold.
To run the models on an MCU target with TensorFlow Lite Micro, see the
i.MX Machine Learning User Guide
and the eIQ MPP camera_persondetect_view example linked above.
Regenerating the assets with recipe.sh
The recipe installs the eIQ Neutron SDK from the NXP eIQ PyPI Distribution Portal. Configure pip once to use it:
python -m pip config --user set global.index https://eiq.nxp.com/repository
python -m pip config --user set global.index-url https://eiq.nxp.com/repository
python -m pip config --user set global.trusted-host eiq.nxp.com
python -m pip config --user set global.extra-index-url https://pypi.org/simple
python -m pip config --user set global.find-links https://eiq.nxp.com/repository/
To regenerate all the shippable header files (Neutron compilation of the NPU variants and the
xxd-based header conversion), run:
bash recipe.sh [EIQ_NEUTRON_SDK_VERSION]
The default eIQ Neutron SDK version is 3.2.3. The script:
- Creates a virtualenv and installs
eiq-neutron-sdk. - Compiles the two base models with the Neutron Compiler for
mcxn94x(160x128 only) andimxrt700(both resolutions). - Converts every
.tfliteinto a C header with the NXP banner and placement macros.
The generated header files are:
assets/rt1170/persondetect_160_128_tflite.h- i.MX RT1170 (CPU), 160x128assets/rt1170/persondetect_220_220_tflite.h- i.MX RT1170 (CPU), 220x220assets/mcxn947/<neutron_ver>/persondetect_160_128_npu16_tflite.h- MCX-N947 (Neutron-16), 160x128assets/rt700/<neutron_ver>/persondetect_160_128_npu64_tflite.h- i.MX RT700 (Neutron-64), 160x128assets/rt700/<neutron_ver>/persondetect_220_220_npu64_tflite.h- i.MX RT700 (Neutron-64), 220x220
(where <neutron_ver> is the eIQ Neutron SDK version, e.g. 3.2.2 or 3.2.3). Each *_tflite.h is
accompanied by a matching *_tflite_info.h metadata header.
MCU targets (TensorFlow Lite Micro)
On MCUs the shippable artifacts are C header files containing the model as a byte array, to be
compiled directly into the firmware image - not .tflite files. The i.MX RT1170 headers use the
plain int8 model (no NPU) and live directly under assets/rt1170/. The Neutron (NPU) headers are
versioned by the eIQ Neutron SDK used to compile them, so they go into a version-named subdirectory
(assets/mcxn947/<version>/, assets/rt700/<version>/).
| MCU | Core / Accelerator | Neutron target | Resolutions shipped | Directory |
|---|---|---|---|---|
| i.MX RT1170 | Cortex-M7, CPU only | - | 160x128, 220x220 | assets/rt1170/ |
| MCX-N947 | Neutron-16 (small NPU) | mcxn94x |
160x128 only | assets/mcxn947/<version>/ |
| i.MX RT700 | Neutron-64 (large NPU) | imxrt700 |
160x128, 220x220 | assets/rt700/<version>/ |
License
This model, including the quantized and compiled assets and the scripts in this repository, is released under the BSD 3-Clause License (see LICENSE). The original FastestDet source code is also released under the BSD 3-Clause License (see MODEL_LICENSE).
Citation
If you use this model, please cite both this work and the original model:
- FastestDet: https://github.com/dog-qiuqiu/FastestDet
- N. Ma, X. Zhang, H.-T. Zheng and J. Sun, "ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design," Proceedings of the European Conference on Computer Vision (ECCV), 2018.
- M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn and A. Zisserman, "The PASCAL Visual Object Classes (VOC) Challenge," International Journal of Computer Vision, vol. 88, pp. 303-338, 2010.
@misc{fastestdet,
title = {FastestDet: Ultra lightweight anchor-free object detection model},
author = {xuehao.ma (dog-qiuqiu)},
howpublished = {\url{https://github.com/dog-qiuqiu/FastestDet}},
year = {2022}
}
@InProceedings{Ma_2018_ECCV,
author = {Ma, Ningning and Zhang, Xiangyu and Zheng, Hai-Tao and Sun, Jian},
title = {ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
month = {September},
year = {2018}
}
@article{EveringhamGWWZ10,
author = {Mark Everingham and
Luc Van Gool and
Christopher K. I. Williams and
John M. Winn and
Andrew Zisserman},
title = {The Pascal Visual Object Classes (VOC) Challenge},
journal = {International Journal of Computer Vision},
volume = {88},
number = {2},
pages = {303--338},
year = {2010},
url = {https://doi.org},
doi = {10.1007/s11263-009-0275-4}
}
- Downloads last month
- 28
(