nxp/deepface-emotion-imx

Deepface Emotion

Overview

Deepface-emotion is a simple Convolutional Neural Network (CNN) for classification of facial expressions into 7 emotion categories: angry, disgust, fear, happy, sad, surprise, and neutral. The model takes a grayscale 48x48 face crop as input and outputs a vector of 7 class probabilities. We use the pretrained weights from the DeepFace library.

Note: This is not a face detection model. Faces must be detected and cropped before being passed to this model.

This repository contains the model quantized to int8 and compiled/converted for NXP i.MX 8M Plus, i.MX 93, i.MX 95, and i.MX 952 processors, together with the scripts used to reproduce the conversion (recipe.sh), run inference (example.py), and evaluate accuracy (evaluate.py).

Key Features

• Lightweight CNN optimized for edge deployment
• 7 emotion classes: angry, disgust, fear, happy, sad, surprise, neutral
• Grayscale 48x48 input - low memory footprint
• int8 quantized for accelerated NPU inference on NXP i.MX platforms

Model Description

This repository contains the Deepface emotion model quantized to int8 and compiled/converted for NXP i.MX 8M Plus, i.MX 93, i.MX 95, and i.MX 952 processors.

• Base Model: serengil/deepface (facial_expression_model_weights.h5)
• Original Model Authors: Sefik Ilkin Serengil
• Original License: MIT
• Modified by: NXP

Modifications

This model is a derivative work with the following changes from the original:

• Quantization: Converted from Keras float32 to TensorFlow Lite int8 (uint8 input, float32 output). Calibration uses a synthetic Gaussian-noise representative dataset (100 images), since the FER2013 dataset cannot be redistributed for this purpose.
• Compilation: Compiled for i.MX 93 with Ethos-U65 NPU using Vela (BSP tags lf-6.18.20_2.0.0 and lf-6.18.37_2.1.0).
• Conversion: Converted for i.MX 95 and i.MX 952 with the eIQ Neutron NPU using eIQ Neutron SDK versions 3.1.3, 3.2.1, and 3.2.3.

Original model available at: serengil/deepface.

Model Information

Information	Value
Input shape	Grayscale face crop (1, 48, 48, 1), dtype uint8
Input example	(Image from FER2013 dataset [1])
Output shape	Vector of probabilities shape (1, 7), dtype float32
Output labels	angry, disgust, fear, happy, sad, surprise, neutral
Output example	[[0., 0., 0., 0.996, 0., 0., 0.]] -> Recognized emotion: happy
FLOPS	58.5 MOPS
Number of parameters	1,485,831
Source framework	TensorFlow / Keras
File size (int8)	1.44 MB (1,510,984 bytes, reference / i.MX 8M Plus asset)

Supported Hardware and Software Configurations

The same information is captured in release_manifest.json.

Asset	Runtime	Precision	HW target	BSP & SDK versions	Link to asset
i.MX 8M Plus Quantized	LiteRT	INT8	i.MX 8M Plus	-	Download
i.MX 93 Precompiled	LiteRT	INT8	i.MX 93	BSP lf-6.18.20_2.0.0, VELA_TAG = lf-6.18.20_2.0.0	Download
i.MX 93 Precompiled	LiteRT	INT8	i.MX 93	BSP lf-6.18.37_2.1.0, VELA_TAG = lf-6.18.37_2.1.0	Download
i.MX 95 Precompiled	LiteRT	INT8	i.MX 95	BSP lf-6.18.20_2.0.0, Neutron SDK v3.1.3	Download
i.MX 95 Precompiled	LiteRT	INT8	i.MX 95	BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.1	Download
i.MX 95 Precompiled	LiteRT	INT8	i.MX 95	BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.3	Download
i.MX 952 Precompiled	LiteRT	INT8	i.MX 952	BSP lf-6.18.20_2.0.0, Neutron SDK v3.1.3	Download
i.MX 952 Precompiled	LiteRT	INT8	i.MX 952	BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.1	Download
i.MX 952 Precompiled	LiteRT	INT8	i.MX 952	BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.3	Download

Notes:

• The i.MX 93 assets for both BSP tags are byte-identical (same sha256); Vela compilation for Ethos-U65 did not change between these two BSP releases.
• The i.MX 8M Plus asset is byte-identical to assets/quantized_tflite_model/emotion_uint8_float32.tflite, which is also kept as the host (x86 CPU) reference model used by example.py and evaluate.py and is not listed as a separate row above.

Limitations and Biases

• Task scope: This model only classifies emotions from an already-cropped face image; it does not perform face detection. A separate face detector must run first.
• Fixed input resolution: The model only accepts a 48x48 grayscale crop; other resolutions or color images must be resized/converted before inference.
• Dataset bias: The model was trained on the FER2013 dataset, which is known to contain label noise and an imbalanced distribution across emotion classes and demographics. Performance may vary across different populations, lighting conditions, and image qualities not well represented in FER2013.

Training

The model was trained by the original DeepFace project on the FER2013 dataset [1]. NXP did not retrain or fine-tune the model; the pretrained Keras weights from serengil/deepface are used as-is, before quantization. The original training procedure is described here.

Evaluation

The evaluate.py script evaluates a TFLite model (quantized int8, uint8 input / float32 output) on the FER2013 PrivateTest split (3,589 images), running on a host PC CPU. For each image it quantizes the input using the model's quantization parameters, runs inference, and compares the predicted emotion class against the FER2013 label.

Download and Run

The recipe installs the eIQ Neutron SDK from the NXP eIQ PyPI Distribution Portal. Configure pip once to use it:

python -m pip config --user set global.index https://eiq.nxp.com/repository
python -m pip config --user set global.index-url https://eiq.nxp.com/repository
python -m pip config --user set global.trusted-host eiq.nxp.com
python -m pip config --user set global.extra-index-url https://pypi.org/simple
python -m pip config --user set global.find-links https://eiq.nxp.com/repository/

To recreate the TFLite models (quantized int8 with uint8 input and float32 output), run:

bash recipe.sh [IMX_ETHOS_U_VELA_TAG] [EIQ_NEUTRON_SDK_VERSION]

Both arguments are optional; they default to lf-6.18.20_2.0.0 and 3.2.3 respectively, for example:

bash recipe.sh lf-6.18.37_2.1.0 3.2.1

The recipe uses a synthetic Gaussian-noise representative dataset for calibration, so no external dataset or Kaggle account is required. It generates:

• assets/quantized_tflite_model/emotion_uint8_float32.tflite - int8 quantized model for i.MX 8M Plus (also linked as assets/imx8mplus/emotion_uint8_float32.tflite).
• assets/imx93/<IMX_ETHOS_U_VELA_TAG>/emotion_uint8_float32_vela.tflite - Vela-compiled model for i.MX 93 (Ethos-U65 NPU).
• assets/imx95/<EIQ_NEUTRON_SDK_VERSION>/emotion_uint8_float32_converted.tflite - Neutron-converted model for i.MX 95.
• assets/imx952/<EIQ_NEUTRON_SDK_VERSION>/emotion_uint8_float32_converted.tflite - Neutron-converted model for i.MX 952.

An example of how to use the model is in example.py (run on a PC):

python example.py -m assets/quantized_tflite_model/emotion_uint8_float32.tflite -i example_input.jpg

example.py accepts:

• -m / --model: path to the TFLite model file (default assets/quantized_tflite_model/emotion_uint8_float32.tflite).
• -i / --input: path to a grayscale face crop image, resized internally to 48x48 (default example_input.jpg).

Please refer to the i.MX Machine Learning User Guide for details on how to run the models on each platform.

Full on-device examples leveraging GStreamer/NNStreamer and this model can be found in nxp-nnstreamer-examples on GitHub.

License

This repository (code and scripts) is released under the MIT License (see LICENSE). The model weights are derived from the original serengil/deepface model, also released under the MIT License (see MODEL_LICENSE).

Citation

If you use this model, please cite both this work and the original model and dataset:

• Model implementation: serengil/deepface
• I. J. Goodfellow et al., "Challenges in representation learning: A report on three machine learning contests," International Conference on Neural Information Processing. Springer, Berlin, Heidelberg, 2013.

@inproceedings{goodfellow2013challenges,
  title={Challenges in representation learning: A report on three machine learning contests},
  author={Goodfellow, Ian J and Erhan, Dumitru and Carrier, Pierre Luc and Courville, Aaron and Mirza, Mehdi and Hamner, Ben and Cukierski, Will and Tang, Yichuan and Thaler, David and Lee, Dong-Hyun and others},
  booktitle={International conference on neural information processing},
  pages={117--124},
  year={2013},
  organization={Springer}
}