| --- |
| license: other |
| license_name: sla0044 |
| license_link: >- |
| https://github.com/STMicroelectronics/stm32ai-modelzoo/raw/refs/heads/main/image_classification/LICENSE.md |
| pipeline_tag: image-classification |
| --- |
| # ResNet50 v2 |
|
|
| ## **Use case** : `Image classification` |
|
|
| # Model description |
|
|
|
|
| ResNets family is a well known architecture that uses skip connections to enable stronger gradients in much deeper networks. This variant has 50 layers. |
|
|
| The model is quantized in int8 using tensorflow lite converter. A mixed precision version is also provided using onnx-runtime and our own quantization scripts. |
|
|
| ## Network information |
|
|
|
|
| | Network Information | Value | |
| |-------------------------|-----------------| |
| | Framework | TensorFlow Lite | |
| | MParams | 25.6 M | |
| | Quantization | int8 | |
| | Provenance | https://www.tensorflow.org/api_docs/python/tf/keras/applications/ResNet50V2 | |
| | Paper | https://arxiv.org/abs/1603.05027 | |
| |
| The models are quantized using tensorflow lite converter. |
| |
| |
| ## Network inputs / outputs |
| |
| |
| For an image resolution of NxM and P classes |
| |
| | Input Shape | Description | |
| | ----- | ----------- | |
| | (1, N, M, 3) | Single NxM RGB image with UINT8 values between 0 and 255 | |
| |
| | Output Shape | Description | |
| | ----- | ----------- | |
| | (1, P) | Per-class confidence for P classes in FLOAT32| |
| |
| |
| ## Recommended platforms |
| |
| |
| | Platform | Supported | Recommended | |
| |----------|-----------|-----------| |
| | STM32L0 |[]|[]| |
| | STM32L4 |[]|[]| |
| | STM32U5 |[]|[]| |
| | STM32H7 |[x]|[]| |
| | STM32MP1 |[x]|[]| |
| | STM32MP2 |[x]|[x]| |
| | STM32N6 |[x]|[x]| |
| |
| # Performances |
| |
| ## Metrics |
| |
| - Measures are done with default STM32Cube.AI configuration with enabled input / output allocated option. |
| - `tfs` stands for "training from scratch", meaning that the model weights were randomly initialized before training. |
| - `tl` stands for "transfer learning", meaning that the model backbone weights were initialized from a pre-trained model, then only the last layer was unfrozen during the training. |
| - `fft` stands for "full fine-tuning", meaning that the full model weights were initialized from a transfer learning pre-trained model, and all the layers were unfrozen during the training. |
| |
| |
| ### Reference **NPU** memory footprint on food101 and imagenet dataset (see Accuracy for details on dataset) |
| |Model | Dataset | Format | Resolution | Series | Internal RAM | External RAM | Weights Flash | STEdgeAI Core version | |
| |----------|------------------|--------|-------------|------------------|--------------|--------------|----------------------|-------------------------| |
| | [ResNet50 v2 fft](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/ST_pretrainedmodel_public_dataset/food101/resnet50v2_224_fft/resnet50v2_224_fft_int8.tflite) | food101 | Int8 | 224x224x3 | STM32N6 | 2308.06 | 3136 | 23833.67 | 3.0.0 | |
| | [ResNet50 v2 fft](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/ST_pretrainedmodel_public_dataset/food101/resnet50v2_224_fft/resnet50v2_224_fft_qdq_w4_91.4%_w8_8.6%_a8_100%_acc_80.17.onnx) | food101 | Int8/Int4 | 224x224x3 | STM32N6 | 2308.06 | 2352 | 13268.39 | 3.0.0 | |
| | [ResNet50 v2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/Public_pretrainedmodel_public_dataset/imagenet/resnet50v2_224/resnet50v2_224_int8.tflite) | imagenet | Int8 | 224x224x3 | STM32N6 | 2308.06 | 3136.0 | 25633.61 | 3.0.0 | |
| | [ResNet50 v2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/Public_pretrainedmodel_public_dataset/imagenet/resnet50v2_224/resnet50v2_224_qdq_w4_35.98%_w8_64.02%_a8_100%_acc_67.45.onnx) | imagenet | Int8/Int4 | 224x224x3 | STM32N6 | 2308.06 | 2352 | 21154.53 | 3.0.0 | |
| |
| ### Reference **NPU** inference time on food101 and imagenet dataset (see Accuracy for details on dataset) |
| | Model | Dataset | Format | Resolution | Board | Execution Engine | Inference time (ms) | Inf / sec | STEdgeAI Core version | |
| |--------|------------------|--------|-------------|------------------|------------------|---------------------|-----------|-------------------------| |
| | [ResNet50 v2 fft](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/ST_pretrainedmodel_public_dataset/food101/resnet50v2_224_fft/resnet50v2_224_fft_int8.tflite) | food101 | Int8 | 224x224x3 | STM32N6570-DK | NPU/MCU | 238.49 | 4.19 | 3.0.0 | |
| | [ResNet50 v2 fft](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/ST_pretrainedmodel_public_dataset/food101/resnet50v2_224_fft/resnet50v2_224_fft_qdq_w4_91.4%_w8_8.6%_a8_100%_acc_80.17.onnx) | food101 | Int8/Int4 | 224x224x3 | STM32N6570-DK | NPU/MCU | 267.33 | 3.74 | 3.0.0 | |
| | [ResNet50 v2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/Public_pretrainedmodel_public_dataset/imagenet/resnet50v2_224/resnet50v2_224_int8.tflite) | imagenet | Int8 | 224x224x3 | STM32N6570-DK | NPU/MCU | 243.04 | 4.11 | 3.0.0 | |
| | [ResNet50 v2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/Public_pretrainedmodel_public_dataset/imagenet/resnet50v2_224/resnet50v2_224_qdq_w4_35.98%_w8_64.02%_a8_100%_acc_67.45.onnx) | imagenet | Int8/Int4 | 224x224x3 | STM32N6570-DK | NPU/MCU | 286.06 | 3.5 | 3.0.0 | |
| |
| |
| ### Reference **MCU** memory footprint based on Food-101 and imagenet dataset (see Accuracy for details on dataset) |
| |
| | Model | Dataset | Format | Resolution | Series | Activation RAM | Runtime RAM | Weights Flash | Code Flash | Total RAM | Total Flash | STEdgeAI Core version | |
| |-----------|---------------------------------------------------------------------------------------------------------------------------|--------|------------|---------|----------------|-------------|---------------|------------|-------------|-----------------|-----------------------| |
| | [ResNet50 v2 fft](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/ST_pretrainedmodel_public_dataset/food101/resnet50v2_224_fft/resnet50v2_224_fft_int8.tflite) | food101 | Int8 | 224x224x3 | STM32H7 | 1816.2 KiB | 14.56 KiB | 23240.96 KiB | 169.12 KiB | 1830.76 KiB | 23410.08 KiB | 3.0.0 | |
| | [ResNet50 v2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/Public_pretrainedmodel_public_dataset/imagenet/resnet50v2_224/resnet50v2_224_int8.tflite) | imagenet | Int8 | 224x224x3 | STM32H7 | 2142.07 KiB | 41.03 KiB | 25042.47 KiB | 225.32 KiB | 2183.1 KiB | 25267.79 KiB | 3.0.0 | |
| |
| |
| ### Reference **MCU** inference time based on Food-101 and imagenet dataset (see Accuracy for details on dataset) |
| |
| |
| | Model | Dataset | Format | Resolution | Board | Execution Engine | Frequency | Inference time (ms) | STEdgeAI Core version | |
| |-------------------|-----|---|------------|------------------|------------------|-----------|---------------------|-----------------------| |
| | [ResNet50 v2 fft](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/ST_pretrainedmodel_public_dataset/food101/resnet50v2_224_fft/resnet50v2_224_fft_int8.tflite) | food101 | Int8 | 224x224x3 | STM32H747I-DISCO | 1 CPU | 400 MHz | 11314.82 | 3.0.0 | |
| | [ResNet50 v2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/Public_pretrainedmodel_public_dataset/imagenet/resnet50v2_224/resnet50v2_224_int8.tflite) | imagenet | Int8 | 224x224x3 | STM32H747I-DISCO | 1 CPU | 400 MHz | 11370.07 | 3.0.0 | |
| |
| |
| |
| ### Accuracy with Food-101 dataset |
| |
| Dataset details: [link](https://data.vision.ee.ethz.ch/cvl/datasets_extra/food-101/), Quotation[[1]](#1) , Number of classes: 101 , Number of images: 101 000 |
| |
| | Model | Format | Resolution | Top 1 Accuracy | |
| |-------|--------|------------|----------------| |
| | [ResNet50 v2 fft](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/ST_pretrainedmodel_public_dataset/food101/resnet50v2_224_fft/resnet50v2_224_fft.keras) | Float | 224x224x3 | 82.2 % | |
| | [ResNet50 v2 fft](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/ST_pretrainedmodel_public_dataset/food101/resnet50v2_224_fft/resnet50v2_224_fft_int8.tflite) | Int8 | 224x224x3 | 81.03 % | |
| | [ResNet50 v2 fft](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/ST_pretrainedmodel_public_dataset/food101/resnet50v2_224_fft/resnet50v2_224_fft_qdq_w4_91.4%_w8_8.6%_a8_100%_acc_80.17.onnx) | Int8/Int4 | 224x224x3 | 80.17 % | |
| |
| ### Accuracy with imagenet dataset |
| |
| Dataset details: [link](https://www.image-net.org), Quotation[[4]](#4). |
| Number of classes: 1000. |
| To perform the quantization, we calibrated the activations with a random subset of the training set. |
| For the sake of simplicity, the accuracy reported here was estimated on the 50000 labelled images of the validation set. |
| |
| |model | Format | Resolution | Top 1 Accuracy | |
| |---------|--------|------------|----------------| |
| | [ResNet50 v2 ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/Public_pretrainedmodel_public_dataset/imagenet/resnet50v2_224/resnet50v2_224.keras) | Float | 224x224x3 | 68.73 % | |
| | [ResNet50 v2 ](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/Public_pretrainedmodel_public_dataset/imagenet/resnet50v2_224/resnet50v2_224_int8.tflite) | Int8 | 224x224x3 | 67.99 % | |
| | [ResNet50 v2](https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/image_classification/resnet50v2/Public_pretrainedmodel_public_dataset/imagenet/resnet50v2_224/resnet50v2_224_qdq_w4_35.98%_w8_64.02%_a8_100%_acc_67.45.onnx) | Int8/Int4 | 224x224x3 | 67.45 % | |
| |
| |
| ## Retraining and Integration in a simple example: |
| |
| Please refer to the stm32ai-modelzoo-services GitHub [here](https://github.com/STMicroelectronics/stm32ai-modelzoo-services) |
| |
| |
| |
| # References |
| |
| <a id="1">[1]</a> |
| L. Bossard, M. Guillaumin, and L. Van Gool, "Food-101 -- Mining Discriminative Components with Random Forests." European Conference on Computer Vision, 2014. |
