Update README.md

286c4bd verified 6 days ago

8.66 kB

---
language:
- en
license: gpl-3.0
library_name: keras
tags:
- satellite-imagery
- earth-observation
- remote-sensing
- semantic-segmentation
- u-net
- inceptionresnetv2
- tflite
- edge-ai
- post-training-quantization
datasets:
- humans-in-the-loop/dubai-satellite-imagery
metrics:
- mean_iou
- dice
- accuracy
pipeline_tag: image-segmentation
---
# InceptionResNetV2-UNet: Dubai Satellite Semantic Segmentation & Edge Optimization

This repository hosts a production-grade, edge-optimized deep learning model designed for multi-class land-cover semantic segmentation from high-resolution satellite imagery ([Dubai Satellite Imagery Dataset](https://humansintheloop.org/resources/datasets/semantic-segmentation-dataset/)).

This work explores the **accuracy vs. model footprint trade-offs** for deploying aerial and satellite segmentation models to edge and on-orbit compute platforms.
![image](https://cdn-uploads.huggingface.co/production/uploads/69254ef0b14cd5831f1aafd0/B30oG-lD1Kys9Ck59FNZO.png)

![image](https://cdn-uploads.huggingface.co/production/uploads/69254ef0b14cd5831f1aafd0/hdTuWiyVVoXrFeBO-5kYg.png)

---



## 📌 Architecture & Model Overview

- **Architecture:** U-Net with an `InceptionResNetV2` feature extraction encoder
- **Total Parameters:** `36,793,414` (~36.79M)
- **Trainable Parameters:** `36,752,390` (~36.75M)
- **Non-trainable Parameters:** `41,024` (~160 KB)
- **Input Resolution:** `512 × 512 × 3` (RGB, normalized to [0, 1])
- **Output Resolution:** `512 × 512 × 6` (6-class probability distribution)
- **Quantization:** Dynamic Range Post-Training Quantization (PTQ) via TensorFlow Lite / LiteRT

---

## ⚙️ Training Specifications & Hyperparameters

- **Dataset Split:** 504 augmented training tiles, 16 validation tiles (512 x 512)
- **Batch Size:** `4`
- **Steps per Epoch:** `126`
- **Epochs Trained:** `33` (converged via EarlyStopping)
- **Optimizer:** `Adam`
- **Initial Learning Rate:** `0.0001` with Exponential Decay Scheduling
- **Loss Function:** `categorical_crossentropy`
- **Evaluation Metrics:** `dice_coef`, `categorical_accuracy`
- **Peak Validation Performance(Converged via Early Stopping at Epoch 33/50):**
  - Validation Accuracy: **`86.33%`**
  - Validation Dice Coefficient: **`81.10%`**
  - Validation Loss: **`0.4367`**

---


![image](https://cdn-uploads.huggingface.co/production/uploads/69254ef0b14cd5831f1aafd0/GJtk_fi1k1sbAJRORVNqN.png)

## 📊 Quantization & Deployment Benchmarks

We evaluated the performance, storage, and latency trade-offs between standard floating-point precision (FP32) and dynamic 8-bit integer quantization (INT8):

| Model Format | Precision | Disk Size | Size Reduction | Overall mIoU | 5-Class Terrain mIoU | CPU Latency | GPU Latency | Throughput |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **Keras (`.keras`)** | FP32 | 423.14 MB | Baseline (1.0x) | **59.09%** | **70.34%** | 4,693.27 ms | **638.39 ms** | 1.57 FPS (GPU) |
| **TFLite (`.tflite`)** | FP32 | 140.36 MB | ~3.0x smaller | **59.09%** | **70.34%** | ~4,200 ms | — | 0.24 FPS (CPU) |
| **TFLite INT8 (`.tflite`)** | Dynamic INT8 | **35.89 MB** | **11.8x smaller (-91.5%)** | **51.08%** | **60.28%** | 15,911.85 ms | — | 0.06 FPS (CPU) |

> **Key Finding:** Dynamic INT8 quantization slashes memory storage by **91.5%** (saving **387.25 MB** of storage), while retaining **60.28%** 5-class terrain mIoU.

---

## 🔍 Per-Class Evaluation Breakdown

Detailed analysis of class preservation between baseline FP32 and dynamic INT8:

| Class | RGB Color | Baseline IoU (FP32) | INT8 IoU (Quantized) | Change (Delta IoU) | Status |
| :--- | :---: | :---: | :---: | :---: | :--- |
| 🌊 **Water** | `(226, 169, 41)` | **0.9015** | 0.8849 | -0.0166 (-1.7%) | 🟢 Preserved almost perfectly |
| 🌿 **Vegetation** | `(254, 221, 58)` | 0.6163 | **0.6190** | +0.0027 (+0.3%) | 🟢 Completely preserved |
| 🏜️ **Land** | `(132, 41, 246)` | **0.8083** | 0.7761 | -0.0322 (-3.2%) | 🟢 Very stable |
| 🛣️ **Road** | `(110, 193, 228)` | **0.5105** | 0.3327 | -0.1778 (-17.8%) | 🟡 Moderate drop |
| 🏢 **Building** | `(60, 16, 152)` | **0.6806** | 0.4012 | -0.2794 (-27.9%) | 🔴 Major drop |
| ❓ **Unlabeled** | `(155, 155, 155)` | 0.0282 | 0.0509 | +0.0227 | — |
| **Overall mIoU** | — | **59.09%** | **51.08%** | **-8.01%** | |
| **5-Class Terrain mIoU** | — | **70.34%** | **60.28%** | **-10.06%** | |

---
### 📊 Model Performance: Baseline (FP32) vs. Quantized (INT8)

| Class | Baseline IoU (FP32) | INT8 IoU (PTQ) | Baseline Dice / F1 | INT8 Dice / F1 | Baseline Precision | INT8 Precision | Baseline Recall | INT8 Recall |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| 🌊 **Water** | **0.9015** | 0.8849 | **0.9482** | 0.9390 | **0.9429** | 0.9123 | 0.9536 | **0.9672** |
| 🏜️ **Land** | **0.8083** | 0.7761 | **0.8940** | 0.8740 | **0.8733** | 0.8101 | 0.9157 | **0.9488** |
| 🏢 **Building** | **0.6806** | 0.4012 | **0.8099** | 0.5727 | 0.7799 | **0.9023** | **0.8423** | 0.4195 |
| 🌿 **Vegetation** | 0.6163 | **0.6190** | 0.7626 | **0.7647** | **0.8441** | 0.7270 | 0.6954 | **0.8065** |
| 🛣️ **Road** | **0.5105** | 0.3327 | **0.6759** | 0.4993 | **0.6999** | 0.3820 | 0.6535 | **0.7203** |
| ❓ **Unlabeled** | 0.0282 | 0.0509 | 0.0548 | 0.0968 | 0.3341 | 0.3701 | 0.0298 | 0.0557 |
| **Mean (Overall)** | **0.5909** | **0.5108** | **0.6909** | **0.6244** | **0.7457** | **0.6840** | **0.6817** | **0.6530** |
| **5-Class Terrain** | **0.7034** | **0.6028** | **0.8181** | **0.7299** | **0.8280** | **0.7467** | **0.8121** | **0.7725** |

## Error Maps 
![image](https://cdn-uploads.huggingface.co/production/uploads/69254ef0b14cd5831f1aafd0/HT1WWmC5NRQ38I6Ei4iZt.png)

## 📂 Repository File Manifest

- `InceptionResNetV2-UNet.keras`: Native Keras 3 format containing architecture, weights, and optimizer state.
- `InceptionResNetV2-UNet.h5`: Keras HDF5 format for backward compatibility with legacy pipelines.
- `InceptionResNetV2-UNet.weights.h5`: Checkpoint containing only trained layer weights.
- `InceptionResNetV2-UNet_fp32.tflite`: Standalone TensorFlow Lite FP32 model for mobile and edge runtime.
- `InceptionResNetV2-UNet_int8.tflite`: Dynamically quantized 8-bit model optimized for ultra-low memory footprints (35.89 MB).

---

## 💻 Quickstart Inference Guide
### 1. Keras Full Precision Model (FP32)
```python
import tensorflow as tf
from huggingface_hub import hf_hub_download
model_path = hf_hub_download(
    repo_id="iamabhayaditya/InceptionResNetV2-UNet-MBRSC",
    filename="InceptionResNetV2-UNet.keras"
)
model = tf.keras.models.load_model(model_path, compile=False)
```
### 2. Edge / CPU Inference with Quantized TFLite (INT8)
```python
import numpy as np
import tensorflow as tf
from huggingface_hub import hf_hub_download
tflite_path = hf_hub_download(
    repo_id="iamabhayaditya/InceptionResNetV2-UNet-MBRSC",
    filename="InceptionResNetV2-UNet_int8.tflite"
)
interpreter = tf.lite.Interpreter(model_path=tflite_path)
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
# Input image shape: (1, 512, 512, 3), float32 normalized to [0, 1]
interpreter.set_tensor(input_details[0]['index'], input_image)
interpreter.invoke()
# Output shape: (1, 512, 512, 6)
predictions = interpreter.get_tensor(output_details[0]['index'])
pred_mask = np.argmax(predictions[0], axis=-1)
```

**GitHub Repo Link: https://github.com/imadi124/Drishti**
---
## 📜 Lineage & Attribution
This project is derived from the open-source implementation by [Ayush Dabra](https://github.com/ayushdabra/dubai-satellite-imagery-segmentation) licensed under **GNU General Public License v3.0 (GPL-3.0)**. 
**Modifications & Modernization:**
- Manually documented and commented every pipeline component, training routine, and utility function to ensure strict reproducibility and clean code standards.
- Engineered diagnostic visualization routines—including error heatmaps, class activation visualizations, and confusion matrices—to isolate edge-case failures (e.g., small building boundaries vs. narrow asphalt roads).
- Achieved superior convergence, lifting validation performance to **~86.24% categorical accuracy** and an **~81.72% Dice coefficient** (peak: **81.89%**).
- Upgraded deprecated dependencies to modern TensorFlow 2.x / Keras 3.
- Engineered dynamic Post-Training Quantization (PTQ) routines for TensorFlow Lite / LiteRT deployment.
- Established edge deployment benchmarking suites (measuring GPU vs. CPU latency, memory footprints, and per-class quantization degradation).

	---
	language:
	- en
	license: gpl-3.0
	library_name: keras
	tags:
	- satellite-imagery
	- earth-observation
	- remote-sensing
	- semantic-segmentation
	- u-net
	- inceptionresnetv2
	- tflite
	- edge-ai
	- post-training-quantization
	datasets:
	- humans-in-the-loop/dubai-satellite-imagery
	metrics:
	- mean_iou
	- dice
	- accuracy
	pipeline_tag: image-segmentation
	---

	# InceptionResNetV2-UNet: Dubai Satellite Semantic Segmentation & Edge Optimization

	This repository hosts a production-grade, edge-optimized deep learning model designed for multi-class land-cover semantic segmentation from high-resolution satellite imagery ([Dubai Satellite Imagery Dataset](https://humansintheloop.org/resources/datasets/semantic-segmentation-dataset/)).

	This work explores the accuracy vs. model footprint trade-offs for deploying aerial and satellite segmentation models to edge and on-orbit compute platforms.
	![image](https://cdn-uploads.huggingface.co/production/uploads/69254ef0b14cd5831f1aafd0/B30oG-lD1Kys9Ck59FNZO.png)

	![image](https://cdn-uploads.huggingface.co/production/uploads/69254ef0b14cd5831f1aafd0/hdTuWiyVVoXrFeBO-5kYg.png)

	---



	## 📌 Architecture & Model Overview

	- Architecture: U-Net with an `InceptionResNetV2` feature extraction encoder
	- Total Parameters: `36,793,414` (~36.79M)
	- Trainable Parameters: `36,752,390` (~36.75M)
	- Non-trainable Parameters: `41,024` (~160 KB)
	- Input Resolution: `512 × 512 × 3` (RGB, normalized to [0, 1])
	- Output Resolution: `512 × 512 × 6` (6-class probability distribution)
	- Quantization: Dynamic Range Post-Training Quantization (PTQ) via TensorFlow Lite / LiteRT

	---

	## ⚙️ Training Specifications & Hyperparameters

	- Dataset Split: 504 augmented training tiles, 16 validation tiles (512 x 512)
	- Batch Size: `4`
	- Steps per Epoch: `126`
	- Epochs Trained: `33` (converged via EarlyStopping)
	- Optimizer: `Adam`
	- Initial Learning Rate: `0.0001` with Exponential Decay Scheduling
	- Loss Function: `categorical_crossentropy`
	- Evaluation Metrics: `dice_coef`, `categorical_accuracy`
	- Peak Validation Performance(Converged via Early Stopping at Epoch 33/50):
	- Validation Accuracy: `86.33%`
	- Validation Dice Coefficient: `81.10%`
	- Validation Loss: `0.4367`

	---


	![image](https://cdn-uploads.huggingface.co/production/uploads/69254ef0b14cd5831f1aafd0/GJtk_fi1k1sbAJRORVNqN.png)

	## 📊 Quantization & Deployment Benchmarks

	We evaluated the performance, storage, and latency trade-offs between standard floating-point precision (FP32) and dynamic 8-bit integer quantization (INT8):

	\| Model Format \| Precision \| Disk Size \| Size Reduction \| Overall mIoU \| 5-Class Terrain mIoU \| CPU Latency \| GPU Latency \| Throughput \|
	\| :--- \| :---: \| :---: \| :---: \| :---: \| :---: \| :---: \| :---: \| :---: \|
	\| Keras (`.keras`) \| FP32 \| 423.14 MB \| Baseline (1.0x) \| 59.09% \| 70.34% \| 4,693.27 ms \| 638.39 ms \| 1.57 FPS (GPU) \|
	\| TFLite (`.tflite`) \| FP32 \| 140.36 MB \| ~3.0x smaller \| 59.09% \| 70.34% \| ~4,200 ms \| — \| 0.24 FPS (CPU) \|
	\| TFLite INT8 (`.tflite`) \| Dynamic INT8 \| 35.89 MB \| 11.8x smaller (-91.5%) \| 51.08% \| 60.28% \| 15,911.85 ms \| — \| 0.06 FPS (CPU) \|

	> Key Finding: Dynamic INT8 quantization slashes memory storage by 91.5% (saving 387.25 MB of storage), while retaining 60.28% 5-class terrain mIoU.

	---

	## 🔍 Per-Class Evaluation Breakdown

	Detailed analysis of class preservation between baseline FP32 and dynamic INT8:

	\| Class \| RGB Color \| Baseline IoU (FP32) \| INT8 IoU (Quantized) \| Change (Delta IoU) \| Status \|
	\| :--- \| :---: \| :---: \| :---: \| :---: \| :--- \|
	\| 🌊 Water \| `(226, 169, 41)` \| 0.9015 \| 0.8849 \| -0.0166 (-1.7%) \| 🟢 Preserved almost perfectly \|
	\| 🌿 Vegetation \| `(254, 221, 58)` \| 0.6163 \| 0.6190 \| +0.0027 (+0.3%) \| 🟢 Completely preserved \|
	\| 🏜️ Land \| `(132, 41, 246)` \| 0.8083 \| 0.7761 \| -0.0322 (-3.2%) \| 🟢 Very stable \|
	\| 🛣️ Road \| `(110, 193, 228)` \| 0.5105 \| 0.3327 \| -0.1778 (-17.8%) \| 🟡 Moderate drop \|
	\| 🏢 Building \| `(60, 16, 152)` \| 0.6806 \| 0.4012 \| -0.2794 (-27.9%) \| 🔴 Major drop \|
	\| ❓ Unlabeled \| `(155, 155, 155)` \| 0.0282 \| 0.0509 \| +0.0227 \| — \|
	\| Overall mIoU \| — \| 59.09% \| 51.08% \| -8.01% \| \|
	\| 5-Class Terrain mIoU \| — \| 70.34% \| 60.28% \| -10.06% \| \|

	---
	### 📊 Model Performance: Baseline (FP32) vs. Quantized (INT8)

	\| Class \| Baseline IoU (FP32) \| INT8 IoU (PTQ) \| Baseline Dice / F1 \| INT8 Dice / F1 \| Baseline Precision \| INT8 Precision \| Baseline Recall \| INT8 Recall \|
	\| :--- \| :---: \| :---: \| :---: \| :---: \| :---: \| :---: \| :---: \| :---: \|
	\| 🌊 Water \| 0.9015 \| 0.8849 \| 0.9482 \| 0.9390 \| 0.9429 \| 0.9123 \| 0.9536 \| 0.9672 \|
	\| 🏜️ Land \| 0.8083 \| 0.7761 \| 0.8940 \| 0.8740 \| 0.8733 \| 0.8101 \| 0.9157 \| 0.9488 \|
	\| 🏢 Building \| 0.6806 \| 0.4012 \| 0.8099 \| 0.5727 \| 0.7799 \| 0.9023 \| 0.8423 \| 0.4195 \|
	\| 🌿 Vegetation \| 0.6163 \| 0.6190 \| 0.7626 \| 0.7647 \| 0.8441 \| 0.7270 \| 0.6954 \| 0.8065 \|
	\| 🛣️ Road \| 0.5105 \| 0.3327 \| 0.6759 \| 0.4993 \| 0.6999 \| 0.3820 \| 0.6535 \| 0.7203 \|
	\| ❓ Unlabeled \| 0.0282 \| 0.0509 \| 0.0548 \| 0.0968 \| 0.3341 \| 0.3701 \| 0.0298 \| 0.0557 \|
	\| Mean (Overall) \| 0.5909 \| 0.5108 \| 0.6909 \| 0.6244 \| 0.7457 \| 0.6840 \| 0.6817 \| 0.6530 \|
	\| 5-Class Terrain \| 0.7034 \| 0.6028 \| 0.8181 \| 0.7299 \| 0.8280 \| 0.7467 \| 0.8121 \| 0.7725 \|

	## Error Maps
	![image](https://cdn-uploads.huggingface.co/production/uploads/69254ef0b14cd5831f1aafd0/HT1WWmC5NRQ38I6Ei4iZt.png)

	## 📂 Repository File Manifest

	- `InceptionResNetV2-UNet.keras`: Native Keras 3 format containing architecture, weights, and optimizer state.
	- `InceptionResNetV2-UNet.h5`: Keras HDF5 format for backward compatibility with legacy pipelines.
	- `InceptionResNetV2-UNet.weights.h5`: Checkpoint containing only trained layer weights.
	- `InceptionResNetV2-UNet_fp32.tflite`: Standalone TensorFlow Lite FP32 model for mobile and edge runtime.
	- `InceptionResNetV2-UNet_int8.tflite`: Dynamically quantized 8-bit model optimized for ultra-low memory footprints (35.89 MB).

	---

	## 💻 Quickstart Inference Guide
	### 1. Keras Full Precision Model (FP32)
	```python
	import tensorflow as tf
	from huggingface_hub import hf_hub_download
	model_path = hf_hub_download(
	repo_id="iamabhayaditya/InceptionResNetV2-UNet-MBRSC",
	filename="InceptionResNetV2-UNet.keras"
	)
	model = tf.keras.models.load_model(model_path, compile=False)
	```
	### 2. Edge / CPU Inference with Quantized TFLite (INT8)
	```python
	import numpy as np
	import tensorflow as tf
	from huggingface_hub import hf_hub_download
	tflite_path = hf_hub_download(
	repo_id="iamabhayaditya/InceptionResNetV2-UNet-MBRSC",
	filename="InceptionResNetV2-UNet_int8.tflite"
	)
	interpreter = tf.lite.Interpreter(model_path=tflite_path)
	interpreter.allocate_tensors()
	input_details = interpreter.get_input_details()
	output_details = interpreter.get_output_details()
	# Input image shape: (1, 512, 512, 3), float32 normalized to [0, 1]
	interpreter.set_tensor(input_details[0]['index'], input_image)
	interpreter.invoke()
	# Output shape: (1, 512, 512, 6)
	predictions = interpreter.get_tensor(output_details[0]['index'])
	pred_mask = np.argmax(predictions[0], axis=-1)
	```

	GitHub Repo Link: https://github.com/imadi124/Drishti
	---
	## 📜 Lineage & Attribution
	This project is derived from the open-source implementation by [Ayush Dabra](https://github.com/ayushdabra/dubai-satellite-imagery-segmentation) licensed under GNU General Public License v3.0 (GPL-3.0).
	Modifications & Modernization:
	- Manually documented and commented every pipeline component, training routine, and utility function to ensure strict reproducibility and clean code standards.
	- Engineered diagnostic visualization routines—including error heatmaps, class activation visualizations, and confusion matrices—to isolate edge-case failures (e.g., small building boundaries vs. narrow asphalt roads).
	- Achieved superior convergence, lifting validation performance to ~86.24% categorical accuracy and an ~81.72% Dice coefficient (peak: 81.89%).
	- Upgraded deprecated dependencies to modern TensorFlow 2.x / Keras 3.
	- Engineered dynamic Post-Training Quantization (PTQ) routines for TensorFlow Lite / LiteRT deployment.
	- Established edge deployment benchmarking suites (measuring GPU vs. CPU latency, memory footprints, and per-class quantization degradation).