# Advanced Classification Uncertainty Quantification: Remote Sensing & Hyperspectral Analysis

[![Python](https://img.shields.io/badge/Python-3.9%2B-blue?style=for-the-badge&logo=python&logoColor=white)](https://img.shields.io/badge/Python-3.9%2B-blue?style=for-the-badge&logo=python&logoColor=white)
[![TensorFlow](https://img.shields.io/badge/TensorFlow-2.10%2B-orange?style=for-the-badge&logo=tensorflow&logoColor=white)](https://img.shields.io/badge/TensorFlow-2.10%2B-orange?style=for-the-badge&logo=tensorflow&logoColor=white)
[![Uncertainty](https://img.shields.io/badge/Uncertainty-Conformal%20%7C%20Ensemble%20%7C%20Bayesian-green?style=for-the-badge)](https://img.shields.io/badge/Uncertainty-Conformal%20%7C%20Ensemble%20%7C%20Bayesian-green?style=for-the-badge)
[![Status](https://img.shields.io/badge/Status-Active-success?style=for-the-badge)](https://img.shields.io/badge/Status-Active-success?style=for-the-badge)

Welcome to the **Classification Uncertainty Quantification (UQ) Analysis** framework. This repository provides a comprehensive, modular pipeline for quantifying predictive uncertainty in complex classification tasks, specifically optimized for high-dimensional remote sensing and multispectral imagery.

Beyond traditional point-wise accuracy, this framework implements state-of-the-art methods including **Conformal Prediction**, **Credal Deep Ensembles**, and **Deep Adaptive Predictive Modeling**, ensuring every classification decision is backed by a statistically valid measure of confidence.

---

## 📑 Table of Contents

1. [📌 Project Overview](#-project-overview)
2. [📂 Repository Structure](#-repository-structure)
3. [📊 Dataset & Usage](#-dataset--usage)
4. [🛠️ Workflow & Methodology](#-workflow--methodology)
    * [Method 1: Baseline Uncertainty](#method-1-baseline-uncertainty)
    * [Method 2: CREDIT (Calibration-Aware)](#method-2-credit-calibration-aware)
    * [Method 3: DAPM (Deep Adaptive Modeling)](#method-3-dapm-deep-adaptive-modeling)
    * [Method 4: Ensemble (CreDE)](#method-4-ensemble-crede)
    * [Method 5: MultiCP (Multi-head Conformal)](#method-5-multicp-multi-head-conformal)
    * [Method 6: SACP (Self-Adaptive Conformal)](#method-6-sacp-self-adaptive-conformal)
    * [Method 7: SCMCP (Spatial MultiCP)](#method-7-scmcp-spatial-multicp)
    * [Method 8: Focal Loss & CB Focal Loss](#method-8-focal-loss--cb-focal-loss)
    * [Method 9: EDL (Evidential Deep Learning)](#method-9-edl-evidential-deep-learning)
    * [Method 10: CDL (Credal Deep Learning)](#method-10-cdl-credal-deep-learning)
    * [Method 11: MambaHSI](#method-11-mambahsi)
5. [🚀 Getting Started](#-getting-started)
6. [📚 Resources & References](#-resources--references)
7. [🌐 6 Bands Data - Directory & Kaggle Links](#-6-bands-data---directory--kaggle-links)

---

## 📌 Project Overview

This framework addresses the critical need for reliability in machine learning models deployed for Earth Observation and spatial analysis. By integrating multiple UQ paradigms, it allows researchers to:

* **Identify Out-of-Distribution (OOD) Pixels**: Detect areas where the model is likely to fail.
* **Calibrate Probabilistic Outputs**: Ensure that a 90% confidence score actually corresponds to 90% accuracy.
* **Generate Valid Prediction Sets**: Use Conformal Prediction to produce sets of classes that contain the true label with a user-specified probability (e.g., 95%).
* **Analyze Spatial Uncertainty**: Visualize uncertainty maps across entire scenes (Hyperspectral/Multispectral).

**Key Features:**
* **Deep Architectures**: AlexNet CNN, GFNet (Global Filter Network), and ViT-UNet (Vision Transformer).
* **Modular Design**: Each method is self-contained with its own training logic, detailed theory/summary logs in `.md` format, and result visualization.
* **Remote Sensing Ready**: Built-in support for multispectral data structures.

---

## 📂 Repository Structure

The project is organized into a modular architecture optimized for portability and experimental reproducibility. The main source code is organized within the `Classification/` directory:

```text
.
├── Classification/    # 📂 Main Project Folder (Upload this to MyDrive/)
│   ├── baseline/      # 📂 Standard UQ (MC Dropout, Temp Scaling)
│   ├── credit/        # 📂 CREDIT: Calibration-aware training
│   ├── dapm/          # 📂 DAPM: Deep Adaptive Predictive Modeling
│   ├── data/          # 📊 Raw Datasets (data.csv, multispectral/)
│   ├── dofa/          # 📂 DOFA: Dynamic Wavelength Tokenization
│   ├── ensemble/      # 📂 CreDE: Credal Deep Ensembles
│   ├── georsclip/     # 📂 GeoRSCLIP: Geospatial Vision-Language Foundation Model (New)
│   ├── mambahsi/      # 📂 MambaHSI Architecture for classification
│   ├── multicp/       # 📂 MultiCP: Multi-head Conformal Prediction
│   ├── multicp_sacp/  # 📂 SCMCP: Spatial MultiCP
│   ├── sacp/          # 📂 SACP: Self-Adaptive Conformal Prediction
│   ├── spatialgcn/    # 📂 SpatialGCN: Spatial Graph Convolutional Networks (New)
│   └── trials/        # 🧪 Experimental Methods
│       ├── cb_focal_loss/
│       ├── cdl/
│       ├── conformal_reg/
│       ├── edl/
│       ├── edl_v2/
│       └── focal_loss/
├── examples/          # 📁 Example Dataset Suites (untouched)
└── README.md          # 📑 Documentation
```

---

## 📊 Dataset & Usage

**This is a Template Pipeline.**

To use this repository with your own classification data:

1. **Prepare Data**: Ensure your features and labels are in `.csv` or `.npy` format.
2. **Path Configuration**:
    * Place your data in the `Classification/data/` directory.
    * Notebooks use relative paths (e.g., `../data/data.csv`) for compatibility with Google Colab and local environments.
3. **Model Selection**: Choose from AlexNet, GFNet, or ViT-UNet by modifying the configuration in the respective notebooks.

---

## 🛠️ Workflow & Methodology

> **Note**: Each method subfolder in `Classification/` includes a comprehensive `.md` file detailing the mathematical framework, implementation logic, and experimental logs.

---

### Method 1: Baseline Uncertainty
**Location**: `Classification/baseline`
Establishes the performance and uncertainty floor using standard Bayesian approximations and Conformal Prediction.
* **Techniques**: Monte Carlo Dropout, Temperature Scaling, Conformal Prediction (SplitCP, CcCP, RC3P, ClusteredCP, RAPS).
* **Models**: AlexNet-CNN, GFNet, ViT-UNet.
* **Trained Weights Artifacts**: `AlexNet_CNN_best.keras`, `GFNet_best.keras`, `ViT_UNet_best.keras` (Input: 9×9×6, Output: 7 classes).

#### 6-Band Baseline Performance (100 Epochs, Kaggle GPU)
| Model | Test Accuracy | Cohen's Kappa | Macro F1 | Weighted F1 | Test Brier | Test ECE (15-bin) | Status |
|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
| **AlexNet_CNN** | **99.70%** | **0.9959** | **0.9955** | **0.9970** | **0.0044** | **0.0013** | ✅ Verified |
| **GFNet** | 99.63% | 0.9950 | 0.9954 | 0.9963 | 0.0057 | 0.0089 | ✅ Verified |
| **ViT_UNet** | 99.28% | 0.9902 | 0.9904 | 0.9928 | 0.0136 | 0.0388 | ✅ Verified |

#### Conformal Prediction Evaluation Summary ($\alpha = 0.05$, Target Coverage = 95.0%)
| Method | AlexNet Coverage | GFNet Coverage | ViT-UNet Coverage | Avg Set Size | Median Set Size |
|:---|:---:|:---:|:---:|:---:|:---:|
| **SplitCP** | 95.78% | 94.99% | 95.41% | 0.95 - 0.96 | 1.0 |
| **CcCP (Class-Conditional)** | 96.47% | 96.19% | 95.59% | 0.96 | 1.0 |
| **RC3P (Rank-Calibrated)** | 97.17% | 96.57% | 96.29% | 0.96 - 0.97 | 1.0 |
| **ClusteredCP (K=4)** | 96.52% | 95.96% | 95.36% | 0.96 | 1.0 |
| **RAPS** | 99.91% | 99.77% | 99.40% | 1.00 | 1.0 |

### Method 2: CREDIT
**Location**: `Classification/credit`
Confidence-Calibrated Robustness for Deep Image Classification.
* **Focus**: Improving the alignment between model confidence and actual performance.
* **Output**: Calibrated spatial uncertainty maps.

### Method 3: DAPM
**Location**: `Classification/dapm`
Deep Adaptive Predictive Modeling for classification under distribution shift.
* **Architecture**: Includes Encoder, Diffusion, and Multi-head decoders.
* **Result**: High-resolution p-value distributions and uncertainty masks.

### Method 4: Ensemble (CreDE)
**Location**: `Classification/ensemble`
Credal Deep Ensembles (CreDE) for robust uncertainty.
* **Process**: Training multiple model instances to capture epistemic uncertainty.
* **Metrics**: Credal entropy and variance-based measures.

### Method 5: MultiCP
**Location**: `Classification/multicp`
Multi-head Conformal Prediction.
* **Goal**: Distribution-free uncertainty sets with finite-sample validity.
* **Output**: Performance measures across different significance levels ($\alpha$).

### Method 6: SACP
**Location**: `Classification/sacp`
Self-Adaptive Conformal Prediction.
* **Advantage**: Online calibration and adaptation across varying spatial windows (ws=3, 5, 7, 9).
* **Summary**: Combined per-class coverage reports for all models.

### Method 7: SCMCP (Spatial MultiCP)
**Location**: `Classification/multicp_sacp`
Spatial Multi-Head Conformal Prediction.
* **Core**: Combines spatial probability smoothing with multi-head conformal intersection.
* **Feature**: Produces tight, spatially-coherent uncertainty regions by intersecting calibrated sets from $K$ heads after local smoothing.

### Method 8: Focal Loss & CB Focal Loss
**Location**: `Classification/trials/focal_loss` & `Classification/trials/cb_focal_loss`
* **Focus**: Addressing class imbalance using standard and Class-Balanced Focal Loss.
* **Output**: Improved minority class prediction with respective uncertainty estimates.

### Method 9: EDL (Evidential Deep Learning)
**Location**: `Classification/trials/edl` & `Classification/trials/edl_v2`
* **Core**: Uses Subjective Logic to place a Dirichlet distribution over class probabilities.
* **Goal**: Directly quantifies epistemic and aleatoric uncertainty without sampling.

### Method 10: CDL (Credal Deep Learning)
**Location**: `Classification/trials/cdl`
* **Goal**: Robust uncertainty quantification using imprecise probabilities and credal sets.

### Method 11: MambaHSI
**Location**: `Classification/mambahsi`
* **Architecture**: State-of-the-art Mamba state-space models adapted for Hyperspectral Imagery (HSI).
* **Focus**: Efficient long-range spatial-spectral dependencies modeling.

---

### Method 12: DOFA
**Location**: `Classification/dofa`
Dynamic Wavelength Tokenization.
* **Architecture**: DOFA Spectral and DOFA Hiera Fusion for robust multispectral tokenization.
* **Focus**: Analyzing prediction stability across continuous spectral channels.

---

### Method 13: GeoRSCLIP
**Location**: `Classification/georsclip`
Geospatial Vision-Language Foundation Model.
* **Architecture**: Text-head fine-tuning on GeoRSCLIP representations.
* **Focus**: Leveraging multimodal pre-training for remote sensing tasks.

### Method 14: SpatialGCN
**Location**: `Classification/spatialgcn`
Spatial Graph Convolutional Networks.
* **Architecture**: GCN processing on spatial grid topologies.
* **Focus**: Capturing complex spatial dependencies in classification maps.

---

## 🚀 Getting Started (Colab-First)

1. **Clone the Repository**:
   ```bash
   git clone https://github.com/DaneshSelwal/Classification_Uncertainty_Quantification_Analysis
   ```
2. **Upload to Google Drive**:
   Upload the `Classification/` folder to your `MyDrive/` directory.
3. **Run in Colab**:
   Navigate to any phase (e.g., `baseline/Model_training.ipynb`) and open with Google Colab.
   * Notebooks are pre-configured to mount `/content/drive` and find data in `/content/drive/MyDrive/Classification/data/`.
4. **Execution Order**:
   **Data Prep** $\rightarrow$ **Baseline Training** $\rightarrow$ **Advanced UQ Methods** $\rightarrow$ **Visualization**.

---

## 📚 Resources & References

This project leverages state-of-the-art research in Uncertainty Quantification and Remote Sensing. Below are the key resources and research papers utilized in this pipeline:

### 📖 Research Papers

* **Weighted Aggregation of Conformity Scores**: *Weighted Aggregation of Conformity Scores for Classification*. [ArXiv:2407.10230](https://arxiv.org/abs/2407.10230)
* **Uncertainty Sets for Image Classifiers**: *Uncertainty Sets for Image Classifiers using Conformal Prediction*. [ArXiv:2009.14193](https://arxiv.org/abs/2009.14193)
* **Conformal Prediction via Label Ranking**: *Conformal Prediction for Deep Classifier via Label Ranking*. [ArXiv:2310.06430](https://arxiv.org/abs/2310.06430)
* **Credal Ensemble Distillation**: *Credal Ensemble Distillation for Uncertainty Quantification*. [ArXiv:2511.13766](https://arxiv.org/abs/2511.13766)
* **Calibration of Modern Neural Networks**: *On Calibration of Modern Neural Networks*. [ICML 2017](https://arxiv.org/abs/1706.04599)
* **Deep Ensembles**: Lakshminarayanan, B., et al. (2017). *Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles*. [NeurIPS 2017](https://arxiv.org/abs/1612.01474)

### 🛠️ Libraries & Frameworks

* **SACP (Self-Adaptive CP)**: [GitHub](https://github.com/J4ckLiu/SACP)
* **TensorFlow / Keras**: Deep learning frameworks utilized for modeling architectures like AlexNet, GFNet, and ViT-UNet.

---
_This repository is developed under the guidance of Dr. Mahesh Pal._

---

# 6 Bands Data - Directory & Kaggle Links

Below is the directory structure mirroring our local repository, with direct links to the notebooks on Kaggle:

* 📁 **baseline/**
    * 📓 [Model_training.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-baseline-k01) — `cuq6-baseline-k01` ✅ **COMPLETE** (Fresh weights & metrics verified)
    * 📓 [Model_uncertainty_comparison.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-uncertainty-k02) — `cuq6-uncertainty-k02` ✅ **COMPLETE** (Conformal prediction evaluated)
* 📁 **credit/**
    * 📓 [Model_training_credit.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-credit-k03) — `cuq6-credit-k03` ✅ **COMPLETE** (CreDE Evidential distillation verified)
* 📁 **data/**
* 📁 **ensemble/**
    * 📓 [Model_training_ensembles.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-ensemble-k04) — `cuq6-ensemble-k04` ⚠️ **CANCELLED** (not re-run in this pass)
    * 📓 [Model_uncertainty_CreDE.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-crede-k05) — `cuq6-crede-k05` ✅ **COMPLETE** (Credal Deep Ensemble evaluation)
* 📁 **multicp/**
    * 📓 [Model_training_multihead.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-multicp-k06) — `cuq6-multicp-k06` ⚠️ **CANCELLED** (not re-run in this pass)
    * 📓 [Model_uncertainty_multicp.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-multicp-uq-k07) — `cuq6-multicp-uq-k07` ❌ **ERROR** (depends on multicp-k06)
* 📁 **multicp_sacp/**
    * 📓 [Model_uncertainty_multicp_sacp.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-multicp-sacp-k08) — `cuq6-multicp-sacp-k08` ❌ **ERROR** (depends on multicp-k06)
* 📁 **sacp/**
    * 📓 [Model_sacp_comparison.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-sacp-k09) — `cuq6-sacp-k09` ✅ **COMPLETE** (Multi-window SACP evaluated)
* 📁 **mambahsi/**
    * 📓 [Model_training_mambahsi.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-mambahsi-k10) — `cuq6-mambahsi-k10` ✅ **COMPLETE** (Fresh 6-band SSM weights trained)
    * 📓 [Model_uncertainty_comparison_mambahsi.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-mambahsi-uq-k11) — `cuq6-mambahsi-uq-k11` ✅ **COMPLETE** (SSM conformal prediction evaluated)
* 📁 **dofa/**
    * 📓 [Model_training.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-dofa-k12) — `cuq6-dofa-k12` ✅ **COMPLETE** (Fresh 6-band DOFA/Hiera weights trained)
    * 📓 [Model_uncertainty_comparison.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-dofa-uq-k13) — `cuq6-dofa-uq-k13` ✅ **COMPLETE** (DOFA conformal prediction evaluated)
* 📁 **georsclip/**
    * 📓 [Model_training.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-georsclip-k14) — `cuq6-georsclip-k14` ✅ **COMPLETE** (Vision-language text-head architecture)
    * 📓 [Model_uncertainty_comparison.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-georsclip-uq-k15) — `cuq6-georsclip-uq-k15` ✅ **COMPLETE** (GeoRSCLIP conformal prediction evaluated)
* 📁 **spatialgcn/**
    * 📓 [Model_training.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-spatialgcn-k16) — `cuq6-spatialgcn-k16` ✅ **COMPLETE** (Spatial graph convolution architecture)
    * 📓 [Model_uncertainty_comparison.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-spatialgcn-uq-k17) — `cuq6-spatialgcn-uq-k17` ✅ **COMPLETE** (SpatialGCN conformal prediction evaluated)