Spaces:

DaneshSelwal
/

Classification-6Bands-Workspace

Configuration error

App Files Files Community

Classification-6Bands-Workspace / README.md

DaneshSelwal

docs: accurate kernel statuses

a5c36db verified 21 days ago

preview code

Raw History Blame Contribute Delete

17 kB

# Advanced Classification Uncertainty Quantification: Remote Sensing & Hyperspectral Analysis

[![Python](https://img.shields.io/badge/Python-3.9%2B-blue?style=for-the-badge&logo=python&logoColor=white)](https://img.shields.io/badge/Python-3.9%2B-blue?style=for-the-badge&logo=python&logoColor=white)
[![TensorFlow](https://img.shields.io/badge/TensorFlow-2.10%2B-orange?style=for-the-badge&logo=tensorflow&logoColor=white)](https://img.shields.io/badge/TensorFlow-2.10%2B-orange?style=for-the-badge&logo=tensorflow&logoColor=white)
[![Uncertainty](https://img.shields.io/badge/Uncertainty-Conformal%20%7C%20Ensemble%20%7C%20Bayesian-green?style=for-the-badge)](https://img.shields.io/badge/Uncertainty-Conformal%20%7C%20Ensemble%20%7C%20Bayesian-green?style=for-the-badge)
[![Status](https://img.shields.io/badge/Status-Active-success?style=for-the-badge)](https://img.shields.io/badge/Status-Active-success?style=for-the-badge)

Welcome to the **Classification Uncertainty Quantification (UQ) Analysis** framework. This repository provides a comprehensive, modular pipeline for quantifying predictive uncertainty in complex classification tasks, specifically optimized for high-dimensional remote sensing and multispectral imagery.

Beyond traditional point-wise accuracy, this framework implements state-of-the-art methods including **Conformal Prediction**, **Credal Deep Ensembles**, and **Deep Adaptive Predictive Modeling**, ensuring every classification decision is backed by a statistically valid measure of confidence.

---

## 📑 Table of Contents

1. [📌 Project Overview](#-project-overview)
2. [📂 Repository Structure](#-repository-structure)
3. [📊 Dataset & Usage](#-dataset--usage)
4. [🛠️ Workflow & Methodology](#-workflow--methodology)
    * [Method 1: Baseline Uncertainty](#method-1-baseline-uncertainty)
    * [Method 2: CREDIT (Calibration-Aware)](#method-2-credit-calibration-aware)
    * [Method 3: DAPM (Deep Adaptive Modeling)](#method-3-dapm-deep-adaptive-modeling)
    * [Method 4: Ensemble (CreDE)](#method-4-ensemble-crede)
    * [Method 5: MultiCP (Multi-head Conformal)](#method-5-multicp-multi-head-conformal)
    * [Method 6: SACP (Self-Adaptive Conformal)](#method-6-sacp-self-adaptive-conformal)
    * [Method 7: SCMCP (Spatial MultiCP)](#method-7-scmcp-spatial-multicp)
    * [Method 8: Focal Loss & CB Focal Loss](#method-8-focal-loss--cb-focal-loss)
    * [Method 9: EDL (Evidential Deep Learning)](#method-9-edl-evidential-deep-learning)
    * [Method 10: CDL (Credal Deep Learning)](#method-10-cdl-credal-deep-learning)
    * [Method 11: MambaHSI](#method-11-mambahsi)
5. [🚀 Getting Started](#-getting-started)
6. [📚 Resources & References](#-resources--references)
7. [🌐 6 Bands Data - Directory & Kaggle Links](#-6-bands-data---directory--kaggle-links)

---

## 📌 Project Overview

This framework addresses the critical need for reliability in machine learning models deployed for Earth Observation and spatial analysis. By integrating multiple UQ paradigms, it allows researchers to:

* **Identify Out-of-Distribution (OOD) Pixels**: Detect areas where the model is likely to fail.
* **Calibrate Probabilistic Outputs**: Ensure that a 90% confidence score actually corresponds to 90% accuracy.
* **Generate Valid Prediction Sets**: Use Conformal Prediction to produce sets of classes that contain the true label with a user-specified probability (e.g., 95%).
* **Analyze Spatial Uncertainty**: Visualize uncertainty maps across entire scenes (Hyperspectral/Multispectral).

**Key Features:**
* **Deep Architectures**: AlexNet CNN, GFNet (Global Filter Network), and ViT-UNet (Vision Transformer).
* **Modular Design**: Each method is self-contained with its own training logic, detailed theory/summary logs in `.md` format, and result visualization.
* **Remote Sensing Ready**: Built-in support for multispectral data structures.

---

## 📂 Repository Structure

The project is organized into a modular architecture optimized for portability and experimental reproducibility. The main source code is organized within the `Classification/` directory:

```text
.
├── Classification/    # 📂 Main Project Folder (Upload this to MyDrive/)
│   ├── baseline/      # 📂 Standard UQ (MC Dropout, Temp Scaling)
│   ├── credit/        # 📂 CREDIT: Calibration-aware training
│   ├── dapm/          # 📂 DAPM: Deep Adaptive Predictive Modeling
│   ├── data/          # 📊 Raw Datasets (data.csv, multispectral/)
│   ├── dofa/          # 📂 DOFA: Dynamic Wavelength Tokenization
│   ├── ensemble/      # 📂 CreDE: Credal Deep Ensembles
│   ├── georsclip/     # 📂 GeoRSCLIP: Geospatial Vision-Language Foundation Model (New)
│   ├── mambahsi/      # 📂 MambaHSI Architecture for classification
│   ├── multicp/       # 📂 MultiCP: Multi-head Conformal Prediction
│   ├── multicp_sacp/  # 📂 SCMCP: Spatial MultiCP
│   ├── sacp/          # 📂 SACP: Self-Adaptive Conformal Prediction
│   ├── spatialgcn/    # 📂 SpatialGCN: Spatial Graph Convolutional Networks (New)
│   └── trials/        # 🧪 Experimental Methods
│       ├── cb_focal_loss/
│       ├── cdl/
│       ├── conformal_reg/
│       ├── edl/
│       ├── edl_v2/
│       └── focal_loss/
├── examples/          # 📁 Example Dataset Suites (untouched)
└── README.md          # 📑 Documentation
```

---

## 📊 Dataset & Usage

**This is a Template Pipeline.**

To use this repository with your own classification data:

1. **Prepare Data**: Ensure your features and labels are in `.csv` or `.npy` format.
2. **Path Configuration**:
    * Place your data in the `Classification/data/` directory.
    * Notebooks use relative paths (e.g., `../data/data.csv`) for compatibility with Google Colab and local environments.
3. **Model Selection**: Choose from AlexNet, GFNet, or ViT-UNet by modifying the configuration in the respective notebooks.

---

## 🛠️ Workflow & Methodology

> **Note**: Each method subfolder in `Classification/` includes a comprehensive `.md` file detailing the mathematical framework, implementation logic, and experimental logs.

---

### Method 1: Baseline Uncertainty
**Location**: `Classification/baseline`
Establishes the performance and uncertainty floor using standard Bayesian approximations and Conformal Prediction.
* **Techniques**: Monte Carlo Dropout, Temperature Scaling, Conformal Prediction (SplitCP, CcCP, RC3P, ClusteredCP, RAPS).
* **Models**: AlexNet-CNN, GFNet, ViT-UNet.
* **Trained Weights Artifacts**: `AlexNet_CNN_best.keras`, `GFNet_best.keras`, `ViT_UNet_best.keras` (Input: 9×9×6, Output: 7 classes).

#### 6-Band Baseline Performance (100 Epochs, Kaggle GPU)
| Model | Test Accuracy | Cohen's Kappa | Macro F1 | Weighted F1 | Test Brier | Test ECE (15-bin) | Status |
|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
| **AlexNet_CNN** | **99.70%** | **0.9959** | **0.9955** | **0.9970** | **0.0044** | **0.0013** | ✅ Verified |
| **GFNet** | 99.63% | 0.9950 | 0.9954 | 0.9963 | 0.0057 | 0.0089 | ✅ Verified |
| **ViT_UNet** | 99.28% | 0.9902 | 0.9904 | 0.9928 | 0.0136 | 0.0388 | ✅ Verified |

#### Conformal Prediction Evaluation Summary ($\alpha = 0.05$, Target Coverage = 95.0%)
| Method | AlexNet Coverage | GFNet Coverage | ViT-UNet Coverage | Avg Set Size | Median Set Size |
|:---|:---:|:---:|:---:|:---:|:---:|
| **SplitCP** | 95.78% | 94.99% | 95.41% | 0.95 - 0.96 | 1.0 |
| **CcCP (Class-Conditional)** | 96.47% | 96.19% | 95.59% | 0.96 | 1.0 |
| **RC3P (Rank-Calibrated)** | 97.17% | 96.57% | 96.29% | 0.96 - 0.97 | 1.0 |
| **ClusteredCP (K=4)** | 96.52% | 95.96% | 95.36% | 0.96 | 1.0 |
| **RAPS** | 99.91% | 99.77% | 99.40% | 1.00 | 1.0 |

### Method 2: CREDIT
**Location**: `Classification/credit`
Confidence-Calibrated Robustness for Deep Image Classification.
* **Focus**: Improving the alignment between model confidence and actual performance.
* **Output**: Calibrated spatial uncertainty maps.

### Method 3: DAPM
**Location**: `Classification/dapm`
Deep Adaptive Predictive Modeling for classification under distribution shift.
* **Architecture**: Includes Encoder, Diffusion, and Multi-head decoders.
* **Result**: High-resolution p-value distributions and uncertainty masks.

### Method 4: Ensemble (CreDE)
**Location**: `Classification/ensemble`
Credal Deep Ensembles (CreDE) for robust uncertainty.
* **Process**: Training multiple model instances to capture epistemic uncertainty.
* **Metrics**: Credal entropy and variance-based measures.

### Method 5: MultiCP
**Location**: `Classification/multicp`
Multi-head Conformal Prediction.
* **Goal**: Distribution-free uncertainty sets with finite-sample validity.
* **Output**: Performance measures across different significance levels ($\alpha$).

### Method 6: SACP
**Location**: `Classification/sacp`
Self-Adaptive Conformal Prediction.
* **Advantage**: Online calibration and adaptation across varying spatial windows (ws=3, 5, 7, 9).
* **Summary**: Combined per-class coverage reports for all models.

### Method 7: SCMCP (Spatial MultiCP)
**Location**: `Classification/multicp_sacp`
Spatial Multi-Head Conformal Prediction.
* **Core**: Combines spatial probability smoothing with multi-head conformal intersection.
* **Feature**: Produces tight, spatially-coherent uncertainty regions by intersecting calibrated sets from $K$ heads after local smoothing.

### Method 8: Focal Loss & CB Focal Loss
**Location**: `Classification/trials/focal_loss` & `Classification/trials/cb_focal_loss`
* **Focus**: Addressing class imbalance using standard and Class-Balanced Focal Loss.
* **Output**: Improved minority class prediction with respective uncertainty estimates.

### Method 9: EDL (Evidential Deep Learning)
**Location**: `Classification/trials/edl` & `Classification/trials/edl_v2`
* **Core**: Uses Subjective Logic to place a Dirichlet distribution over class probabilities.
* **Goal**: Directly quantifies epistemic and aleatoric uncertainty without sampling.

### Method 10: CDL (Credal Deep Learning)
**Location**: `Classification/trials/cdl`
* **Goal**: Robust uncertainty quantification using imprecise probabilities and credal sets.

### Method 11: MambaHSI
**Location**: `Classification/mambahsi`
* **Architecture**: State-of-the-art Mamba state-space models adapted for Hyperspectral Imagery (HSI).
* **Focus**: Efficient long-range spatial-spectral dependencies modeling.

---

### Method 12: DOFA
**Location**: `Classification/dofa`
Dynamic Wavelength Tokenization.
* **Architecture**: DOFA Spectral and DOFA Hiera Fusion for robust multispectral tokenization.
* **Focus**: Analyzing prediction stability across continuous spectral channels.

---

### Method 13: GeoRSCLIP
**Location**: `Classification/georsclip`
Geospatial Vision-Language Foundation Model.
* **Architecture**: Text-head fine-tuning on GeoRSCLIP representations.
* **Focus**: Leveraging multimodal pre-training for remote sensing tasks.

### Method 14: SpatialGCN
**Location**: `Classification/spatialgcn`
Spatial Graph Convolutional Networks.
* **Architecture**: GCN processing on spatial grid topologies.
* **Focus**: Capturing complex spatial dependencies in classification maps.

---

## 🚀 Getting Started (Colab-First)

1. **Clone the Repository**:
   ```bash
   git clone https://github.com/DaneshSelwal/Classification_Uncertainty_Quantification_Analysis
   ```
2. **Upload to Google Drive**:
   Upload the `Classification/` folder to your `MyDrive/` directory.
3. **Run in Colab**:
   Navigate to any phase (e.g., `baseline/Model_training.ipynb`) and open with Google Colab.
   * Notebooks are pre-configured to mount `/content/drive` and find data in `/content/drive/MyDrive/Classification/data/`.
4. **Execution Order**:
   **Data Prep** $\rightarrow$ **Baseline Training** $\rightarrow$ **Advanced UQ Methods** $\rightarrow$ **Visualization**.

---

## 📚 Resources & References

This project leverages state-of-the-art research in Uncertainty Quantification and Remote Sensing. Below are the key resources and research papers utilized in this pipeline:

### 📖 Research Papers

* **Weighted Aggregation of Conformity Scores**: *Weighted Aggregation of Conformity Scores for Classification*. [ArXiv:2407.10230](https://arxiv.org/abs/2407.10230)
* **Uncertainty Sets for Image Classifiers**: *Uncertainty Sets for Image Classifiers using Conformal Prediction*. [ArXiv:2009.14193](https://arxiv.org/abs/2009.14193)
* **Conformal Prediction via Label Ranking**: *Conformal Prediction for Deep Classifier via Label Ranking*. [ArXiv:2310.06430](https://arxiv.org/abs/2310.06430)
* **Credal Ensemble Distillation**: *Credal Ensemble Distillation for Uncertainty Quantification*. [ArXiv:2511.13766](https://arxiv.org/abs/2511.13766)
* **Calibration of Modern Neural Networks**: *On Calibration of Modern Neural Networks*. [ICML 2017](https://arxiv.org/abs/1706.04599)
* **Deep Ensembles**: Lakshminarayanan, B., et al. (2017). *Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles*. [NeurIPS 2017](https://arxiv.org/abs/1612.01474)

### 🛠️ Libraries & Frameworks

* **SACP (Self-Adaptive CP)**: [GitHub](https://github.com/J4ckLiu/SACP)
* **TensorFlow / Keras**: Deep learning frameworks utilized for modeling architectures like AlexNet, GFNet, and ViT-UNet.

---
_This repository is developed under the guidance of Dr. Mahesh Pal._

---

# 6 Bands Data - Directory & Kaggle Links

Below is the directory structure mirroring our local repository, with direct links to the notebooks on Kaggle:

* 📁 **baseline/**
    * 📓 [Model_training.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-baseline-k01) — `cuq6-baseline-k01` ✅ **COMPLETE** (Fresh weights & metrics verified)
    * 📓 [Model_uncertainty_comparison.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-uncertainty-k02) — `cuq6-uncertainty-k02` ✅ **COMPLETE** (Conformal prediction evaluated)
* 📁 **credit/**
    * 📓 [Model_training_credit.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-credit-k03) — `cuq6-credit-k03` ✅ **COMPLETE** (CreDE Evidential distillation verified)
* 📁 **data/**
* 📁 **ensemble/**
    * 📓 [Model_training_ensembles.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-ensemble-k04) — `cuq6-ensemble-k04` ⚠️ **CANCELLED** (not re-run in this pass)
    * 📓 [Model_uncertainty_CreDE.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-crede-k05) — `cuq6-crede-k05` ✅ **COMPLETE** (Credal Deep Ensemble evaluation)
* 📁 **multicp/**
    * 📓 [Model_training_multihead.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-multicp-k06) — `cuq6-multicp-k06` ⚠️ **CANCELLED** (not re-run in this pass)
    * 📓 [Model_uncertainty_multicp.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-multicp-uq-k07) — `cuq6-multicp-uq-k07` ❌ **ERROR** (depends on multicp-k06)
* 📁 **multicp_sacp/**
    * 📓 [Model_uncertainty_multicp_sacp.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-multicp-sacp-k08) — `cuq6-multicp-sacp-k08` ❌ **ERROR** (depends on multicp-k06)
* 📁 **sacp/**
    * 📓 [Model_sacp_comparison.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-sacp-k09) — `cuq6-sacp-k09` ✅ **COMPLETE** (Multi-window SACP evaluated)
* 📁 **mambahsi/**
    * 📓 [Model_training_mambahsi.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-mambahsi-k10) — `cuq6-mambahsi-k10` ✅ **COMPLETE** (Fresh 6-band SSM weights trained)
    * 📓 [Model_uncertainty_comparison_mambahsi.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-mambahsi-uq-k11) — `cuq6-mambahsi-uq-k11` ✅ **COMPLETE** (SSM conformal prediction evaluated)
* 📁 **dofa/**
    * 📓 [Model_training.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-dofa-k12) — `cuq6-dofa-k12` ✅ **COMPLETE** (Fresh 6-band DOFA/Hiera weights trained)
    * 📓 [Model_uncertainty_comparison.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-dofa-uq-k13) — `cuq6-dofa-uq-k13` ✅ **COMPLETE** (DOFA conformal prediction evaluated)
* 📁 **georsclip/**
    * 📓 [Model_training.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-georsclip-k14) — `cuq6-georsclip-k14` ✅ **COMPLETE** (Vision-language text-head architecture)
    * 📓 [Model_uncertainty_comparison.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-georsclip-uq-k15) — `cuq6-georsclip-uq-k15` ✅ **COMPLETE** (GeoRSCLIP conformal prediction evaluated)
* 📁 **spatialgcn/**
    * 📓 [Model_training.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-spatialgcn-k16) — `cuq6-spatialgcn-k16` ✅ **COMPLETE** (Spatial graph convolution architecture)
    * 📓 [Model_uncertainty_comparison.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-spatialgcn-uq-k17) — `cuq6-spatialgcn-uq-k17` ✅ **COMPLETE** (SpatialGCN conformal prediction evaluated)

	# Advanced Classification Uncertainty Quantification: Remote Sensing & Hyperspectral Analysis

	[![Python](https://img.shields.io/badge/Python-3.9%2B-blue?style=for-the-badge&logo=python&logoColor=white)](https://img.shields.io/badge/Python-3.9%2B-blue?style=for-the-badge&logo=python&logoColor=white)
	[![TensorFlow](https://img.shields.io/badge/TensorFlow-2.10%2B-orange?style=for-the-badge&logo=tensorflow&logoColor=white)](https://img.shields.io/badge/TensorFlow-2.10%2B-orange?style=for-the-badge&logo=tensorflow&logoColor=white)
	[![Uncertainty](https://img.shields.io/badge/Uncertainty-Conformal%20%7C%20Ensemble%20%7C%20Bayesian-green?style=for-the-badge)](https://img.shields.io/badge/Uncertainty-Conformal%20%7C%20Ensemble%20%7C%20Bayesian-green?style=for-the-badge)
	[![Status](https://img.shields.io/badge/Status-Active-success?style=for-the-badge)](https://img.shields.io/badge/Status-Active-success?style=for-the-badge)

	Welcome to the Classification Uncertainty Quantification (UQ) Analysis framework. This repository provides a comprehensive, modular pipeline for quantifying predictive uncertainty in complex classification tasks, specifically optimized for high-dimensional remote sensing and multispectral imagery.

	Beyond traditional point-wise accuracy, this framework implements state-of-the-art methods including Conformal Prediction, Credal Deep Ensembles, and Deep Adaptive Predictive Modeling, ensuring every classification decision is backed by a statistically valid measure of confidence.

	---

	## 📑 Table of Contents

	1. [📌 Project Overview](#-project-overview)
	2. [📂 Repository Structure](#-repository-structure)
	3. [📊 Dataset & Usage](#-dataset--usage)
	4. [🛠️ Workflow & Methodology](#-workflow--methodology)
	* [Method 1: Baseline Uncertainty](#method-1-baseline-uncertainty)
	* [Method 2: CREDIT (Calibration-Aware)](#method-2-credit-calibration-aware)
	* [Method 3: DAPM (Deep Adaptive Modeling)](#method-3-dapm-deep-adaptive-modeling)
	* [Method 4: Ensemble (CreDE)](#method-4-ensemble-crede)
	* [Method 5: MultiCP (Multi-head Conformal)](#method-5-multicp-multi-head-conformal)
	* [Method 6: SACP (Self-Adaptive Conformal)](#method-6-sacp-self-adaptive-conformal)
	* [Method 7: SCMCP (Spatial MultiCP)](#method-7-scmcp-spatial-multicp)
	* [Method 8: Focal Loss & CB Focal Loss](#method-8-focal-loss--cb-focal-loss)
	* [Method 9: EDL (Evidential Deep Learning)](#method-9-edl-evidential-deep-learning)
	* [Method 10: CDL (Credal Deep Learning)](#method-10-cdl-credal-deep-learning)
	* [Method 11: MambaHSI](#method-11-mambahsi)
	5. [🚀 Getting Started](#-getting-started)
	6. [📚 Resources & References](#-resources--references)
	7. [🌐 6 Bands Data - Directory & Kaggle Links](#-6-bands-data---directory--kaggle-links)

	---

	## 📌 Project Overview

	This framework addresses the critical need for reliability in machine learning models deployed for Earth Observation and spatial analysis. By integrating multiple UQ paradigms, it allows researchers to:

	* Identify Out-of-Distribution (OOD) Pixels: Detect areas where the model is likely to fail.
	* Calibrate Probabilistic Outputs: Ensure that a 90% confidence score actually corresponds to 90% accuracy.
	* Generate Valid Prediction Sets: Use Conformal Prediction to produce sets of classes that contain the true label with a user-specified probability (e.g., 95%).
	* Analyze Spatial Uncertainty: Visualize uncertainty maps across entire scenes (Hyperspectral/Multispectral).

	Key Features:
	* Deep Architectures: AlexNet CNN, GFNet (Global Filter Network), and ViT-UNet (Vision Transformer).
	* Modular Design: Each method is self-contained with its own training logic, detailed theory/summary logs in `.md` format, and result visualization.
	* Remote Sensing Ready: Built-in support for multispectral data structures.

	---

	## 📂 Repository Structure

	The project is organized into a modular architecture optimized for portability and experimental reproducibility. The main source code is organized within the `Classification/` directory:

	```text
	.
	├── Classification/ # 📂 Main Project Folder (Upload this to MyDrive/)
	│ ├── baseline/ # 📂 Standard UQ (MC Dropout, Temp Scaling)
	│ ├── credit/ # 📂 CREDIT: Calibration-aware training
	│ ├── dapm/ # 📂 DAPM: Deep Adaptive Predictive Modeling
	│ ├── data/ # 📊 Raw Datasets (data.csv, multispectral/)
	│ ├── dofa/ # 📂 DOFA: Dynamic Wavelength Tokenization
	│ ├── ensemble/ # 📂 CreDE: Credal Deep Ensembles
	│ ├── georsclip/ # 📂 GeoRSCLIP: Geospatial Vision-Language Foundation Model (New)
	│ ├── mambahsi/ # 📂 MambaHSI Architecture for classification
	│ ├── multicp/ # 📂 MultiCP: Multi-head Conformal Prediction
	│ ├── multicp_sacp/ # 📂 SCMCP: Spatial MultiCP
	│ ├── sacp/ # 📂 SACP: Self-Adaptive Conformal Prediction
	│ ├── spatialgcn/ # 📂 SpatialGCN: Spatial Graph Convolutional Networks (New)
	│ └── trials/ # 🧪 Experimental Methods
	│ ├── cb_focal_loss/
	│ ├── cdl/
	│ ├── conformal_reg/
	│ ├── edl/
	│ ├── edl_v2/
	│ └── focal_loss/
	├── examples/ # 📁 Example Dataset Suites (untouched)
	└── README.md # 📑 Documentation
	```

	---

	## 📊 Dataset & Usage

	This is a Template Pipeline.

	To use this repository with your own classification data:

	1. Prepare Data: Ensure your features and labels are in `.csv` or `.npy` format.
	2. Path Configuration:
	* Place your data in the `Classification/data/` directory.
	* Notebooks use relative paths (e.g., `../data/data.csv`) for compatibility with Google Colab and local environments.
	3. Model Selection: Choose from AlexNet, GFNet, or ViT-UNet by modifying the configuration in the respective notebooks.

	---

	## 🛠️ Workflow & Methodology

	> Note: Each method subfolder in `Classification/` includes a comprehensive `.md` file detailing the mathematical framework, implementation logic, and experimental logs.

	---

	### Method 1: Baseline Uncertainty
	Location: `Classification/baseline`
	Establishes the performance and uncertainty floor using standard Bayesian approximations and Conformal Prediction.
	* Techniques: Monte Carlo Dropout, Temperature Scaling, Conformal Prediction (SplitCP, CcCP, RC3P, ClusteredCP, RAPS).
	* Models: AlexNet-CNN, GFNet, ViT-UNet.
	* Trained Weights Artifacts: `AlexNet_CNN_best.keras`, `GFNet_best.keras`, `ViT_UNet_best.keras` (Input: 9×9×6, Output: 7 classes).

	#### 6-Band Baseline Performance (100 Epochs, Kaggle GPU)
	\| Model \| Test Accuracy \| Cohen's Kappa \| Macro F1 \| Weighted F1 \| Test Brier \| Test ECE (15-bin) \| Status \|
	\|:---\|:---:\|:---:\|:---:\|:---:\|:---:\|:---:\|:---:\|
	\| AlexNet_CNN \| 99.70% \| 0.9959 \| 0.9955 \| 0.9970 \| 0.0044 \| 0.0013 \| ✅ Verified \|
	\| GFNet \| 99.63% \| 0.9950 \| 0.9954 \| 0.9963 \| 0.0057 \| 0.0089 \| ✅ Verified \|
	\| ViT_UNet \| 99.28% \| 0.9902 \| 0.9904 \| 0.9928 \| 0.0136 \| 0.0388 \| ✅ Verified \|

	#### Conformal Prediction Evaluation Summary ($\alpha = 0.05$, Target Coverage = 95.0%)
	\| Method \| AlexNet Coverage \| GFNet Coverage \| ViT-UNet Coverage \| Avg Set Size \| Median Set Size \|
	\|:---\|:---:\|:---:\|:---:\|:---:\|:---:\|
	\| SplitCP \| 95.78% \| 94.99% \| 95.41% \| 0.95 - 0.96 \| 1.0 \|
	\| CcCP (Class-Conditional) \| 96.47% \| 96.19% \| 95.59% \| 0.96 \| 1.0 \|
	\| RC3P (Rank-Calibrated) \| 97.17% \| 96.57% \| 96.29% \| 0.96 - 0.97 \| 1.0 \|
	\| ClusteredCP (K=4) \| 96.52% \| 95.96% \| 95.36% \| 0.96 \| 1.0 \|
	\| RAPS \| 99.91% \| 99.77% \| 99.40% \| 1.00 \| 1.0 \|

	### Method 2: CREDIT
	Location: `Classification/credit`
	Confidence-Calibrated Robustness for Deep Image Classification.
	* Focus: Improving the alignment between model confidence and actual performance.
	* Output: Calibrated spatial uncertainty maps.

	### Method 3: DAPM
	Location: `Classification/dapm`
	Deep Adaptive Predictive Modeling for classification under distribution shift.
	* Architecture: Includes Encoder, Diffusion, and Multi-head decoders.
	* Result: High-resolution p-value distributions and uncertainty masks.

	### Method 4: Ensemble (CreDE)
	Location: `Classification/ensemble`
	Credal Deep Ensembles (CreDE) for robust uncertainty.
	* Process: Training multiple model instances to capture epistemic uncertainty.
	* Metrics: Credal entropy and variance-based measures.

	### Method 5: MultiCP
	Location: `Classification/multicp`
	Multi-head Conformal Prediction.
	* Goal: Distribution-free uncertainty sets with finite-sample validity.
	* Output: Performance measures across different significance levels ($\alpha$).

	### Method 6: SACP
	Location: `Classification/sacp`
	Self-Adaptive Conformal Prediction.
	* Advantage: Online calibration and adaptation across varying spatial windows (ws=3, 5, 7, 9).
	* Summary: Combined per-class coverage reports for all models.

	### Method 7: SCMCP (Spatial MultiCP)
	Location: `Classification/multicp_sacp`
	Spatial Multi-Head Conformal Prediction.
	* Core: Combines spatial probability smoothing with multi-head conformal intersection.
	* Feature: Produces tight, spatially-coherent uncertainty regions by intersecting calibrated sets from $K$ heads after local smoothing.

	### Method 8: Focal Loss & CB Focal Loss
	Location: `Classification/trials/focal_loss` & `Classification/trials/cb_focal_loss`
	* Focus: Addressing class imbalance using standard and Class-Balanced Focal Loss.
	* Output: Improved minority class prediction with respective uncertainty estimates.

	### Method 9: EDL (Evidential Deep Learning)
	Location: `Classification/trials/edl` & `Classification/trials/edl_v2`
	* Core: Uses Subjective Logic to place a Dirichlet distribution over class probabilities.
	* Goal: Directly quantifies epistemic and aleatoric uncertainty without sampling.

	### Method 10: CDL (Credal Deep Learning)
	Location: `Classification/trials/cdl`
	* Goal: Robust uncertainty quantification using imprecise probabilities and credal sets.

	### Method 11: MambaHSI
	Location: `Classification/mambahsi`
	* Architecture: State-of-the-art Mamba state-space models adapted for Hyperspectral Imagery (HSI).
	* Focus: Efficient long-range spatial-spectral dependencies modeling.

	---

	### Method 12: DOFA
	Location: `Classification/dofa`
	Dynamic Wavelength Tokenization.
	* Architecture: DOFA Spectral and DOFA Hiera Fusion for robust multispectral tokenization.
	* Focus: Analyzing prediction stability across continuous spectral channels.

	---

	### Method 13: GeoRSCLIP
	Location: `Classification/georsclip`
	Geospatial Vision-Language Foundation Model.
	* Architecture: Text-head fine-tuning on GeoRSCLIP representations.
	* Focus: Leveraging multimodal pre-training for remote sensing tasks.

	### Method 14: SpatialGCN
	Location: `Classification/spatialgcn`
	Spatial Graph Convolutional Networks.
	* Architecture: GCN processing on spatial grid topologies.
	* Focus: Capturing complex spatial dependencies in classification maps.

	---

	## 🚀 Getting Started (Colab-First)

	1. Clone the Repository:
	```bash
	git clone https://github.com/DaneshSelwal/Classification_Uncertainty_Quantification_Analysis
	```
	2. Upload to Google Drive:
	Upload the `Classification/` folder to your `MyDrive/` directory.
	3. Run in Colab:
	Navigate to any phase (e.g., `baseline/Model_training.ipynb`) and open with Google Colab.
	* Notebooks are pre-configured to mount `/content/drive` and find data in `/content/drive/MyDrive/Classification/data/`.
	4. Execution Order:
	Data Prep $\rightarrow$ Baseline Training $\rightarrow$ Advanced UQ Methods $\rightarrow$ Visualization.

	---

	## 📚 Resources & References

	This project leverages state-of-the-art research in Uncertainty Quantification and Remote Sensing. Below are the key resources and research papers utilized in this pipeline:

	### 📖 Research Papers

	* Weighted Aggregation of Conformity Scores: Weighted Aggregation of Conformity Scores for Classification. [ArXiv:2407.10230](https://arxiv.org/abs/2407.10230)
	* Uncertainty Sets for Image Classifiers: Uncertainty Sets for Image Classifiers using Conformal Prediction. [ArXiv:2009.14193](https://arxiv.org/abs/2009.14193)
	* Conformal Prediction via Label Ranking: Conformal Prediction for Deep Classifier via Label Ranking. [ArXiv:2310.06430](https://arxiv.org/abs/2310.06430)
	* Credal Ensemble Distillation: Credal Ensemble Distillation for Uncertainty Quantification. [ArXiv:2511.13766](https://arxiv.org/abs/2511.13766)
	* Calibration of Modern Neural Networks: On Calibration of Modern Neural Networks. [ICML 2017](https://arxiv.org/abs/1706.04599)
	* Deep Ensembles: Lakshminarayanan, B., et al. (2017). Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. [NeurIPS 2017](https://arxiv.org/abs/1612.01474)

	### 🛠️ Libraries & Frameworks

	* SACP (Self-Adaptive CP): [GitHub](https://github.com/J4ckLiu/SACP)
	* TensorFlow / Keras: Deep learning frameworks utilized for modeling architectures like AlexNet, GFNet, and ViT-UNet.

	---
	_This repository is developed under the guidance of Dr. Mahesh Pal._

	---

	# 6 Bands Data - Directory & Kaggle Links

	Below is the directory structure mirroring our local repository, with direct links to the notebooks on Kaggle:

	* 📁 baseline/
	* 📓 [Model_training.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-baseline-k01) — `cuq6-baseline-k01` ✅ COMPLETE (Fresh weights & metrics verified)
	* 📓 [Model_uncertainty_comparison.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-uncertainty-k02) — `cuq6-uncertainty-k02` ✅ COMPLETE (Conformal prediction evaluated)
	* 📁 credit/
	* 📓 [Model_training_credit.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-credit-k03) — `cuq6-credit-k03` ✅ COMPLETE (CreDE Evidential distillation verified)
	* 📁 data/
	* 📁 ensemble/
	* 📓 [Model_training_ensembles.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-ensemble-k04) — `cuq6-ensemble-k04` ⚠️ CANCELLED (not re-run in this pass)
	* 📓 [Model_uncertainty_CreDE.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-crede-k05) — `cuq6-crede-k05` ✅ COMPLETE (Credal Deep Ensemble evaluation)
	* 📁 multicp/
	* 📓 [Model_training_multihead.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-multicp-k06) — `cuq6-multicp-k06` ⚠️ CANCELLED (not re-run in this pass)
	* 📓 [Model_uncertainty_multicp.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-multicp-uq-k07) — `cuq6-multicp-uq-k07` ❌ ERROR (depends on multicp-k06)
	* 📁 multicp_sacp/
	* 📓 [Model_uncertainty_multicp_sacp.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-multicp-sacp-k08) — `cuq6-multicp-sacp-k08` ❌ ERROR (depends on multicp-k06)
	* 📁 sacp/
	* 📓 [Model_sacp_comparison.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-sacp-k09) — `cuq6-sacp-k09` ✅ COMPLETE (Multi-window SACP evaluated)
	* 📁 mambahsi/
	* 📓 [Model_training_mambahsi.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-mambahsi-k10) — `cuq6-mambahsi-k10` ✅ COMPLETE (Fresh 6-band SSM weights trained)
	* 📓 [Model_uncertainty_comparison_mambahsi.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-mambahsi-uq-k11) — `cuq6-mambahsi-uq-k11` ✅ COMPLETE (SSM conformal prediction evaluated)
	* 📁 dofa/
	* 📓 [Model_training.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-dofa-k12) — `cuq6-dofa-k12` ✅ COMPLETE (Fresh 6-band DOFA/Hiera weights trained)
	* 📓 [Model_uncertainty_comparison.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-dofa-uq-k13) — `cuq6-dofa-uq-k13` ✅ COMPLETE (DOFA conformal prediction evaluated)
	* 📁 georsclip/
	* 📓 [Model_training.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-georsclip-k14) — `cuq6-georsclip-k14` ✅ COMPLETE (Vision-language text-head architecture)
	* 📓 [Model_uncertainty_comparison.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-georsclip-uq-k15) — `cuq6-georsclip-uq-k15` ✅ COMPLETE (GeoRSCLIP conformal prediction evaluated)
	* 📁 spatialgcn/
	* 📓 [Model_training.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-spatialgcn-k16) — `cuq6-spatialgcn-k16` ✅ COMPLETE (Spatial graph convolution architecture)
	* 📓 [Model_uncertainty_comparison.ipynb](https://www.kaggle.com/code/daneshselwal/cuq6-spatialgcn-uq-k17) — `cuq6-spatialgcn-uq-k17` ✅ COMPLETE (SpatialGCN conformal prediction evaluated)