HDS / README.md
Chrizz67's picture
Update README.md
ea74012 verified
|
Raw History Blame Contribute Delete
8.13 kB
metadata
license: mit
author: Christopher Alberto Hamonangan
datasets:
  - BhinnekaAI/TSB-Fraud-Tabular
  - BhinnekaAI/MNT-Log-Traffic
  - BhinnekaAI/CLB-Patient-Screening
  - BhinnekaAI/IMC-Sensor-Telemetry
  - BhinnekaAI/ESCL-Logistics-Audit
tags:
  - Machine-Learning
  - Unsupervised
  - Algorithm
  - AI

Homeostatic Distribution Score (HDS)

An Unsupervised Multi-Context Anomaly Detection Framework

Author : Christopher Alberto Hamonangan

HDS (Homeostatic Distribution Score) is an experimental unsupervised anomaly detection framework designed to identify samples that deviate from a learned reference distribution through multi-context structural analysis.

The framework combines multiscale spatial distribution analysis, linear relational inconsistency, hierarchical neighborhood analysis, adaptive token fusion, and episodic memory retrieval into a unified anomaly scoring mechanism.

Key Features

  • Multiscale Distribution Analysis: Evaluates neighborhood sparsity using a hybrid Euclidean–Mahalanobis approach.
  • Linear Network: Measures global projection residuals and local relational deviations.
  • Tree Network: Uses KD-Tree neighborhood distances to assess spatial inconsistency.
  • Adaptive Token Fusion: Combines structural signals through sample-dependent weighting.
  • Episodic Memory: Retrieves similar reference samples to incorporate contextual information into anomaly scoring.

Model Variants

Variant Local Relational Engine Design Goal
HDS-Deep Local SVD-based analysis Capture local neighborhood structure and relational deviations
HDS-Fast Neighborhood mean-deviation Reduce the computational cost of local relational analysis

Both variants produce continuous anomaly scores for ranking potentially anomalous samples. Their relative performance depends on the dataset and evaluation conditions.

Benchmark Evaluation

HDS-Deep and HDS-Fast are evaluated against conventional anomaly detection methods across five tabular anomaly detection domains.

Evaluation Metrics

  • ROC-AUC: Measures the ability to rank anomalous samples above normal samples across classification thresholds.
  • PR-AUC: Measures the precision–recall trade-off and is particularly informative for imbalanced datasets.

Higher scores indicate better ranking performance.

Benchmark Datasets

The following links provide access to the datasets used in the evaluation. Replace each placeholder with the corresponding public dataset repository.

Dataset Repository
Credit Card Fraud TSB-Fraud-Tabular
Network Intrusion MNT-Log-Traffic
Patient Screening CLB-Patient-Screening
IoT Sensor Anomaly IMC-Sensor-Telemetry
Logistics Audit ESCL-Logistics-Audit

1. Credit Card Fraud Simulator

Method ROC-AUC PR-AUC
HDS-Deep 0.885 0.659
HDS-Fast 0.856 0.606
Isolation Forest 0.568 0.125
HBOS 0.487 0.101
KNN-Distance 0.762 0.286
LOF 0.773 0.304
One-Class SVM 0.616 0.173
Robust Covariance 0.372 0.076

2. Network Intrusion Logs

Method ROC-AUC PR-AUC
HDS-Deep 0.660 0.152
HDS-Fast 0.666 0.153
Isolation Forest 0.989 0.835
HBOS 1.000 0.990
KNN-Distance 0.824 0.226
LOF 0.337 0.069
One-Class SVM 0.849 0.259
Robust Covariance 0.379 0.074

3. Patient Biomarker Screening

Method ROC-AUC PR-AUC
HDS-Deep 0.902 0.756
HDS-Fast 0.903 0.763
Isolation Forest 0.742 0.312
HBOS 0.666 0.193
KNN-Distance 0.852 0.549
LOF 0.831 0.353
One-Class SVM 0.784 0.400
Robust Covariance 0.889 0.734

4. IoT Sensor Anomaly

Method ROC-AUC PR-AUC
HDS-Deep 0.805 0.370
HDS-Fast 0.822 0.398
Isolation Forest 0.946 0.736
HBOS 0.541 0.116
KNN-Distance 0.998 0.984
LOF 0.398 0.076
One-Class SVM 0.680 0.240
Robust Covariance 0.740 0.162

5. Supply Chain Inventory Audit

Method ROC-AUC PR-AUC
HDS-Deep 0.468 0.092
HDS-Fast 0.494 0.101
Isolation Forest 0.553 0.126
HBOS 0.552 0.132
KNN-Distance 0.549 0.135
LOF 0.541 0.124
One-Class SVM 0.551 0.133
Robust Covariance 0.547 0.134

HDS-Deep vs. HDS-Fast: Direct Comparison

The following table compares the two HDS variants across all five benchmark categories.

Dataset Deep ROC-AUC Fast ROC-AUC Deep PR-AUC Fast PR-AUC
Credit Card Fraud 0.885 0.856 0.659 0.606
Network Intrusion 0.660 0.666 0.152 0.153
Patient Biomarker 0.902 0.903 0.756 0.763
IoT Sensor 0.805 0.822 0.370 0.398
Supply Chain 0.468 0.494 0.092 0.101
Macro Average 0.744 0.748 0.406 0.404

Head-to-Head Summary

  • ROC-AUC: HDS-Fast achieves a macro average of 0.748, compared with 0.744 for HDS-Deep.
  • PR-AUC: HDS-Deep achieves a macro average of 0.406, compared with 0.404 for HDS-Fast.
  • Credit Card Fraud: HDS-Deep performs better on both metrics.
  • Other four datasets: HDS-Fast records slightly higher ROC-AUC and PR-AUC scores.

The aggregate differences are small. Neither variant demonstrates universal superiority across all datasets. Runtime and memory measurements are needed to determine whether HDS-Fast provides a meaningful computational advantage.

Evaluation Protocol

The reported benchmark uses train–test evaluation for each dataset.

The primary evaluation metrics are ROC-AUC and PR-AUC. HDS-Deep and HDS-Fast are compared with Isolation Forest, HBOS, KNN-Distance, LOF, One-Class SVM, and Robust Covariance.

For reproducible evaluation, the accompanying experiment code should specify:

  • Dataset versions and preprocessing procedures.
  • Train–test split strategy and random seeds.
  • Feature selection and scaling procedures.
  • Anomaly label definitions and score direction.
  • Implementation details and hyperparameters for each baseline.

All preprocessing and model fitting should be performed without using test labels. Test labels should be reserved for final metric calculation.

Intended Use

HDS is intended for research and experimentation in:

  • Unsupervised anomaly detection.
  • Multicontext structural scoring.
  • Distribution-based anomaly ranking.
  • Comparative evaluation of tabular anomaly detection methods.

HDS is an experimental research framework. Its performance varies across datasets, and the reported results should not be interpreted as evidence of universal superiority over established methods.

Technical Summary

Property Description
Framework Homeostatic Distribution Score (HDS)
Task Unsupervised anomaly detection
Input Tabular feature vectors
Output Continuous anomaly ranking scores
Variants HDS-Deep, HDS-Fast
Primary metrics ROC-AUC, PR-AUC
Benchmark domains Five
Baseline methods Isolation Forest, HBOS, KNN-Distance, LOF, One-Class SVM, Robust Covariance

Project Links

Research Status

HDS is an experimental framework for multi-context unsupervised anomaly detection. Further testing across random seeds, dataset variations, and controlled computational benchmarks can strengthen its reproducibility and robustness.

Contributions, feedback, and independent evaluations are welcome.