Upload folder using huggingface_hub

5187db5 verified 14 days ago

9.54 kB

---
license: mit
language:
  - en
library_name: tensorflow
tags:
  - tensorflow
  - keras
  - anomaly-detection
  - explainable-ai
  - vae-gan
  - computer-vision
  - industrial-ai
  - mvtec-ad
  - arxiv:2407.19951
---
# Explainable Anomaly Detection with VAE-GAN — TensorFlow Checkpoints

Official TensorFlow/Keras VAE-GAN model checkpoints associated with the research paper:

**Can I Trust My Anomaly Detection System? A Case Study Based on Explainable AI**

**Muhammad Rashid, Elvio Amparore, Enrico Ferrari, and Damiano Verda**

_2nd World Conference on eXplainable Artificial Intelligence (XAI 2024), Explainable Artificial Intelligence, Springer, pp. 243–254._

[Research Paper (Springer)](https://doi.org/10.1007/978-3-031-63803-9_13) | [arXiv](https://arxiv.org/abs/2407.19951) | [GitHub Implementation](https://github.com/rashidrao-pk/anomaly_detection_trust_case_study/tree/xai-2024-paper)

## Overview

This model collection provides pretrained TensorFlow VAE-GAN checkpoints for reconstruction-based visual anomaly detection and explainability experiments.

The research investigates whether anomaly detection systems make decisions for the correct visual reasons by combining:

- **VAE-GAN:** Reconstruction-based anomaly detection
- **LIME:** Local interpretable explanations
- **SHAP:** Feature attribution and explanation analysis
- **Jaccard similarity:** Quantitative comparison between explanations and ground-truth defect regions

The associated experiments investigate the MVTec AD Screw and Hazelnut categories.

## Model Architecture

The model architecture is defined in [`models.py`](https://github.com/rashidrao-pk/anomaly_detection_trust_case_study/blob/xai-2024-paper/models.py).

| Component         | Description                                                                           |
| ----------------- | ------------------------------------------------------------------------------------- |
| Framework         | TensorFlow / Keras                                                                    |
| Architecture      | Variational Autoencoder–GAN                                                           |
| Input             | 128 × 128 × 3 RGB                                                                     |
| Latent dimension  | 64                                                                                    |
| Encoder           | Convolutional network producing latent mean, log variance, and sampled representation |
| Decoder           | Transposed-convolution reconstruction network                                         |
| Discriminator     | Convolutional real/fake classifier                                                    |
| Output            | Reconstructed RGB image                                                               |
| Checkpoint format | TensorFlow weight checkpoints                                                         |

## Available Checkpoints

| MVTec AD Category | Training Checkpoint      | Components                                    |
| ----------------- | ------------------------ | --------------------------------------------- |
| Hazelnut          | `hazelnut_VAE_GAN_30000` | Encoder, Decoder, Discriminator, VAE, VAE-GAN |
| Screw             | `screw_VAE_GAN_30000`    | Encoder, Decoder, Discriminator, VAE, VAE-GAN |

Each model component has a matching TensorFlow `.index` and `.data-00000-of-00001` file.

The `30000` suffix is retained from the original checkpoint naming convention.

## Installation

Clone the official research implementation and install its dependencies:

```bash
git clone --branch xai-2024-paper \
  https://github.com/rashidrao-pk/anomaly_detection_trust_case_study.git
cd anomaly_detection_trust_case_study
pip install -r requirements.txt
pip install huggingface_hub
```

The original environment specifies TensorFlow 2.15.0 and Keras 2.15.0. This version combination is recommended for reproducing the saved checkpoints.

## Downloading the Model Weights

You can download the model files directly from Hugging Face using `snapshot_download`:

```python
from huggingface_hub import snapshot_download
model_dir = snapshot_download(
    repo_id="rashidrao/AD_MVTec_XAI_TensorFlow",
    allow_patterns=[
        "weights/hazelnut_VAE_GAN_30000/*"
    ]
)
print("Downloaded to:", model_dir)
```

Replace `hazelnut` with `screw` to download the alternative category.

## Loading the TensorFlow Weights

The original research implementation exposes model-building functions through `Model_VAE_GAN_functions` in `models.py`.

For reconstruction-based inference, load the pretrained encoder and decoder:

```python
from pathlib import Path
import tensorflow as tf
from huggingface_hub import snapshot_download
from models import Model_VAE_GAN_functions
CATEGORY = "hazelnut"
LATENT_DIM = 64
IMAGE_SIZE = [128, 128]
repo_dir = snapshot_download(
    repo_id="rashidrao/AD_MVTec_XAI_TensorFlow",
    allow_patterns=[
        f"weights/{CATEGORY}_VAE_GAN_30000/*"
    ]
)
checkpoint_dir = (
    Path(repo_dir)
    / "weights"
    / f"{CATEGORY}_VAE_GAN_30000"
)
encoder = Model_VAE_GAN_functions.get_encoder(
    latent_dim=LATENT_DIM,
    image_size=IMAGE_SIZE
)
decoder = Model_VAE_GAN_functions.get_decoder(
    latent_dim=LATENT_DIM
)
encoder.load_weights(
    str(checkpoint_dir / f"{CATEGORY}_VAE_GAN_30000_encoder")
)
decoder.load_weights(
    str(checkpoint_dir / f"{CATEGORY}_VAE_GAN_30000_decoder")
)
print("Encoder and decoder weights loaded.")
```

**Note:** Checkpoint prefixes must match the uploaded filenames exactly. The associated `.index` and `.data-*` files must remain together.

## Example: Reconstruct an Image

After loading the encoder and decoder, the following example demonstrates reconstruction.

```python
import numpy as np
import tensorflow as tf
from PIL import Image
image = Image.open("test_image.png").convert("RGB")
image = image.resize((128, 128))
x = np.asarray(image, dtype=np.float32) / 255.0
x = np.expand_dims(x, axis=0)
# Encoder returns mean, log variance and sampled latent vector
z_mean, z_log_var, z = encoder(x, training=False)
# Use mean for a deterministic reconstruction
reconstruction = decoder(z_mean, training=False)
# Pixel-wise absolute reconstruction error
anomaly_map = tf.reduce_mean(
    tf.abs(x - reconstruction),
    axis=-1
)
anomaly_score = tf.reduce_mean(anomaly_map)
print("Anomaly score:", float(anomaly_score.numpy()))
reconstructed_image = (
    reconstruction.numpy()[0] * 255
).clip(0, 255).astype(np.uint8)
Image.fromarray(reconstructed_image).save(
    "reconstruction.png"
)
```

This example demonstrates reconstruction and a simple reconstruction-error score. For exact replication of the paper's reported anomaly detection and explainability results, refer to the original notebooks and scoring/evaluation procedures.

## Explainable AI Evaluation

The accompanying research implementation provides experiments using LIME and SHAP to analyze anomaly detections and investigate whether explanation regions correspond to actual defects.

Original experiment notebooks:

- [Screw Experiment](https://github.com/rashidrao-pk/anomaly_detection_trust_case_study/blob/xai-2024-paper/AD_VAE_GAN_SCREW.ipynb)
- [Hazelnut Experiment](https://github.com/rashidrao-pk/anomaly_detection_trust_case_study/blob/xai-2024-paper/AD_VAE_GAN_HAZELNUT.ipynb)

The explanations can be compared against ground-truth anomaly masks through the quantitative evaluation described in the paper.

## Intended Use

These checkpoints are intended for:

- Explainable visual anomaly detection research
- Industrial anomaly localization
- Reconstruction-based anomaly detection
- LIME and SHAP experiments
- Trustworthiness evaluation of anomaly detection systems
- Reproducibility and comparative research

## Limitations

- The original experiments focus on Screw and Hazelnut from MVTec AD.
- Images are processed at 128 × 128 resolution.
- Anomaly scores depend on preprocessing and scoring strategies.
- Explainability methods may require substantial computation.
- The checkpoints are research artifacts and are not validated for safety-critical deployment.
- Compatibility with newer TensorFlow/Keras releases is not guaranteed.

## Citation

If you use these TensorFlow checkpoints, the architecture, or the associated explainability methodology, please cite:

```bibtex
@InProceedings{10.1007/978-3-031-63803-9_13,
  author    = {Rashid, Muhammad and Amparore, Elvio
               and Ferrari, Enrico and Verda, Damiano},
  editor    = {Longo, Luca and Lapuschkin, Sebastian
               and Seifert, Christin},
  title     = {Can I Trust My Anomaly Detection System?
               A Case Study Based on Explainable AI},
  booktitle = {Explainable Artificial Intelligence},
  year      = {2024},
  publisher = {Springer Nature Switzerland},
  address   = {Cham},
  pages     = {243--254},
  doi       = {10.1007/978-3-031-63803-9_13}
}
```

## Resources

- **Model weights:** https://huggingface.co/rashidrao/AD_MVTec_XAI_TensorFlow
- **GitHub implementation:** https://github.com/rashidrao-pk/anomaly_detection_trust_case_study/tree/xai-2024-paper
- **Springer publication:** https://doi.org/10.1007/978-3-031-63803-9_13
- **Hugging Face paper:** https://huggingface.co/papers/2407.19951

## Authors

**Muhammad Rashid, Elvio Amparore, Enrico Ferrari, Damiano Verda**

University of Turin and RuleX Innovation Labs, Italy.

## License

MIT License, consistent with the accompanying research code repository.

	---
	license: mit
	language:
	- en
	library_name: tensorflow
	tags:
	- tensorflow
	- keras
	- anomaly-detection
	- explainable-ai
	- vae-gan
	- computer-vision
	- industrial-ai
	- mvtec-ad
	- arxiv:2407.19951
	---

	# Explainable Anomaly Detection with VAE-GAN — TensorFlow Checkpoints

	Official TensorFlow/Keras VAE-GAN model checkpoints associated with the research paper:

	Can I Trust My Anomaly Detection System? A Case Study Based on Explainable AI

	Muhammad Rashid, Elvio Amparore, Enrico Ferrari, and Damiano Verda

	_2nd World Conference on eXplainable Artificial Intelligence (XAI 2024), Explainable Artificial Intelligence, Springer, pp. 243–254._

	[Research Paper (Springer)](https://doi.org/10.1007/978-3-031-63803-9_13) \| [arXiv](https://arxiv.org/abs/2407.19951) \| [GitHub Implementation](https://github.com/rashidrao-pk/anomaly_detection_trust_case_study/tree/xai-2024-paper)

	## Overview

	This model collection provides pretrained TensorFlow VAE-GAN checkpoints for reconstruction-based visual anomaly detection and explainability experiments.

	The research investigates whether anomaly detection systems make decisions for the correct visual reasons by combining:

	- VAE-GAN: Reconstruction-based anomaly detection
	- LIME: Local interpretable explanations
	- SHAP: Feature attribution and explanation analysis
	- Jaccard similarity: Quantitative comparison between explanations and ground-truth defect regions

	The associated experiments investigate the MVTec AD Screw and Hazelnut categories.

	## Model Architecture

	The model architecture is defined in [`models.py`](https://github.com/rashidrao-pk/anomaly_detection_trust_case_study/blob/xai-2024-paper/models.py).

	\| Component \| Description \|
	\| ----------------- \| ------------------------------------------------------------------------------------- \|
	\| Framework \| TensorFlow / Keras \|
	\| Architecture \| Variational Autoencoder–GAN \|
	\| Input \| 128 × 128 × 3 RGB \|
	\| Latent dimension \| 64 \|
	\| Encoder \| Convolutional network producing latent mean, log variance, and sampled representation \|
	\| Decoder \| Transposed-convolution reconstruction network \|
	\| Discriminator \| Convolutional real/fake classifier \|
	\| Output \| Reconstructed RGB image \|
	\| Checkpoint format \| TensorFlow weight checkpoints \|

	## Available Checkpoints

	\| MVTec AD Category \| Training Checkpoint \| Components \|
	\| ----------------- \| ------------------------ \| --------------------------------------------- \|
	\| Hazelnut \| `hazelnut_VAE_GAN_30000` \| Encoder, Decoder, Discriminator, VAE, VAE-GAN \|
	\| Screw \| `screw_VAE_GAN_30000` \| Encoder, Decoder, Discriminator, VAE, VAE-GAN \|

	Each model component has a matching TensorFlow `.index` and `.data-00000-of-00001` file.

	The `30000` suffix is retained from the original checkpoint naming convention.

	## Installation

	Clone the official research implementation and install its dependencies:

	```bash
	git clone --branch xai-2024-paper \
	https://github.com/rashidrao-pk/anomaly_detection_trust_case_study.git

	cd anomaly_detection_trust_case_study

	pip install -r requirements.txt
	pip install huggingface_hub
	```

	The original environment specifies TensorFlow 2.15.0 and Keras 2.15.0. This version combination is recommended for reproducing the saved checkpoints.

	## Downloading the Model Weights

	You can download the model files directly from Hugging Face using `snapshot_download`:

	```python
	from huggingface_hub import snapshot_download

	model_dir = snapshot_download(
	repo_id="rashidrao/AD_MVTec_XAI_TensorFlow",
	allow_patterns=[
	"weights/hazelnut_VAE_GAN_30000/*"
	]
	)

	print("Downloaded to:", model_dir)
	```

	Replace `hazelnut` with `screw` to download the alternative category.

	## Loading the TensorFlow Weights

	The original research implementation exposes model-building functions through `Model_VAE_GAN_functions` in `models.py`.

	For reconstruction-based inference, load the pretrained encoder and decoder:

	```python
	from pathlib import Path

	import tensorflow as tf
	from huggingface_hub import snapshot_download
	from models import Model_VAE_GAN_functions

	CATEGORY = "hazelnut"
	LATENT_DIM = 64
	IMAGE_SIZE = [128, 128]

	repo_dir = snapshot_download(
	repo_id="rashidrao/AD_MVTec_XAI_TensorFlow",
	allow_patterns=[
	f"weights/{CATEGORY}_VAE_GAN_30000/*"
	]
	)

	checkpoint_dir = (
	Path(repo_dir)
	/ "weights"
	/ f"{CATEGORY}_VAE_GAN_30000"
	)

	encoder = Model_VAE_GAN_functions.get_encoder(
	latent_dim=LATENT_DIM,
	image_size=IMAGE_SIZE
	)

	decoder = Model_VAE_GAN_functions.get_decoder(
	latent_dim=LATENT_DIM
	)

	encoder.load_weights(
	str(checkpoint_dir / f"{CATEGORY}_VAE_GAN_30000_encoder")
	)

	decoder.load_weights(
	str(checkpoint_dir / f"{CATEGORY}_VAE_GAN_30000_decoder")
	)

	print("Encoder and decoder weights loaded.")
	```

	Note: Checkpoint prefixes must match the uploaded filenames exactly. The associated `.index` and `.data-*` files must remain together.

	## Example: Reconstruct an Image

	After loading the encoder and decoder, the following example demonstrates reconstruction.

	```python
	import numpy as np
	import tensorflow as tf
	from PIL import Image

	image = Image.open("test_image.png").convert("RGB")
	image = image.resize((128, 128))

	x = np.asarray(image, dtype=np.float32) / 255.0
	x = np.expand_dims(x, axis=0)

	# Encoder returns mean, log variance and sampled latent vector
	z_mean, z_log_var, z = encoder(x, training=False)

	# Use mean for a deterministic reconstruction
	reconstruction = decoder(z_mean, training=False)

	# Pixel-wise absolute reconstruction error
	anomaly_map = tf.reduce_mean(
	tf.abs(x - reconstruction),
	axis=-1
	)

	anomaly_score = tf.reduce_mean(anomaly_map)

	print("Anomaly score:", float(anomaly_score.numpy()))

	reconstructed_image = (
	reconstruction.numpy()[0] * 255
	).clip(0, 255).astype(np.uint8)

	Image.fromarray(reconstructed_image).save(
	"reconstruction.png"
	)
	```

	This example demonstrates reconstruction and a simple reconstruction-error score. For exact replication of the paper's reported anomaly detection and explainability results, refer to the original notebooks and scoring/evaluation procedures.

	## Explainable AI Evaluation

	The accompanying research implementation provides experiments using LIME and SHAP to analyze anomaly detections and investigate whether explanation regions correspond to actual defects.

	Original experiment notebooks:

	- [Screw Experiment](https://github.com/rashidrao-pk/anomaly_detection_trust_case_study/blob/xai-2024-paper/AD_VAE_GAN_SCREW.ipynb)
	- [Hazelnut Experiment](https://github.com/rashidrao-pk/anomaly_detection_trust_case_study/blob/xai-2024-paper/AD_VAE_GAN_HAZELNUT.ipynb)

	The explanations can be compared against ground-truth anomaly masks through the quantitative evaluation described in the paper.

	## Intended Use

	These checkpoints are intended for:

	- Explainable visual anomaly detection research
	- Industrial anomaly localization
	- Reconstruction-based anomaly detection
	- LIME and SHAP experiments
	- Trustworthiness evaluation of anomaly detection systems
	- Reproducibility and comparative research

	## Limitations

	- The original experiments focus on Screw and Hazelnut from MVTec AD.
	- Images are processed at 128 × 128 resolution.
	- Anomaly scores depend on preprocessing and scoring strategies.
	- Explainability methods may require substantial computation.
	- The checkpoints are research artifacts and are not validated for safety-critical deployment.
	- Compatibility with newer TensorFlow/Keras releases is not guaranteed.

	## Citation

	If you use these TensorFlow checkpoints, the architecture, or the associated explainability methodology, please cite:

	```bibtex
	@InProceedings{10.1007/978-3-031-63803-9_13,
	author = {Rashid, Muhammad and Amparore, Elvio
	and Ferrari, Enrico and Verda, Damiano},
	editor = {Longo, Luca and Lapuschkin, Sebastian
	and Seifert, Christin},
	title = {Can I Trust My Anomaly Detection System?
	A Case Study Based on Explainable AI},
	booktitle = {Explainable Artificial Intelligence},
	year = {2024},
	publisher = {Springer Nature Switzerland},
	address = {Cham},
	pages = {243--254},
	doi = {10.1007/978-3-031-63803-9_13}
	}
	```

	## Resources

	- Model weights: https://huggingface.co/rashidrao/AD_MVTec_XAI_TensorFlow
	- GitHub implementation: https://github.com/rashidrao-pk/anomaly_detection_trust_case_study/tree/xai-2024-paper
	- Springer publication: https://doi.org/10.1007/978-3-031-63803-9_13
	- Hugging Face paper: https://huggingface.co/papers/2407.19951

	## Authors

	Muhammad Rashid, Elvio Amparore, Enrico Ferrari, Damiano Verda

	University of Turin and RuleX Innovation Labs, Italy.

	## License

	MIT License, consistent with the accompanying research code repository.