README.md

---
license: gpl-2.0
tags:
- anomaly-detection
- clip
- zero-shot
- few-shot
- industrial-inspection
- universal-anomaly-detection
pipeline_tag: image-segmentation
library_name: pytorch
datasets:
- MVTec-AD
- VisA
language:
- en
base_model:
- openai/clip-vit-large-patch14-336
---

# AdaptCLIP

Universal Visual Anomaly Detection model based on CLIP with learnable adapters.

## Model Description

AdaptCLIP is a universal (zero-shot and few-shot) anomaly detection framework that leverages CLIP's vision-language capabilities with lightweight learnable adapters for open-word industrial and medical anomaly detection.

## Model Variants

| Checkpoint | Training Dataset | Description |
|------------|------------------|-------------|
| `adaptclip_checkpoints/12_4_128_train_on_mvtec_3adapters_batch8/epoch_15.pth` | MVTec-AD | Trained on MVTec-AD dataset |
| `adaptclip_checkpoints/12_4_128_train_on_visa_3adapters_batch8/epoch_15.pth` | VisA | Trained on VisA dataset |

## Usage

```python
import os
import torch
from huggingface_hub import hf_hub_download

# Hugging Face Repository Configuration
REPO_ID = "csgaobb/AdaptCLIP"

def load_adaptclip_checkpoint(ckpt_relative_path: str, save_dir: str = "./adaptclip_checkpoints"):
    """
    Automatically downloads and loads a PyTorch checkpoint from the Hugging Face Hub.
    
    Args:
        ckpt_relative_path (str): Relative file path of the checkpoint inside the repo.
        save_dir (str): Local directory where checkpoints will be cached.

    Returns:
        dict/torch.nn.Module: Loaded PyTorch checkpoint object.
    """
    print(f"[*] Checking local cache or downloading checkpoint from Hub: {ckpt_relative_path}")
    
    # Automatically checks local cache; downloads from HF server if not found (triggers download count on HF)
    local_file_path = hf_hub_download(
        repo_id=REPO_ID,
        filename=ckpt_relative_path,
        local_dir=save_dir,
        local_dir_use_symlinks=False
    )
    
    print(f"[✓] Checkpoint ready at: {local_file_path}")
    
    # Load the PyTorch checkpoint
    device = "cuda" if torch.cuda.is_available() else "cpu"
    checkpoint = torch.load(local_file_path, map_location=device)
    return checkpoint

if __name__ == "__main__":
    # 1. Automatically download and load checkpoint trained on MVTec-AD
    mvtec_ckpt_path = "12_4_128_train_on_mvtec_3adapters_batch8/epoch_15.pth"
    mvtec_checkpoint = load_adaptclip_checkpoint(mvtec_ckpt_path)

    # 2. Automatically download and load checkpoint trained on VisA
    visa_ckpt_path = "12_4_128_train_on_visa_3adapters_batch8/epoch_15.pth"
    visa_checkpoint = load_adaptclip_checkpoint(visa_ckpt_path)
```

## Citation

If you find this model useful, please cite our work.

```shell
@inproceedings{adaptclip,
  title={AdaptCLIP: Adapting CLIP for Universal Visual Anomaly Detection},
  author={Gao, Bin-Bin and Zhou, Yue and Yan, Jiangtao and Cai, Yuezhi and Zhang, Weixi and Wang, Meng and Liu, Jun and Liu, Yong and Wang, Lei and Wang, Chengjie},
  booktitle={AAAI}
  year={2026}
}
```



## License

gpl-2.0