AdaptCLIP

Universal Visual Anomaly Detection model based on CLIP with learnable adapters.

Model Description

AdaptCLIP is a universal (zero-shot and few-shot) anomaly detection framework that leverages CLIP's vision-language capabilities with lightweight learnable adapters for open-word industrial and medical anomaly detection.

Model Variants

Checkpoint	Training Dataset	Description
adaptclip_checkpoints/12_4_128_train_on_mvtec_3adapters_batch8/epoch_15.pth	MVTec-AD	Trained on MVTec-AD dataset
adaptclip_checkpoints/12_4_128_train_on_visa_3adapters_batch8/epoch_15.pth	VisA	Trained on VisA dataset

Usage

import os
import torch
from huggingface_hub import hf_hub_download

# Hugging Face Repository Configuration
REPO_ID = "csgaobb/AdaptCLIP"

def load_adaptclip_checkpoint(ckpt_relative_path: str, save_dir: str = "./adaptclip_checkpoints"):
    """
    Automatically downloads and loads a PyTorch checkpoint from the Hugging Face Hub.
    
    Args:
        ckpt_relative_path (str): Relative file path of the checkpoint inside the repo.
        save_dir (str): Local directory where checkpoints will be cached.

    Returns:
        dict/torch.nn.Module: Loaded PyTorch checkpoint object.
    """
    print(f"[*] Checking local cache or downloading checkpoint from Hub: {ckpt_relative_path}")
    
    # Automatically checks local cache; downloads from HF server if not found (triggers download count on HF)
    local_file_path = hf_hub_download(
        repo_id=REPO_ID,
        filename=ckpt_relative_path,
        local_dir=save_dir,
        local_dir_use_symlinks=False
    )
    
    print(f"[✓] Checkpoint ready at: {local_file_path}")
    
    # Load the PyTorch checkpoint
    device = "cuda" if torch.cuda.is_available() else "cpu"
    checkpoint = torch.load(local_file_path, map_location=device)
    return checkpoint

if __name__ == "__main__":
    # 1. Automatically download and load checkpoint trained on MVTec-AD
    mvtec_ckpt_path = "12_4_128_train_on_mvtec_3adapters_batch8/epoch_15.pth"
    mvtec_checkpoint = load_adaptclip_checkpoint(mvtec_ckpt_path)

    # 2. Automatically download and load checkpoint trained on VisA
    visa_ckpt_path = "12_4_128_train_on_visa_3adapters_batch8/epoch_15.pth"
    visa_checkpoint = load_adaptclip_checkpoint(visa_ckpt_path)

Citation

If you find this model useful, please cite our work.

@inproceedings{adaptclip,
  title={AdaptCLIP: Adapting CLIP for Universal Visual Anomaly Detection},
  author={Gao, Bin-Bin and Zhou, Yue and Yan, Jiangtao and Cai, Yuezhi and Zhang, Weixi and Wang, Meng and Liu, Jun and Liu, Yong and Wang, Lei and Wang, Chengjie},
  booktitle={AAAI}
  year={2026}
}

License

gpl-2.0