upload

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+.pth filter=lfs diff=lfs merge=lfs -text
+.pt filter=lfs diff=lfs merge=lfs -text

DEPLOYMENT.md

@@ -0,0 +1,124 @@
+# Hugging Face Spaces Deployment Guide
+
+## Prerequisites
+
+1. A Hugging Face account
+2. Git LFS installed locally: `git lfs install`
+3. Model weights downloaded to the correct directories
+
+## Deployment Steps
+
+### 1. Prepare Model Weights
+
+You have two options:
+
+#### Option A: Upload weights via Git LFS (Recommended for public spaces)
+
+```bash
+# Initialize Git LFS
+git lfs install
+
+# Track large files
+git lfs track "*.pt"
+git lfs track "*.pth"
+git lfs track "*.pkl"
+
+# Add weights
+git add .gitattributes
+git add detectors/*/checkpoint/pretrained/weights/best.pt
+git add detectors/P2G/src/utils/classes.pkl
+git commit -m "Add model weights"
+```
+
+#### Option B: Configure automatic download
+
+1. Upload your model weights to Google Drive or another host
+2. Update `download_weights.py` with the correct URLs
+3. Weights will download automatically when the Space starts
+
+### 2. Create Hugging Face Space
+
+1. Go to https://huggingface.co/spaces
+2. Click "Create new Space"
+3. Choose:
+   - **Name**: deepfake-detection-library (or your preferred name)
+   - ** SDK**: Gradio
+   - **License**: MIT
+   - **Hardware**: CPU Basic (free) or upgrade to GPU if needed
+
+### 3. Push to Hugging Face
+
+```bash
+# Add HF remote (replace YOUR_USERNAME and SPACE_NAME)
+git remote add hf https://huggingface.co/spaces/YOUR_USERNAME/SPACE_NAME
+
+# Rename README for HF
+mv README.md README_github.md
+mv README_HF.md README.md
+
+# Push to Hugging Face
+git add .
+git commit -m "Initial commit for HF Spaces"
+git push hf main
+```
+
+### 4. Configure Space
+
+In your Space settings on Hugging Face:
+
+- **Hardware**: Start with CPU Basic (free), upgrade to GPU if needed
+- **Secrets**: Add any API keys if needed (none required currently)
+- **Variables**: No special environment variables needed
+
+### 5. Verify Deployment
+
+1. Wait for the Space to build (may take 5-10 minutes)
+2. Test each detector with sample images
+3. Check logs for any errors
+
+## File Size Considerations
+
+- **Git LFS** is required for files >10MB
+- Each model weight file (~100-500MB) will be stored via LFS
+- Free HF Spaces have storage limits; consider:
+  - Upgrading to Pro for more storage
+  - Using automatic download instead of uploading weights
+
+## Troubleshooting
+
+### Space fails to build
+
+- Check `requirements.txt` for incompatible versions
+- Review build logs in the Space interface
+- Ensure all dependencies are listed
+
+### Weights not loading
+
+- Verify Git LFS tracked the files: `git lfs ls-files`
+- Check file sizes: LFS pointer files are ~130 bytes
+- Update `download_weights.py` if using automatic download
+
+### Out of memory errors
+
+- Upgrade to GPU hardware (T4 small recommended)
+- Reduce batch size or model size if possible
+- Use CPU inference for deployment (already configured)
+
+## Cost Optimization
+
+- **CPU Basic** (free): Works but slower
+- **CPU Upgrade** ($0.03/hour): Faster inference
+- **T4 Small GPU** ($0.60/hour):  Needed for real-time performance
+
+## Maintenance
+
+- Monitor Space usage in HF dashboard
+- Update models by pushing new weights via Git LFS
+- Check Gradio version compatibility: `pip list | grep gradio`
+
+## Support
+
+For issues specific to this deployment, check:
+- [Gradio Documentation](https://gradio.app/docs/)
+- [HF Spaces Documentation](https://huggingface.co/docs/hub/spaces)
+- [GitHub Repository](https://github.com/truebees-ai/Image-Deepfake-Detectors-Public-Library)

README.md

@@ -1,12 +1,46 @@
 ---
-title: DeepShield Detectors
-emoji: 🏆
-colorFrom: gray
+title: Deepfake Detection Space
+emoji: 🔍
+colorFrom: red
 colorTo: yellow
 sdk: gradio
-sdk_version: 6.6.0
+sdk_version: 5.49.1
 app_file: app.py
 pinned: false
+license: mit
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Deepfake Detection Space
+
+This Space provides a unified interface to test multiple state-of-the-art deepfake detection models on your images.
+
+## Available Detectors
+
+- **R50_TF** - ResNet-50 based detector trained on TrueFake dataset
+- **R50_nodown** - ResNet-50 without downsampling operations
+- **CLIP-D** - CLIP-based deepfake detector
+- **P2G** - Prompt2Guard: Conditioned prompt-optimization for continual deepfake detection
+- **NPR** - Neural Posterior Regularization
+
+## Usage
+
+1. Upload an image
+2. Select a detector from the dropdown
+3. Click "Detect" to get the prediction
+
+The detector will return:
+- **Prediction**: Real or Fake
+- **Confidence**: Model confidence score (0-1)
+- **Elapsed Time**: Processing time
+
+## Models
+
+All models have been pretrained on images generated with StyleGAN2 and StableDiffusionXL, and real images from the FFHQ Dataset and the FORLAB Dataset.
+
+## References
+
+For more information about the implementation and benchmarking, visit the [GitHub repository](https://github.com/truebees-ai/Image-Deepfake-Detectors-Public-Library).
+
+## Note
+
+⚠️ Due to file size limitations, model weights need to be downloaded automatically on first use. This may take a few moments.

README_HF.md

@@ -0,0 +1,46 @@
+---
+title: Deepfake Detection Library
+emoji: 🔍
+colorFrom: red
+colorTo: orange
+sdk: gradio
+sdk_version: 5.49.1
+app_file: app.py
+pinned: false
+license: mit
+---
+
+# Deepfake Detection Library
+
+This Space provides a unified interface to test multiple state-of-the-art deepfake detection models on your images.
+
+## Available Detectors
+
+- **R50_TF** - ResNet-50 based detector trained on TrueFake dataset
+- **R50_nodown** - ResNet-50 without downsampling operations
+- **CLIP-D** - CLIP-based deepfake detector
+- **P2G** - Prompt2Guard: Conditioned prompt-optimization for continual deepfake detection
+- **NPR** - Neural Posterior Regularization
+
+## Usage
+
+1. Upload an image
+2. Select a detector from the dropdown
+3. Click "Detect" to get the prediction
+
+The detector will return:
+- **Prediction**: Real or Fake
+- **Confidence**: Model confidence score (0-1)
+- **Elapsed Time**: Processing time
+
+## Models
+
+All models have been pretrained on images generated with StyleGAN2 and StableDiffusionXL, and real images from the FFHQ Dataset and the FORLAB Dataset.
+
+## References
+
+For more information about the implementation and benchmarking, visit the [GitHub repository](https://github.com/truebees-ai/Image-Deepfake-Detectors-Public-Library).
+
+## Note
+
+⚠️ Due to file size limitations, model weights need to be downloaded automatically on first use. This may take a few moments.

app.py

@@ -0,0 +1,385 @@
+# Deepfake Detection Gradio App - v1.1
+import gradio as gr
+import os
+import sys
+import json
+import argparse
+from types import SimpleNamespace
+from PIL import Image
+import matplotlib.pyplot as plt
+import io
+import numpy as np
+
+# Try to import detector - if this fails, we'll show an error in the UI
+try:
+    from support.detect import run_detect
+    DETECTOR_AVAILABLE = True
+    IMPORT_ERROR = None
+except Exception as e:
+    DETECTOR_AVAILABLE = False
+    IMPORT_ERROR = str(e)
+    print(f"Warning: Could not import detector: {e}")
+    # Create a dummy function
+    def run_detect(args):
+        raise ImportError(f"Detector not available: {IMPORT_ERROR}")
+
+# Download weights on first run (for HF Spaces)
+if os.environ.get("SPACE_ID"):
+    try:
+        from download_weights import download_all_weights
+        download_all_weights()
+    except Exception as e:
+        print(f"Warning: Could not download weights: {e}")
+
+# Available detectors based on launcher.py
+DETECTORS = ['ALL', 'R50_TF', 'R50_nodown', 'CLIP-D', 'P2G', 'NPR']
+
+DETECTOR_WEIGHTS = {
+    'CLIP-D': 0.30,
+    'R50_TF': 0.25,
+    'R50_nodown': 0.20,
+    'P2G': 0.15,
+    'NPR': 0.10
+}
+
+def process_image(image_path):
+    """
+    Check if image is larger than 1024x1024 and central crop it if necessary.
+    Returns the path to the processed image (or original if no change).
+    """
+    try:
+        with Image.open(image_path) as img:
+            width, height = img.size
+            
+            # Check if both dimensions are larger than 1024
+            if width > 1024 and height > 1024:
+                print(f"Image size {width}x{height} exceeds 1024x1024. Performing central crop.")
+                
+                # Calculate crop box
+                left = (width - 1024) / 2
+                top = (height - 1024) / 2
+                right = (width + 1024) / 2
+                bottom = (height + 1024) / 2
+                
+                # Crop
+                img_cropped = img.crop((left, top, right, bottom))
+                
+                # Save to new path
+                directory, filename = os.path.split(image_path)
+                name, ext = os.path.splitext(filename)
+                new_filename = f"{name}_cropped{ext}"
+                new_path = os.path.join(directory, new_filename)
+                
+                img_cropped.save(new_path)
+                return new_path
+                
+            return image_path
+    except Exception as e:
+        print(f"Error processing image: {e}")
+        return image_path
+
+def run_single_detection(image_path, detector_name):
+    output_path = f"temp_result_{detector_name}.json"
+    # Mock args object
+    args = SimpleNamespace(
+        image=image_path,
+        detector=detector_name,
+        config_dir='configs',
+        output=output_path,
+        weights='pretrained', # Use default/pretrained
+        device='cpu', # Force CPU
+        dry_run=False,
+        verbose=False
+    )
+    
+    try:
+        run_detect(args)
+        
+        if os.path.exists(output_path):
+            with open(output_path, 'r') as f:
+                result = json.load(f)
+            os.remove(output_path)
+            return result
+        return None
+    except Exception as e:
+        if os.path.exists(output_path):
+            try:
+                os.remove(output_path)
+            except:
+                pass
+        print(f"Error running {detector_name}: {e}")
+        return None
+
+def predict(image_path, detector_name):
+    # Check if detector is available
+    if not DETECTOR_AVAILABLE:
+        return json.dumps({
+            "error": "Detector module not available",
+            "details": IMPORT_ERROR,
+            "message": "The detection system could not be initialized. Please check the logs."
+        }, indent=2), None
+    
+    if not image_path:
+        return json.dumps({"error": "Please upload an image."}, indent=2), None
+    
+    # Process image (central crop if too large)
+    processed_path = image_path
+    try:
+        processed_path = process_image(image_path)
+    except Exception as e:
+        print(f"Warning: Image processing failed: {e}")
+        # Continue with original image if processing fails
+    
+    try:
+        if detector_name == 'ALL':
+            results = []
+            # Filter out 'ALL' from detectors list
+            real_detectors = [d for d in DETECTORS if d != 'ALL']
+            
+            for det in real_detectors:
+                res = run_single_detection(processed_path, det)
+                if res:
+                    results.append((det, res))
+            
+            if not results:
+                return "Error: No results obtained from detectors.", None
+            
+            votes_real = 0.0
+            votes_fake = 0.0
+            total_weight_used = 0.0
+            confidences = []
+            labels = []
+            colors = []
+            
+            for det, res in results:
+                pred = res.get('prediction', 'Unknown')
+                raw_conf = res.get('confidence', 0.0)
+                
+                # Calculate display confidence (confidence of the prediction)
+                if pred == 'fake':
+                    score = raw_conf
+                    color = 'red'
+                else:
+                    score = 1 - raw_conf
+                    color = 'green'
+                
+                labels.append(det)
+                confidences.append(score)
+                colors.append(color)
+                
+                # Weighted Voting logic
+                # Only count vote if confidence > 0.6
+                if score > 0.6:
+                    weight = DETECTOR_WEIGHTS.get(det, 0.0)
+                    if pred == 'fake':
+                        votes_fake += weight * score
+                        total_weight_used += weight
+                    elif pred == 'real':
+                        votes_real += weight * score
+                        total_weight_used += weight
+            
+            # Majority Voting
+            if votes_real > votes_fake:
+                verdict = "REAL"
+            elif votes_fake > votes_real:
+                verdict = "FAKE"
+            else:
+                verdict = "UNCERTAIN"
+            
+            # Calculate weighted average confidence
+            if total_weight_used > 0:
+                weighted_conf = (votes_real + votes_fake) / total_weight_used
+            else:
+                weighted_conf = 0.0
+            
+            # Explanation
+            if verdict == "REAL":
+                explanation = f"Considering the results obtained by all models (weighted by their historical performance), the analyzed image results, with a weighted confidence of {weighted_conf:.4f}, not produced by a generative AI."
+            elif verdict == "FAKE":
+                explanation = f"Considering the results obtained by all models (weighted by their historical performance), the analyzed image results, with a weighted confidence of {weighted_conf:.4f}, produced by a generative AI."
+            else:
+                explanation = f"The result is uncertain. The detectors produced unconsistent results. The weighted confidence is {weighted_conf:.4f}."
+            
+            # Plotting
+            fig, ax = plt.subplots(figsize=(10, 5))
+            bars = ax.bar(labels, confidences, color=colors)
+            ax.set_ylim(0, 1.05)
+            ax.set_ylabel('Confidence')
+            ax.set_title('Detector Confidence Scores')
+            ax.axhline(y=0.6, color='gray', linestyle='--', alpha=0.5, label='Vote Threshold (0.6)')
+            
+            # Add custom legend for colors
+            from matplotlib.patches import Patch
+            legend_elements = [
+                Patch(facecolor='green', label='Real'),
+                Patch(facecolor='red', label='Fake'),
+                ax.lines[0] # The threshold line
+            ]
+            ax.legend(handles=legend_elements)
+            
+            # Add value labels
+            for bar in bars:
+                height = bar.get_height()
+                ax.text(bar.get_x() + bar.get_width()/2., height,
+                        f'{height:.2f}',
+                        ha='center', va='bottom')
+            
+            plt.tight_layout()
+            return explanation, fig
+            
+        else:
+            # Single Detector
+            res = run_single_detection(processed_path, detector_name)
+            
+            if res:
+                prediction = res.get('prediction', 'Unknown')
+                confidence = res.get('confidence', 0.0)
+                elapsed_time = res.get('elapsed_time', 0.0)
+                
+                if prediction == 'fake':  
+                    output = {
+                        "Prediction": prediction,
+                        "Confidence": f"{confidence:.4f}",
+                        "Elapsed Time": f"{elapsed_time:.3f}s"
+                    }
+                else:
+                    output = {
+                        "Prediction": prediction,
+                        "Confidence": f"{1-confidence:.4f}",
+                        "Elapsed Time": f"{elapsed_time:.3f}s"
+                    }
+                return json.dumps(output, indent=2), None
+            else:
+                return json.dumps({"error": "Detection failed"}), None
+
+    except Exception as e:
+        return json.dumps({"error": str(e)}), None
+        
+    finally:
+        # Cleanup cropped image if it's different from original
+        if processed_path != image_path and os.path.exists(processed_path):
+            try:
+                os.remove(processed_path)
+            except Exception as e:
+                print(f"Warning: Could not remove temporary file {processed_path}: {e}")
+
+# Create Gradio Interface
+# Use theme only if gradio version supports it
+demo = gr.Blocks(title="Deepfake Detection Space", theme=gr.themes.Soft())
+
+with demo:
+    gr.Markdown("# 🔍 Deepfake Detection Space")
+    gr.Markdown("""
+    This space collects a series of state-of-the-art methods for deepfake detection, allowing for free and unlimited use.
+    
+    ### Training & Performance
+    All methods have been trained using the **[DeepShield dataset](https://zenodo.org/records/15648378)**, on images generated with **Stable Diffusion XL** and **StyleGAN 2**. 
+    You can expect performance comparable to the results shown in [Dell'Anna et al. (2025)](https://arxiv.org/pdf/2504.20658).
+
+    ### Understanding the Results
+    *   **Prediction**: Tells if an image is **Real** or **Fake**.
+    *   **Confidence**: The confidence with which the model determines if the image is real or fake.
+    *   **Elapsed Time**: The time the model needed to make the prediction (excluding preprocessing or model building).
+
+    ### Understanding the Results produced by "ALL"
+    *   Runs all available detectors (R50_TF, R50_nodown, CLIP-D, P2G, NPR) sequentially on the input image.
+    *   Produces a **Weighted Majority Vote** verdict (Real/Fake). Each model's vote is weighted by a fixed importance score (summing to 1) based on user ranking **and its confidence score**. Only confident predictions (> 0.6) are counted.
+    *   You can find the specific weights used for each model in the **"⚖️ Weight Details"** menu below.
+    *   Also generates a **Confidence Plot** visualizing each model's score and a textual **Explanation** of the consensus.
+    *   In the plot, **Green** bars indicate a **Real** prediction, while **Red** bars indicate a **Fake** prediction.
+
+
+    ### Note
+    ⚠️ Due to file size limitations, model weights need to be downloaded automatically on first use. This may take a few moments. <br>
+    ⚠️ To provide a free service, all models run on CPU. The detection process may take a few seconds, depending on the image size and the selected detector.
+    """)
+    
+    with gr.Row():
+        with gr.Column():
+            image_input = gr.Image(type="filepath", label="Input Image", height=400)
+            detector_input = gr.Dropdown(
+                choices=DETECTORS, 
+                value=DETECTORS[0], 
+                label="Select Detector",
+                info="Choose which deepfake detection model to use"
+            )
+            submit_btn = gr.Button("🔍 Detect", variant="primary")
+        
+        with gr.Column():
+            output_display = gr.Textbox(
+                label="Detection Results", 
+                lines=15, 
+                max_lines=20,
+                show_copy_button=True
+            )
+            plot_output = gr.Plot(label="Confidence Scores")
+    
+    with gr.Accordion("⚖️ Weight Details", open=False):
+        gr.Markdown(f"""
+        ### **Detector Weights**
+        The weights are assigned based on the ranking (based on the results of [TrueFake: A Real World Case Dataset of Last Generation Fake Images also Shared on Social Networks](https://arxiv.org/pdf/2504.20658)): **CLIP-D > R50_TF > R50_nodown > P2G > NPR**, such that their sum equals 1.
+        
+        | Detector | Weight |
+        | :--- | :---: |
+        | **CLIP-D** | {DETECTOR_WEIGHTS['CLIP-D']:.2f} |
+        | **R50_TF** | {DETECTOR_WEIGHTS['R50_TF']:.2f} |
+        | **R50_nodown** | {DETECTOR_WEIGHTS['R50_nodown']:.2f} |
+        | **P2G** | {DETECTOR_WEIGHTS['P2G']:.2f} |
+        | **NPR** | {DETECTOR_WEIGHTS['NPR']:.2f} |
+        """)
+
+    with gr.Accordion("📚 Model Details", open=False):
+        gr.Markdown("""
+        ### **ALL**
+        *   **Description**: Runs all available detectors (R50_TF, R50_nodown, CLIP-D, P2G, NPR) sequentially on the input image.
+        *   **Results**: Produces a **Majority Vote** verdict (Real/Fake) considering only confident predictions (> 0.6). Also generates a **Confidence Plot** visualizing each model's score and a textual **Explanation** of the consensus.
+
+        ### **R50_TF**
+        *   **Description**: A ResNet50 architecture modified to exclude downsampling at the first layer. It uses "learned prototypes" in the classification head for robust detection.
+        *   **Paper**: [TrueFake: A Real World Case Dataset of Last Generation Fake Images also Shared on Social Networks](https://arxiv.org/pdf/2504.20658)
+        *   **Code**: [GitHub Repository](https://github.com/MMLab-unitn/TrueFake-IJCNN25)
+
+        ### **R50_nodown**
+        *   **Description**: A ResNet-50 model without downsampling operations in the first layer, designed to preserve high-frequency artifacts common in synthetic images.
+        *   **Paper**: [On the detection of synthetic images generated by diffusion models](https://arxiv.org/abs/2211.00680)
+        *   **Code**: [GitHub Repository](https://grip-unina.github.io/DMimageDetection/)
+
+        ### **CLIP-D**
+        *   **Description**: A lightweight detection strategy based on CLIP features. It exhibits surprising generalization ability using only a handful of example images.
+        *   **Paper**: [Raising the Bar of AI-generated Image Detection with CLIP](https://arxiv.org/abs/2312.00195v2)
+        *   **Code**: [GitHub Repository](https://grip-unina.github.io/ClipBased-SyntheticImageDetection/)
+
+        ### **P2G (Prompt2Guard)**
+        *   **Description**: Uses Vision-Language Models (VLMs) with conditioned prompt-optimization for continual deepfake detection. It leverages read-only prompts for efficiency.
+        *   **Paper**: [Conditioned Prompt-Optimization for Continual Deepfake Detection](https://arxiv.org/abs/2407.21554)
+        *   **Code**: [GitHub Repository](https://github.com/laitifranz/Prompt2Guard)
+
+        ### **NPR**
+        *   **Description**: Focuses on Neighboring Pixel Relationships (NPR) to capture generalized structural artifacts stemming from up-sampling operations in generative networks.
+        *   **Paper**: [Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection](https://arxiv.org/abs/2312.10461)
+        *   **Code**: [GitHub Repository](https://github.com/chuangchuangtan/NPR-DeepfakeDetection)
+        """)
+
+    gr.Markdown("""
+    ---
+    ### References
+    1. Dell'Anna, S., Montibeller, A., & Boato, G. (2025). *TrueFake: A Real World Case Dataset of Last Generation Fake Images also Shared on Social Networks*. arXiv preprint arXiv:2504.20658.
+    2. Corvi, R., et al. (2023). *On the detection of synthetic images generated by diffusion models*. ICASSP.
+    3. Cozzolino, D., et al. (2023). *Raising the Bar of AI-generated Image Detection with CLIP*. CVPRW.
+    4. Laiti, F., et al. (2024). *Conditioned Prompt-Optimization for Continual Deepfake Detection*. arXiv preprint arXiv:2407.21554.
+    5. Tan, C., et al. (2024). *Rethinking the up-sampling operations in cnn-based generative network for generalizable deepfake detection*. CVPR.
+    """)
+    
+    submit_btn.click(
+        fn=predict,
+        inputs=[image_input, detector_input],
+        outputs=[output_display, plot_output]
+    )
+
+if __name__ == "__main__":
+    # For HF Spaces, configure server settings
+    if os.environ.get("SPACE_ID"):
+        demo.launch(server_name="0.0.0.0", server_port=7860, allowed_paths=["."])
+    else:
+        # Local execution
+        demo.launch()

configs/CLIP-D.yaml

@@ -0,0 +1,72 @@
+global:
+  dataset_path: <PATH_TO_DATASET> 
+  device_override: cpu  # null for auto-selection
+  min_vram: 16000
+  split_file: ./split.json
+  num_threads: 8
+  dry_run: false
+  only_list: false
+
+detector_args:
+  - "--arch"
+  - "opencliplinearnext_clipL14commonpool"
+  - "--norm_type"
+  - "clip"
+  - "--resize_size"
+  - "200"
+  - "--resize_ratio"
+  - "1"
+  - "--resize_prob"
+  - "0.2"
+  - "--cmp_qual"
+  - "65,100"
+  - "--cmp_prob"
+  - "0.5"
+  - "--resizeSize"
+  - "224"
+
+training:
+  - data: gan2:pre&sdXL:pre&realFFHQ:pre&realFORLAB:pre
+
+testing:
+  - realFFHQ:fb
+  - realFORLAB:fb
+  - gan1:fb
+  - gan2:fb
+  - gan3:fb
+  - sd15:fb
+  - sd2:fb
+  - sd3:fb
+  - sdXL:fb
+  - flux:fb
+  - realFFHQ:tl
+  - realFORLAB:tl
+  - gan1:tl
+  - gan2:tl
+  - gan3:tl
+  - sd15:tl
+  - sd2:tl
+  - sd3:tl
+  - sdXL:tl
+  - flux:tl
+  - realFFHQ:tw
+  - realFORLAB:tw
+  - gan1:tw
+  - gan2:tw
+  - gan3:tw
+  - sd15:tw
+  - sd2:tw
+  - sd3:tw
+  - sdXL:tw
+  - flux:tw
+  - realFFHQ:pre
+  - realFORLAB:pre
+  - gan1:pre
+  - gan2:pre
+  - gan3:pre
+  - sd15:pre
+  - sd2:pre
+  - sd3:pre
+  - sdXL:pre
+  - flux:pre
+

configs/NPR.yaml

@@ -0,0 +1,56 @@
+global:
+  dataset_path: <PATH_TO_DATASET> 
+  device_override: cpu  # null for auto-selection
+  min_vram: 16000
+  split_file: ./split.json
+  num_threads: 8
+  dry_run: false
+  only_list: false
+
+detector_args: []
+
+training:
+  - data: gan2:pre&sdXL:pre&realFFHQ:pre&realFORLAB:pre
+
+testing:
+  - realFFHQ:fb
+  - realFORLAB:fb
+  - gan1:fb
+  - gan2:fb
+  - gan3:fb
+  - sd15:fb
+  - sd2:fb
+  - sd3:fb
+  - sdXL:fb
+  - flux:fb
+  - realFFHQ:tl
+  - realFORLAB:tl
+  - gan1:tl
+  - gan2:tl
+  - gan3:tl
+  - sd15:tl
+  - sd2:tl
+  - sd3:tl
+  - sdXL:tl
+  - flux:tl
+  - realFFHQ:tw
+  - realFORLAB:tw
+  - gan1:tw
+  - gan2:tw
+  - gan3:tw
+  - sd15:tw
+  - sd2:tw
+  - sd3:tw
+  - sdXL:tw
+  - flux:tw
+  - realFFHQ:pre
+  - realFORLAB:pre
+  - gan1:pre
+  - gan2:pre
+  - gan3:pre
+  - sd15:pre
+  - sd2:pre
+  - sd3:pre
+  - sdXL:pre
+  - flux:pre
+

configs/P2G.yaml

@@ -0,0 +1,56 @@
+global:
+  dataset_path: <PATH_TO_DATASET> 
+  device_override: cpu  # null for auto-selection
+  min_vram: 16000
+  split_file: ./split.json
+  num_threads: 8
+  dry_run: false
+  only_list: false
+
+detector_args: []
+
+training:
+  - data: gan2:pre&sdXL:pre&realFFHQ:pre&realFORLAB:pre
+
+testing:
+  - realFFHQ:fb
+  - realFORLAB:fb
+  - gan1:fb
+  - gan2:fb
+  - gan3:fb
+  - sd15:fb
+  - sd2:fb
+  - sd3:fb
+  - sdXL:fb
+  - flux:fb
+  - realFFHQ:tl
+  - realFORLAB:tl
+  - gan1:tl
+  - gan2:tl
+  - gan3:tl
+  - sd15:tl
+  - sd2:tl
+  - sd3:tl
+  - sdXL:tl
+  - flux:tl
+  - realFFHQ:tw
+  - realFORLAB:tw
+  - gan1:tw
+  - gan2:tw
+  - gan3:tw
+  - sd15:tw
+  - sd2:tw
+  - sd3:tw
+  - sdXL:tw
+  - flux:tw
+  - realFFHQ:pre
+  - realFORLAB:pre
+  - gan1:pre
+  - gan2:pre
+  - gan3:pre
+  - sd15:pre
+  - sd2:pre
+  - sd3:pre
+  - sdXL:pre
+  - flux:pre
+

configs/R50_TF.yaml

@@ -0,0 +1,61 @@
+global:
+  dataset_path: <PATH_TO_DATASET> 
+  device_override: cpu  # null for auto-selection
+  min_vram: 16000
+  split_file: ./split.json
+  num_threads: 8
+  dry_run: false
+  only_list: false
+
+
+detector_args:
+  - "--arch"
+  - "nodown"
+  - "--prototype"
+  - "--freeze"
+
+training:
+  - data: gan2:pre&sdXL:pre&realFFHQ:pre&realFORLAB:pre
+
+testing:
+  - realFFHQ:fb
+  - realFORLAB:fb
+  - gan1:fb
+  - gan2:fb
+  - gan3:fb
+  - sd15:fb
+  - sd2:fb
+  - sd3:fb
+  - sdXL:fb
+  - flux:fb
+  - realFFHQ:tl
+  - realFORLAB:tl
+  - gan1:tl
+  - gan2:tl
+  - gan3:tl
+  - sd15:tl
+  - sd2:tl
+  - sd3:tl
+  - sdXL:tl
+  - flux:tl
+  - realFFHQ:tw
+  - realFORLAB:tw
+  - gan1:tw
+  - gan2:tw
+  - gan3:tw
+  - sd15:tw
+  - sd2:tw
+  - sd3:tw
+  - sdXL:tw
+  - flux:tw
+  - realFFHQ:pre
+  - realFORLAB:pre
+  - gan1:pre
+  - gan2:pre
+  - gan3:pre
+  - sd15:pre
+  - sd2:pre
+  - sd3:pre
+  - sdXL:pre
+  - flux:pre
+

configs/R50_nodown.yaml

@@ -0,0 +1,84 @@
+global:
+  dataset_path: <PATH_TO_DATASET> 
+  device_override: cpu  # null for auto-selection
+  min_vram: 16000
+  split_file: ./split.json
+  num_threads: 8
+  dry_run: false
+  only_list: false
+
+detector_args:
+  - "--arch"
+  - "res50nodown"
+  - "--norm_type"
+  - "resnet"
+  - "--resize_size"
+  - "256"
+  - "--resize_ratio"
+  - "0.75"
+  - "--resize_prob"
+  - "0.2"
+  - "--cmp_qual"
+  - "30,100"
+  - "--cmp_prob"
+  - "0.5"
+  - "--cropSize"
+  - "96"
+  - "--blur_sig"
+  - "0.1,3.0"
+  - "--blur_prob"
+  - "0.5"
+  - "--jitter_prob"
+  - "0.8"
+  - "--colordist_prob"
+  - "0.2"
+  - "--cutout_prob"
+  - "0.2"
+  - "--noise_prob"
+  - "0.2"
+
+training:
+  - data: gan2:pre&sdXL:pre&realFFHQ:pre&realFORLAB:pre
+
+testing:
+  - realFFHQ:fb
+  - realFORLAB:fb
+  - gan1:fb
+  - gan2:fb
+  - gan3:fb
+  - sd15:fb
+  - sd2:fb
+  - sd3:fb
+  - sdXL:fb
+  - flux:fb
+  - realFFHQ:tl
+  - realFORLAB:tl
+  - gan1:tl
+  - gan2:tl
+  - gan3:tl
+  - sd15:tl
+  - sd2:tl
+  - sd3:tl
+  - sdXL:tl
+  - flux:tl
+  - realFFHQ:tw
+  - realFORLAB:tw
+  - gan1:tw
+  - gan2:tw
+  - gan3:tw
+  - sd15:tw
+  - sd2:tw
+  - sd3:tw
+  - sdXL:tw
+  - flux:tw
+  - realFFHQ:pre
+  - realFORLAB:pre
+  - gan1:pre
+  - gan2:pre
+  - gan3:pre
+  - sd15:pre
+  - sd2:pre
+  - sd3:pre
+  - sdXL:pre
+  - flux:pre
+

demo_images/README.md

@@ -0,0 +1,21 @@
+# Download Demo Dataset
+
+You can download of demo dataset from this [link](https://drive.google.com/file/d/134Bw8l9tEC7oZJpTAeMO80QRqgdJfJS9/view?usp=sharing). The demo dataset contains 200 images randomly samples from the DeepShield Dataset. \
+
+Place sample images for quick testing in `demo_images/`, organized by platform and label:
+
+```
+	demo_images/
+			Facebook/
+					Fake/
+					Real/
+			PreSocial/
+					Fake/
+					Real/
+			Telegram/
+					Fake/
+					Real/
+			X/
+					Fake/
+					Real/
+```

detectors/.gitattributes

@@ -0,0 +1 @@
+*.pkl filter=lfs diff=lfs merge=lfs -text

detectors/CLIP-D/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

detectors/CLIP-D/README.md

@@ -0,0 +1,29 @@
+# ClipBased-SyntheticImageDetection
+
+[![Official Github Repo](https://img.shields.io/badge/Github%20page-222222.svg?style=for-the-badge&logo=github)](https://grip-unina.github.io/ClipBased-SyntheticImageDetection/)
+[![Paper](https://img.shields.io/badge/-arXiv-B31B1B.svg?style=for-the-badge)](https://arxiv.org/abs/2312.00195v2)
+[![GRIP Research Group Website](https://img.shields.io/badge/-GRIP-0888ef.svg?style=for-the-badge)](https://www.grip.unina.it)
+
+Original Paper:
+[Raising the Bar of AI-generated Image Detection with CLIP](https://arxiv.org/abs/2312.00195v2).
+
+Authors: Davide Cozzolino, Giovanni Poggi, Riccardo Corvi, Matthias Nießner, and Luisa Verdoliva.
+
+## Abstract
+
+The aim of this work is to explore the potential of pre-trained vision-language models (VLMs) for universal detection of AI-generated images. We develop a lightweight detection strategy based on CLIP features and study its performance in a wide variety of challenging scenarios. We find that, contrary to previous beliefs, it is neither necessary nor convenient to use a large domain-specific dataset for training. On the contrary, by using only a handful of example images from a single generative model, a CLIP-based detector exhibits surprising generalization ability and high robustness across different architectures, including recent commercial tools such as Dalle-3, Midjourney v5, and Firefly. We match the state-of-the-art (SoTA) on in-distribution data and significantly improve upon it in terms of generalization to out-of-distribution data (+6% AUC) and robustness to impaired/laundered data (+13%).
+
+## Please Cite 
+
+```
+@inproceedings{cozzolino2023raising,
+  author={Davide Cozzolino and Giovanni Poggi and 
+  Riccardo Corvi and Matthias Nießner and Luisa 
+  Verdoliva},
+  title={{Raising the Bar of AI-generated Image 
+  Detection with CLIP}}, 
+  booktitle={IEEE/CVF Conference on Computer Vision 
+  and Pattern Recognition Workshops (CVPRW)},
+  year={2024},
+}
+```

detectors/CLIP-D/checkpoint/pretrained/weights/best.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34315ca92adbf17921f5aa1e28f8fe5bf7d56dd8126205c9e264cbfb26582d12
+size 15452

detectors/CLIP-D/detect.py

@@ -0,0 +1,109 @@
+# ----------------------------------------------------------------------------
+# IMPORTS
+# ----------------------------------------------------------------------------
+import os
+import sys
+import time
+import torch
+import argparse
+from PIL import Image
+import torchvision.transforms as transforms
+
+# Add project root to path for imports
+project_root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+sys.path.append(project_root)
+from support.detect_utils import format_result, save_result, get_device
+
+from networks import create_architecture
+
+# ----------------------------------------------------------------------------
+# IMAGE PREPROCESSING
+# ----------------------------------------------------------------------------
+def preprocess_image(image_path, size=224):
+    """Load and preprocess a single image for model input."""
+    if not os.path.exists(image_path):
+        raise FileNotFoundError(f"Image not found: {image_path}")
+        
+    image = Image.open(image_path).convert('RGB')
+    transform = transforms.Compose([
+        transforms.Resize(size),
+        transforms.CenterCrop(size),
+        transforms.ToTensor(),
+        transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
+    ])
+    return transform(image).unsqueeze(0)  # Add batch dimension
+
+# ----------------------------------------------------------------------------
+# ARGUMENT PARSING
+# ----------------------------------------------------------------------------
+def parse_args():
+    parser = argparse.ArgumentParser(description='CLIP-D single image detector')
+    parser.add_argument('--image', type=str, required=True, help='Path to input image')
+    parser.add_argument('--model', type=str, default='pretrained', help='Name of the model checkpoint directory')
+    parser.add_argument('--output', type=str, help='Path to save detection result JSON')
+    parser.add_argument('--device', type=str, help='Device to run on (e.g., cuda:0, cuda:1, cpu)')
+    return parser.parse_args()
+
+def main():
+    args = parse_args()
+    
+    # Setup device
+    device = torch.device(args.device) if args.device else get_device()
+    
+    # Load model
+    try:
+        load_path = f'./detectors/CLIP-D/checkpoint/{args.model}/weights/best.pt'
+        
+        if not os.path.exists(load_path):
+            raise FileNotFoundError(f"Model weights not found at: {load_path}")
+            
+        checkpoint = torch.load(load_path, map_location=device)
+        # Initialize model and load state
+        model =  create_architecture("opencliplinearnext_clipL14commonpool", pretrained=False, num_classes=1).to(device)
+        if 'model' in checkpoint:
+            model.load_state_dict(checkpoint['model'])
+        else:
+            model.load_state_dict(checkpoint)
+        model.eval()
+    except Exception as e:
+        print(f"Error loading model: {e}")
+        return
+
+    # Load and preprocess image
+    try:
+        image_tensor = preprocess_image(args.image)
+        image_tensor = image_tensor.to(device)
+    except Exception as e:
+        print(f"Error loading image: {e}")
+        return
+
+    # Run detection
+    start_time = time.time()
+    with torch.no_grad():
+        try:
+            score = model(image_tensor)
+            prediction = torch.sigmoid(score)
+            
+            confidence = prediction.item()
+            
+            result = format_result(
+                'fake' if confidence>0.5 else 'real',
+                confidence,
+                time.time() - start_time
+            )
+            
+            # Print result
+            print(f"Prediction: {result['prediction']}")
+            print(f"Confidence: {result['confidence']:.4f}")
+            print(f"Time: {result['elapsed_time']:.3f}s")
+            
+            # Save result if output path provided
+            if args.output:
+                save_result(result, args.output)
+                
+        except Exception as e:
+            print(f"Error during detection: {e}")
+            return
+
+if __name__ == '__main__':
+    main()

detectors/CLIP-D/networks/__init__.py

@@ -0,0 +1,70 @@
+'''                                        
+Copyright 2024 Image Processing Research Group of University Federico
+II of Naples ('GRIP-UNINA'). All rights reserved.
+                        
+Licensed under the Apache License, Version 2.0 (the "License");       
+you may not use this file except in compliance with the License. 
+You may obtain a copy of the License at                    
+                                           
+    http://www.apache.org/licenses/LICENSE-2.0
+                                                      
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,    
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.                         
+See the License for the specific language governing permissions and
+limitations under the License.
+'''
+
+def create_architecture(name_arch, pretrained=False, num_classes=1):
+    if name_arch == "res50nodown":
+        from .resnet_mod import resnet50
+
+        if pretrained:
+            model = resnet50(pretrained=True, stride0=1, dropout=0.5).change_output(num_classes)
+        else:
+            model = resnet50(num_classes=num_classes, stride0=1, dropout=0.5)
+    elif name_arch == "res50":
+        from .resnet_mod import resnet50
+
+        if pretrained:
+            model = resnet50(pretrained=True, stride0=2).change_output(num_classes)
+        else:
+            model = resnet50(num_classes=num_classes, stride0=2)
+    elif name_arch.startswith('opencliplinear_'):
+        from .openclipnet import OpenClipLinear
+        model = OpenClipLinear(num_classes=num_classes, pretrain=name_arch[15:], normalize=True)
+    elif name_arch.startswith('opencliplinearnext_'):
+        from .openclipnet import OpenClipLinear
+        model = OpenClipLinear(num_classes=num_classes, pretrain=name_arch[19:], normalize=True, next_to_last=True)
+    else:
+        assert False
+    return model
+
+def count_parameters(model):
+    return sum(p.numel() for p in model.parameters() if p.requires_grad)
+
+def load_weights(model, model_path):
+    from torch import load
+    dat = load(model_path, map_location='cpu')
+    if 'model' in dat:
+        if ('module._conv_stem.weight' in dat['model']) or \
+           ('module.fc.fc1.weight' in dat['model']) or \
+           ('module.fc.weight' in dat['model']):
+            model.load_state_dict(
+                {key[7:]: dat['model'][key] for key in dat['model']})
+        else:
+            model.load_state_dict(dat['model'])
+    elif 'state_dict' in dat:
+        model.load_state_dict(dat['state_dict'])
+    elif 'net' in dat:
+        model.load_state_dict(dat['net'])
+    elif 'main.0.weight' in dat:
+        model.load_state_dict(dat)
+    elif '_fc.weight' in dat:
+        model.load_state_dict(dat)
+    elif 'conv1.weight' in dat:
+        model.load_state_dict(dat)
+    else:
+        print(list(dat.keys()))
+        assert False
+    return model

detectors/CLIP-D/networks/openclipnet.py

@@ -0,0 +1,85 @@
+'''                                        
+Copyright 2024 Image Processing Research Group of University Federico
+II of Naples ('GRIP-UNINA'). All rights reserved.
+                        
+Licensed under the Apache License, Version 2.0 (the "License");       
+you may not use this file except in compliance with the License. 
+You may obtain a copy of the License at                    
+                                           
+    http://www.apache.org/licenses/LICENSE-2.0
+                                                      
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,    
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.                         
+See the License for the specific language governing permissions and
+limitations under the License.
+'''
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import open_clip
+from .resnet_mod import ChannelLinear
+
+dict_pretrain = {
+    'clipL14openai'     : ('ViT-L-14', 'openai'),
+    'clipL14laion400m'  : ('ViT-L-14', 'laion400m_e32'),
+    'clipL14laion2B'    : ('ViT-L-14', 'laion2b_s32b_b82k'),
+    'clipL14datacomp'   : ('ViT-L-14', 'laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K', 'open_clip_pytorch_model.bin'),
+    'clipL14commonpool' : ('ViT-L-14', "laion/CLIP-ViT-L-14-CommonPool.XL-s13B-b90K", 'open_clip_pytorch_model.bin'),
+    'clipaL14datacomp'  : ('ViT-L-14-CLIPA', 'datacomp1b'),
+    'cocaL14laion2B'    : ('coca_ViT-L-14', 'laion2b_s13b_b90k'),
+    'clipg14laion2B'    : ('ViT-g-14', 'laion2b_s34b_b88k'),
+    'eva2L14merged2b'   : ('EVA02-L-14', 'merged2b_s4b_b131k'),
+    'clipB16laion2B'    : ('ViT-B-16', 'laion2b_s34b_b88k'),
+}
+
+
+class OpenClipLinear(nn.Module):
+    def __init__(self, num_classes=1, pretrain='clipL14commonpool', normalize=True, next_to_last=False):
+        super(OpenClipLinear, self).__init__()
+        
+        # Modified to handle download failures gracefully
+        # The checkpoint only contains fc weights, so we need the pretrained backbone
+        if len(dict_pretrain[pretrain])==2:
+            try:
+                backbone = open_clip.create_model(dict_pretrain[pretrain][0], pretrained=dict_pretrain[pretrain][1])
+            except Exception as e:
+                print(f"WARNING: Could not download pretrained weights ({e}). Using random initialization.")
+                backbone = open_clip.create_model(dict_pretrain[pretrain][0], pretrained=None)
+        else:
+            try:
+                from huggingface_hub import hf_hub_download
+                backbone = open_clip.create_model(dict_pretrain[pretrain][0], pretrained=hf_hub_download(*dict_pretrain[pretrain][1:]))
+            except Exception as e:
+                print(f"WARNING: Could not download pretrained weights ({e}). Using random initialization.")
+                backbone = open_clip.create_model(dict_pretrain[pretrain][0], pretrained=None)
+        
+        if next_to_last:
+            self.num_features = backbone.visual.proj.shape[0]
+            backbone.visual.proj = None
+        else:
+            self.num_features = backbone.visual.output_dim
+        
+        self.bb = [backbone, ]
+        self.normalize = normalize
+        
+        self.fc = ChannelLinear(self.num_features, num_classes)
+        torch.nn.init.normal_(self.fc.weight.data, 0.0, 0.02)
+
+    def to(self, *args, **kwargs):
+        self.bb[0].to(*args, **kwargs)
+        super(OpenClipLinear, self).to(*args, **kwargs)
+        return self
+
+    def forward_features(self, x):
+        with torch.no_grad():
+            self.bb[0].eval()
+            features = self.bb[0].encode_image(x, normalize=self.normalize)
+        return features
+
+    def forward_head(self, x):
+        return self.fc(x)
+
+    def forward(self, x):
+        return self.forward_head(self.forward_features(x))

detectors/CLIP-D/networks/resnet_mod.py

@@ -0,0 +1,335 @@
+'''                                        
+Copyright 2024 Image Processing Research Group of University Federico
+II of Naples ('GRIP-UNINA'). All rights reserved.
+                        
+Licensed under the Apache License, Version 2.0 (the "License");       
+you may not use this file except in compliance with the License. 
+You may obtain a copy of the License at                    
+                                           
+    http://www.apache.org/licenses/LICENSE-2.0
+                                                      
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,    
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.                         
+See the License for the specific language governing permissions and
+limitations under the License.
+'''
+
+import torch
+import torch.nn as nn
+import torch.utils.model_zoo as model_zoo
+
+__all__ = ["ResNet", "resnet18", "resnet34", "resnet50", "resnet101", "resnet152"]
+
+
+model_urls = {
+    "resnet18": "https://download.pytorch.org/models/resnet18-5c106cde.pth",
+    "resnet34": "https://download.pytorch.org/models/resnet34-333f7ec4.pth",
+    "resnet50": "https://download.pytorch.org/models/resnet50-19c8e357.pth",
+    "resnet101": "https://download.pytorch.org/models/resnet101-5d3b4d8f.pth",
+    "resnet152": "https://download.pytorch.org/models/resnet152-b121ed2d.pth",
+}
+
+class ChannelLinear(nn.Linear):
+    def __init__(
+        self, in_features: int, out_features: int, bias: bool = True, pool=None
+    ) -> None:
+        super(ChannelLinear, self).__init__(in_features, out_features, bias)
+        self.compute_axis = 1
+        self.pool = pool
+
+    def forward(self, x):
+        axis_ref = len(x.shape) - 1
+        x = torch.transpose(x, self.compute_axis, axis_ref)
+        out_shape = list(x.shape)
+        out_shape[-1] = self.out_features
+        x = x.reshape(-1, x.shape[-1])
+        x = x.matmul(self.weight.t())
+        if self.bias is not None:
+            x = x + self.bias[None, :]
+        x = torch.transpose(x.view(out_shape), axis_ref, self.compute_axis)
+        if self.pool is not None:
+            x = self.pool(x)
+        return x
+
+
+def conv3x3(in_planes, out_planes, stride=1, padding=1):
+    """3x3 convolution with padding"""
+    return nn.Conv2d(
+        in_planes, out_planes, kernel_size=3, stride=stride, padding=padding, bias=False
+    )
+
+
+def conv1x1(in_planes, out_planes, stride=1):
+    """1x1 convolution"""
+    return nn.Conv2d(in_planes, out_planes, kernel_size=1, stride=stride, bias=False)
+
+
+class BasicBlock(nn.Module):
+    expansion = 1
+
+    def __init__(self, inplanes, planes, stride=1, padding=1, downsample=None):
+        super(BasicBlock, self).__init__()
+        self.conv1 = conv3x3(inplanes, planes, stride, padding=padding)
+        self.bn1 = nn.BatchNorm2d(planes)
+        self.relu = nn.ReLU(inplace=True)
+        self.conv2 = conv3x3(planes, planes, padding=padding)
+        self.bn2 = nn.BatchNorm2d(planes)
+        self.downsample = downsample
+        self.stride = stride
+        self.padding = padding
+
+    def forward(self, x):
+        identity = x
+
+        out = self.conv1(x)
+        out = self.bn1(out)
+        out = self.relu(out)
+
+        out = self.conv2(out)
+        out = self.bn2(out)
+
+        if self.padding == 0:
+            identity = identity[..., 1:-1, 1:-1]
+        if self.downsample is not None:
+            identity = self.downsample(identity)
+        if self.padding == 0:
+            identity = identity[..., 1:-1, 1:-1]
+
+        out += identity
+        out = self.relu(out)
+
+        return out
+
+
+class Bottleneck(nn.Module):
+    expansion = 4
+
+    def __init__(self, inplanes, planes, stride=1, padding=1, downsample=None):
+        super(Bottleneck, self).__init__()
+        self.conv1 = conv1x1(inplanes, planes)
+        self.bn1 = nn.BatchNorm2d(planes)
+        self.conv2 = conv3x3(planes, planes, stride, padding=padding)
+        self.bn2 = nn.BatchNorm2d(planes)
+        self.conv3 = conv1x1(planes, planes * self.expansion)
+        self.bn3 = nn.BatchNorm2d(planes * self.expansion)
+        self.relu = nn.ReLU(inplace=True)
+        self.downsample = downsample
+        self.stride = stride
+        self.padding = padding
+
+    def forward(self, x):
+        identity = x
+
+        out = self.conv1(x)
+        out = self.bn1(out)
+        out = self.relu(out)
+
+        out = self.conv2(out)
+        out = self.bn2(out)
+        out = self.relu(out)
+
+        out = self.conv3(out)
+        out = self.bn3(out)
+
+        if self.padding == 0:
+            identity = identity[..., 1:-1, 1:-1]
+        if self.downsample is not None:
+            identity = self.downsample(identity)
+
+        out += identity
+        out = self.relu(out)
+
+        return out
+
+class ResNet(nn.Module):
+    def __init__(
+        self,
+        block,
+        layers,
+        num_classes=1000,
+        zero_init_residual=False,
+        stride0=2,
+        padding=1,
+        dropout=0.0,
+        gap_size=None,
+    ):
+        super(ResNet, self).__init__()
+        self.inplanes = 64
+
+        self.conv1 = nn.Conv2d(
+            3, 64, kernel_size=7, stride=stride0, padding=3 * padding, bias=False
+        )
+        self.bn1 = nn.BatchNorm2d(64)
+        if dropout > 0:
+            self.dropout = nn.Dropout(dropout)
+        else:
+            self.dropout = None
+        self.relu = nn.ReLU(inplace=True)
+        self.maxpool = nn.MaxPool2d(kernel_size=3, stride=stride0, padding=padding)
+        self.layer1 = self._make_layer(block, 64, layers[0], padding=padding)
+        self.layer2 = self._make_layer(block, 128, layers[1], stride=2, padding=padding)
+        self.layer3 = self._make_layer(block, 256, layers[2], stride=2, padding=padding)
+        self.layer4 = self._make_layer(block, 512, layers[3], stride=2, padding=padding)
+
+        if gap_size is None:
+            self.gap_size = None
+            self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
+        elif gap_size < 0:
+            with torch.no_grad():
+                y = self.forward_features(
+                    torch.zeros((1, 3, -gap_size, -gap_size), dtype=torch.float32)
+                ).shape
+            print("gap_size:", -gap_size, ">>", y[-1])
+            self.gap_size = y[-1]
+            self.avgpool = nn.AvgPool2d(kernel_size=self.gap_size, stride=1, padding=0)
+        elif gap_size == 1:
+            self.gap_size = gap_size
+            self.avgpool = None
+        else:
+            self.gap_size = gap_size
+            self.avgpool = nn.AvgPool2d(kernel_size=self.gap_size, stride=1, padding=0)
+        self.num_features = 512 * block.expansion
+        self.fc = ChannelLinear(self.num_features, num_classes)
+
+        for m in self.modules():
+            if isinstance(m, nn.Conv2d):
+                nn.init.kaiming_normal_(m.weight, mode="fan_out", nonlinearity="relu")
+            elif isinstance(m, nn.BatchNorm2d):
+                nn.init.constant_(m.weight, 1)
+                nn.init.constant_(m.bias, 0)
+
+        if zero_init_residual:
+            for m in self.modules():
+                if isinstance(m, Bottleneck):
+                    nn.init.constant_(m.bn3.weight, 0)
+                elif isinstance(m, BasicBlock):
+                    nn.init.constant_(m.bn2.weight, 0)
+
+    def _make_layer(self, block, planes, blocks, stride=1, padding=1):
+        downsample = None
+        if stride != 1 or self.inplanes != planes * block.expansion:
+            downsample = nn.Sequential(
+                conv1x1(self.inplanes, planes * block.expansion, stride),
+                nn.BatchNorm2d(planes * block.expansion),
+            )
+
+        layers = []
+        layers.append(
+            block(
+                self.inplanes,
+                planes,
+                stride=stride,
+                downsample=downsample,
+                padding=padding,
+            )
+        )
+        self.inplanes = planes * block.expansion
+        for _ in range(1, blocks):
+            layers.append(block(self.inplanes, planes, padding=padding))
+
+        return nn.Sequential(*layers)
+
+    def change_output(self, num_classes):
+        self.fc = ChannelLinear(self.num_features, num_classes)
+        torch.nn.init.normal_(self.fc.weight.data, 0.0, 0.02)
+        return self
+
+    def change_input(self, num_inputs):
+        data = self.conv1.weight.data
+        old_num_inputs = int(data.shape[1])
+        if num_inputs > old_num_inputs:
+            times = num_inputs // old_num_inputs
+            if (times * old_num_inputs) < num_inputs:
+                times = times + 1
+            data = data.repeat(1, times, 1, 1) / times
+        elif num_inputs == old_num_inputs:
+            return self
+
+        data = data[:, :num_inputs, :, :]
+        print(self.conv1.weight.data.shape, "->", data.shape)
+        self.conv1.weight.data = data
+
+        return self
+
+    def forward_features(self, x):
+        x = self.conv1(x)
+        x = self.bn1(x)
+        x = self.relu(x)
+        x = self.maxpool(x)
+
+        x = self.layer1(x)
+        x = self.layer2(x)
+        x = self.layer3(x)
+        x = self.layer4(x)
+        return x
+
+    def forward_head(self, x):
+        if self.avgpool is not None:
+            x = self.avgpool(x)
+        if self.dropout is not None:
+            x = self.dropout(x)
+        y = self.fc(x)
+        if self.gap_size is None:
+            y = torch.squeeze(torch.squeeze(y, -1), -1)
+        return y
+
+    def forward(self, x):
+        x = self.forward_features(x)
+        x = self.forward_head(x)
+        return x
+
+
+def resnet18(pretrained=False, **kwargs):
+    """Constructs a ResNet-18 model.
+    Args:
+        pretrained (bool): If True, returns a model pre-trained on ImageNet
+    """
+    model = ResNet(BasicBlock, [2, 2, 2, 2], **kwargs)
+    if pretrained:
+        model.load_state_dict(model_zoo.load_url(model_urls["resnet18"]))
+    return model
+
+
+def resnet34(pretrained=False, **kwargs):
+    """Constructs a ResNet-34 model.
+    Args:
+        pretrained (bool): If True, returns a model pre-trained on ImageNet
+    """
+    model = ResNet(BasicBlock, [3, 4, 6, 3], **kwargs)
+    if pretrained:
+        model.load_state_dict(model_zoo.load_url(model_urls["resnet34"]))
+    return model
+
+
+def resnet50(pretrained=False, **kwargs):
+    """Constructs a ResNet-50 model.
+    Args:
+        pretrained (bool): If True, returns a model pre-trained on ImageNet
+    """
+    model = ResNet(Bottleneck, [3, 4, 6, 3], **kwargs)
+    if pretrained:
+        model.load_state_dict(model_zoo.load_url(model_urls["resnet50"]))
+    return model
+
+
+def resnet101(pretrained=False, **kwargs):
+    """Constructs a ResNet-101 model.
+    Args:
+        pretrained (bool): If True, returns a model pre-trained on ImageNet
+    """
+    model = ResNet(Bottleneck, [3, 4, 23, 3], **kwargs)
+    if pretrained:
+        model.load_state_dict(model_zoo.load_url(model_urls["resnet101"]))
+    return model
+
+
+def resnet152(pretrained=False, **kwargs):
+    """Constructs a ResNet-152 model.
+    Args:
+        pretrained (bool): If True, returns a model pre-trained on ImageNet
+    """
+    model = ResNet(Bottleneck, [3, 8, 36, 3], **kwargs)
+    if pretrained:
+        model.load_state_dict(model_zoo.load_url(model_urls["resnet152"]))
+    return model

detectors/CLIP-D/parser.py

@@ -0,0 +1,25 @@
+import argparse
+
+def get_parser():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--name", type=str, default="test", help="run name")
+    parser.add_argument("--arch", type=str, default="opencliplinearnext_clipL14commonpresool", help="architecture name")
+
+    parser.add_argument("--task", type=str, help="Task: train/test")
+    parser.add_argument("--device", type=str, default="cuda:0", help="cuda device to use")
+
+    parser.add_argument("--split_file", type=str, help="Path to split json")
+    parser.add_argument("--data_root", type=str, help="Path to dataset")
+    parser.add_argument("--data_keys", type=str, help="Dataset specifications")
+
+    parser.add_argument("--batch_size", type=int, default=64, help='Dataloader batch size')
+    parser.add_argument("--num_threads", type=int, default=14, help='# threads for loading data')
+
+    parser.add_argument("--lr", type=float, default=0.0001, help="initial learning rate")
+    parser.add_argument("--weight_decay", type=float, default=0.0, help="weight decay")
+    parser.add_argument("--beta1", type=float, default=0.9, help="momentum term of adam")
+
+    parser.add_argument("--num_epoches", type=int, default=1000, help="# of epoches at starting learning rate")
+    parser.add_argument("--earlystop_epoch", type=int, default=5, help="Number of epochs without loss reduction before lowering the learning rate")
+    
+    return parser

detectors/CLIP-D/test.py

@@ -0,0 +1,167 @@
+import os
+from tqdm import tqdm
+import torch
+import pandas as pd
+import json
+import time
+import numpy as np
+from sklearn.metrics import roc_auc_score, accuracy_score
+from networks import create_architecture, count_parameters
+from utils.dataset import create_dataloader
+from utils.processing import add_processing_arguments
+from parser import get_parser
+
+def test(loader, model, settings, device):
+    model.eval()
+    
+    start_time = time.time()
+    
+    # File paths
+    output_dir = f'./results/{settings.name}/data/{settings.data_keys}'
+    os.makedirs(output_dir, exist_ok=True)
+    
+    csv_filename = os.path.join(output_dir, 'results.csv')
+    metrics_filename = os.path.join(output_dir, 'metrics.json')
+    image_results_filename = os.path.join(output_dir, 'image_results.json')
+    
+    # Collect all results
+    all_scores = []
+    all_labels = []
+    all_paths = []
+    image_results = []
+    
+    # Extract training dataset keys from model name (format: "training_keys_freeze_down" or "training_keys")
+    training_dataset_keys = []
+    model_name = settings.name
+    if '_freeze_down' in model_name:
+        training_name = model_name.replace('_freeze_down', '')
+    else:
+        training_name = model_name
+    if '&' in training_name:
+        training_dataset_keys = training_name.split('&')
+    else:
+        training_dataset_keys = [training_name]
+    
+    # Write CSV header
+    with open(csv_filename, 'w') as f:
+        f.write(f"{','.join(['name', 'pro', 'flag'])}\n")
+    
+    with torch.no_grad():
+        with tqdm(loader, unit='batch', mininterval=0.5) as tbatch:
+            tbatch.set_description(f'Validation')
+            for data_dict in tbatch:
+                data = data_dict['img'].to(device)
+                labels = data_dict['target'].to(device)
+                paths = data_dict['path']
+
+                scores = model(data).squeeze(1)
+
+                # Collect results
+                for score, label, path in zip(scores, labels, paths):
+                    score_val = score.item()
+                    label_val = label.item()
+                    
+                    all_scores.append(score_val)
+                    all_labels.append(label_val)
+                    all_paths.append(path)
+                    
+                    image_results.append({
+                        'path': path,
+                        'score': score_val,
+                        'label': label_val
+                    })
+                
+                # Write to CSV (maintain backward compatibility)
+                with open(csv_filename, 'a') as f:
+                    for score, label, path in zip(scores, labels, paths):
+                        f.write(f"{path}, {score.item()}, {label.item()}\n")
+
+    # Calculate metrics
+    all_scores = np.array(all_scores)
+    all_labels = np.array(all_labels)
+    
+    # Convert scores to predictions (threshold at 0, as used in train.py: y_pred > 0.0)
+    predictions = (all_scores > 0).astype(int)
+    
+    # Calculate overall metrics
+    total_accuracy = accuracy_score(all_labels, predictions)
+    
+    # TPR (True Positive Rate) = TP / (TP + FN) = accuracy on fake images (label==1)
+    fake_mask = all_labels == 1
+    if fake_mask.sum() > 0:
+        tpr = accuracy_score(all_labels[fake_mask], predictions[fake_mask])
+    else:
+        tpr = 0.0
+    
+    # TNR per dataset key (True Negative Rate) = TN / (TN + FP) = accuracy on real images (label==0)
+    tnr_per_dataset = {}
+    
+    # Calculate TNR on real images (label==0) in the test set
+    real_mask = all_labels == 0
+    if real_mask.sum() > 0:
+        # Overall TNR calculated on all real images in the test set
+        tnr = accuracy_score(all_labels[real_mask], predictions[real_mask])
+    else:
+        tnr = 0.0
+        
+        # Map TNR to training dataset keys (as shown in the example JSON structure)
+        # The TNR is calculated on the test set, but organized by training dataset keys
+        #for training_key in training_dataset_keys:
+        #    tnr_per_dataset[training_key] = overall_tnr
+    
+    # AUC calculation (needs probabilities, so we'll use sigmoid on scores)
+    if len(np.unique(all_labels)) > 1:  # Need both classes for AUC
+        # Apply sigmoid to convert scores to probabilities
+        probabilities = torch.sigmoid(torch.tensor(all_scores)).numpy()
+        auc = roc_auc_score(all_labels, probabilities)
+    else:
+        auc = 0.0
+    
+    execution_time = time.time() - start_time
+    
+    # Prepare metrics JSON
+    metrics = {
+        'TPR': float(tpr),
+        'TNR': float(tnr),
+        'Acc total': float(total_accuracy),
+        'AUC': float(auc),
+        'execution time': float(execution_time)
+    }
+    
+    # Write metrics JSON
+    with open(metrics_filename, 'w') as f:
+        json.dump(metrics, f, indent=2)
+    
+    # Write individual image results JSON
+    with open(image_results_filename, 'w') as f:
+        json.dump(image_results, f, indent=2)
+    
+    print(f'\nMetrics saved to {metrics_filename}')
+    print(f'Image results saved to {image_results_filename}')
+    print(f'\nMetrics:')
+    print(f'  TPR: {tpr:.4f}')
+    print(f'  TNR: {tnr:.4f}')
+    print(f'  Accuracy: {total_accuracy:.4f}')
+    print(f'  AUC: {auc:.4f}')
+    print(f'  Execution time: {execution_time:.2f} seconds')
+
+if __name__ == '__main__':
+    parser = get_parser()
+    parser = add_processing_arguments(parser)
+    settings = parser.parse_args()
+    
+    device = torch.device(settings.device if torch.cuda.is_available() else 'cpu')
+
+    test_dataloader = create_dataloader(settings, split='test')
+
+    model = create_architecture(settings.arch, pretrained=True, num_classes=1).to(device)
+    num_parameters = count_parameters(model)
+    print(f"Arch: {settings.arch} with #parameters {num_parameters}")
+    
+    load_path = f'./checkpoint/{settings.name}/weights/best.pt'
+    
+    print('loading the model from %s' % load_path)
+    model.load_state_dict(torch.load(load_path, map_location=device)['model'])
+    model.to(device)
+
+    test(test_dataloader, model, settings, device)

detectors/CLIP-D/train.py

@@ -0,0 +1,65 @@
+import os
+import tqdm
+from utils import TrainingModel, create_dataloader, EarlyStopping
+from sklearn.metrics import balanced_accuracy_score, roc_auc_score
+from utils.processing import add_processing_arguments
+from parser import get_parser
+
+if __name__ == "__main__":
+    parser = get_parser()
+    parser = add_processing_arguments(parser)
+
+    opt = parser.parse_args()
+
+    os.makedirs(os.path.join('checkpoint', opt.name,'weights'), exist_ok=True)
+
+    valid_data_loader = create_dataloader(opt, split="val")
+    train_data_loader = create_dataloader(opt, split="train")
+    print()
+    print("# validation batches = %d" % len(valid_data_loader))
+    print("#   training batches = %d" % len(train_data_loader))
+    model = TrainingModel(opt)
+    early_stopping = None
+    start_epoch = model.total_steps // len(train_data_loader)
+    print()
+
+    for epoch in range(start_epoch, opt.num_epoches+1):
+        if epoch > start_epoch:
+            # Training
+            pbar = tqdm.tqdm(train_data_loader)
+            for data in pbar:
+                loss = model.train_on_batch(data).item()
+                total_steps = model.total_steps
+                pbar.set_description(f"Train loss: {loss:.4f}")
+
+            # Save model
+            model.save_networks(epoch)
+
+        # Validation
+        print("Validation ...", flush=True)
+        y_true, y_pred, y_path = model.predict(valid_data_loader)
+        acc = balanced_accuracy_score(y_true, y_pred > 0.0)
+        auc = roc_auc_score(y_true, y_pred)
+        lr = model.get_learning_rate()
+        print("After {} epoches: val acc = {}; val auc = {}".format(epoch, acc, auc), flush=True)
+
+        # Early Stopping
+        if early_stopping is None:
+            early_stopping = EarlyStopping(
+                init_score=acc, patience=opt.earlystop_epoch,
+                delta=0.001, verbose=True,
+            )
+            print('Save best model', flush=True)
+            model.save_networks('best')
+        else:
+            if early_stopping(acc):
+                print('Save best model', flush=True)
+                model.save_networks('best')
+            if early_stopping.early_stop:
+                cont_train = model.adjust_learning_rate()
+                if cont_train:
+                    print("Learning rate dropped by 10, continue training ...", flush=True)
+                    early_stopping.reset_counter()
+                else:
+                    print("Early stopping.", flush=True)
+                    break

detectors/CLIP-D/utils/__init__.py

@@ -0,0 +1,52 @@
+'''                                        
+Copyright 2024 Image Processing Research Group of University Federico
+II of Naples ('GRIP-UNINA'). All rights reserved.
+                        
+Licensed under the Apache License, Version 2.0 (the "License");       
+you may not use this file except in compliance with the License. 
+You may obtain a copy of the License at                    
+                                           
+    http://www.apache.org/licenses/LICENSE-2.0
+                                                      
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,    
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.                         
+See the License for the specific language governing permissions and
+limitations under the License.
+''' 
+
+from .dataset import create_dataloader
+from .training import TrainingModel
+class EarlyStopping:
+    def __init__(self, init_score=None, patience=1, verbose=False, delta=0):
+        self.best_score = init_score
+        self.patience = patience
+        self.delta = delta
+        self.verbose = verbose
+        self.count_down = self.patience
+        self.early_stop = False
+
+    def __call__(self, score):
+        if self.best_score is None:
+            if self.verbose:
+                print(f'Score set to {score:.6f}.')
+            self.best_score = score
+            self.count_down = self.patience
+            return True
+        elif score <= self.best_score + self.delta:
+            self.count_down -= 1
+            if self.verbose:
+                print(f'EarlyStopping count_down: {self.count_down} on {self.patience}')
+            if self.count_down <= 0:
+                self.early_stop = True
+            return False
+        else:
+            if self.verbose:
+                print(f'Score increased from ({self.best_score:.6f} to {score:.6f}).')
+            self.best_score = score
+            self.count_down = self.patience
+            return True
+
+    def reset_counter(self):
+        self.count_down = self.patience
+        self.early_stop = False

detectors/CLIP-D/utils/dataset.py

@@ -0,0 +1,144 @@
+'''                                        
+Copyright 2024 Image Processing Research Group of University Federico
+II of Naples ('GRIP-UNINA'). All rights reserved.
+                        
+Licensed under the Apache License, Version 2.0 (the "License");       
+you may not use this file except in compliance with the License. 
+You may obtain a copy of the License at                    
+                                           
+    http://www.apache.org/licenses/LICENSE-2.0
+                                                      
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,    
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.                         
+See the License for the specific language governing permissions and
+limitations under the License.
+''' 
+
+import os
+import json
+import torch
+import bisect
+import numpy as np
+from torch.utils.data.sampler import WeightedRandomSampler, RandomSampler
+from torchvision import datasets
+from .processing import make_processing
+
+from PIL import Image, ImageFile
+ImageFile.LOAD_TRUNCATED_IMAGES = True
+
+def create_dataloader(opt, split=None):
+    if split == "train":
+        opt.split = 'train'
+        is_train=True
+
+    elif split == "val":
+        opt.split = 'val'
+        is_train=False
+    
+    elif split == "test":
+        opt.split = 'test'
+        is_train=False
+    
+    else:
+        raise ValueError(f"Unknown split {split}")
+
+    dataset = TrueFake_dataset(opt)
+
+    data_loader = torch.utils.data.DataLoader(
+        dataset,
+        batch_size=opt.batch_size,
+        shuffle=is_train,
+        num_workers=int(opt.num_threads),
+    )
+    return data_loader
+
+def parse_dataset(settings):
+    gen_keys = {
+        'gan1':['StyleGAN'],
+        'gan2':['StyleGAN2'],
+        'gan3':['StyleGAN3'],
+        'sd15':['StableDiffusion1.5'],
+        'sd2':['StableDiffusion2'],
+        'sd3':['StableDiffusion3'],
+        'sdXL':['StableDiffusionXL'],
+        'flux':['FLUX.1'],
+        'realFFHQ':['FFHQ'],
+        'realFORLAB':['FORLAB']
+    }
+
+    gen_keys['all'] =   [gen_keys[key][0] for key in gen_keys.keys()]
+    # gen_keys['gan'] =   [gen_keys[key][0] for key in gen_keys.keys() if 'gan'   in key]
+    # gen_keys['sd'] =    [gen_keys[key][0] for key in gen_keys.keys() if 'sd'    in key]
+    gen_keys['real'] =  [gen_keys[key][0] for key in gen_keys.keys() if 'real'  in key]
+
+    mod_keys = {
+        'pre':  ['PreSocial'],
+        'fb':   ['Facebook'],
+        'tl':   ['Telegram'],
+        'tw':   ['X'],
+    }
+
+    mod_keys['all'] = [mod_keys[key][0] for key in mod_keys.keys()]
+    mod_keys['shr'] = [mod_keys[key][0] for key in mod_keys.keys() if key in ['fb', 'tl', 'tw']]
+
+    need_real = (settings.split in ['train', 'val'] and not len([data for data in settings.data_keys.split('&') if 'real' in data.split(':')[0]]))
+
+    assert not need_real, 'Train task without real data, this will not get handeled automatically, terminating'
+
+    dataset_list = []
+    for data in settings.data_keys.split('&'):
+        gen, mod = data.split(':')
+        dataset_list.append({'gen':gen_keys[gen], 'mod':mod_keys[mod]})
+    
+    return dataset_list
+
+class TrueFake_dataset(datasets.DatasetFolder):
+    def __init__(self, settings):
+        self.data_root = settings.data_root
+        self.split = settings.split
+
+        with open(settings.split_file, "r") as f:
+            split_list = sorted(json.load(f)[self.split])
+        
+        dataset_list = parse_dataset(settings)
+        
+        self.samples = []
+        self.info = []
+        for dict in dataset_list:
+            generators = dict['gen']
+            modifiers = dict['mod']
+
+            for mod in modifiers:
+                for dataset_root, dataset_dirs, dataset_files in os.walk(os.path.join(self.data_root, mod), topdown=True, followlinks=True):
+                    if len(dataset_dirs):
+                        continue
+                    (label, gen, sub)  = f'{dataset_root}/'.replace(os.path.join(self.data_root, mod) + os.sep, '').split(os.sep)[:3][:3]
+                    
+                    if gen in generators:
+                        for filename in sorted(dataset_files):
+                            if os.path.splitext(filename)[1].lower() in ['.png', '.jpg', '.jpeg']:
+                                if self._in_list(split_list, os.path.join(gen, sub, os.path.splitext(filename)[0])):
+                                    self.samples.append(os.path.join(dataset_root, filename))
+                                    self.info.append((mod, label, gen, sub))
+
+        self.transform = make_processing(settings)
+        print(self.transform)
+
+    def _in_list(self, split, elem):
+        i = bisect.bisect_left(split, elem)
+        return i != len(split) and split[i] == elem
+    
+    def __len__(self):
+        return len(self.samples)
+    
+    def __getitem__(self, index):
+        path = self.samples[index]
+        mod, label, gen, sub = self.info[index]
+
+        sample = Image.open(path).convert('RGB')
+        sample = self.transform(sample)
+
+        target = 1.0 if label == 'Fake' else 0.0
+        
+        return {'img':sample, 'target':target, 'path':path}

detectors/CLIP-D/utils/processing.py

@@ -0,0 +1,133 @@
+'''                                        
+Copyright 2024 Image Processing Research Group of University Federico
+II of Naples ('GRIP-UNINA'). All rights reserved.
+                        
+Licensed under the Apache License, Version 2.0 (the "License");       
+you may not use this file except in compliance with the License. 
+You may obtain a copy of the License at                    
+                                           
+    http://www.apache.org/licenses/LICENSE-2.0
+                                                      
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,    
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.                         
+See the License for the specific language governing permissions and
+limitations under the License.
+''' 
+import torchvision.transforms.v2 as Tv2
+
+def make_processing(opt):
+    opt = parse_arguments(opt)
+    transforms_list = list()  # list of transforms
+
+    if opt.task == 'train':
+        transforms_aug = make_aug(opt)  # make data-augmentation transforms
+        if transforms_aug is not None:
+            transforms_list.append(transforms_aug)
+
+    transforms_post = make_post(opt)  # make post-data-augmentation transforms
+    if transforms_post is not None:
+        transforms_list.append(transforms_post)
+
+    transforms_list.append(make_normalize(opt))  # make normalization
+
+    t = Tv2.Compose(transforms_list)
+
+    return t
+
+
+def add_processing_arguments(parser):
+    # parser is an argparse.ArgumentParser
+    #
+    # ICASSP2023: --cropSize 96 --loadSize -1 --resizeSize -1 --norm_type resnet --resize_prob 0.2 --jitter_prob 0.8 --colordist_prob 0.2 --cutout_prob 0.2 --noise_prob 0.2 --blur_prob 0.5 --cmp_prob 0.5 --rot90_prob 1.0 --hpf_prob 0.0 --blur_sig 0.0,3.0 --cmp_method cv2,pil --cmp_qual 30,100 --resize_size 256 --resize_ratio 0.75
+    # ICME2021  : --cropSize 96 --loadSize -1 --resizeSize -1 --norm_type resnet --resize_prob 0.0 --jitter_prob 0.0 --colordist_prob 0.0 --cutout_prob 0.0 --noise_prob 0.0 --blur_prob 0.5 --cmp_prob 0.5 --rot90_prob 1.0 --hpf_prob 0.0 --blur_sig 0.0,3.0 --cmp_method cv2,pil --cmp_qual 30,100
+    #
+
+    parser.add_argument("--resizeSize", type=int, default=224, help="scale images to this size post augumentation")
+
+    # data-augmentation probabilities
+    parser.add_argument("--resize_prob", type=float, default=0.0)
+    parser.add_argument("--cmp_prob", type=float, default=0.0)
+
+    # data-augmentation parameters
+    parser.add_argument("--cmp_qual", default="75")
+    parser.add_argument("--resize_size", type=int, default=256)
+    parser.add_argument("--resize_ratio", type=float, default=1.0)
+
+    # other
+    parser.add_argument("--norm_type", type=str, default="clip") 
+
+    return parser
+
+
+def parse_arguments(opt):
+    if not isinstance(opt.cmp_qual, list):
+        opt.cmp_qual = [int(s) for s in opt.cmp_qual.split(",")]
+    return opt
+
+
+def make_post(opt):
+    transforms_list = list()
+    if opt.resizeSize > 0:
+        print("\nUsing Post Resizing\n")
+        transforms_list.append(Tv2.Resize(opt.resizeSize, interpolation=Tv2.InterpolationMode.BICUBIC))
+        transforms_list.append(Tv2.CenterCrop((opt.resizeSize, opt.resizeSize)))
+
+    if len(transforms_list) == 0:
+        return None
+    else:
+        return Tv2.Compose(transforms_list)
+
+
+def make_aug(opt):
+    # AUG
+    transforms_list_aug = list()
+
+    if (opt.resize_size > 0) and (opt.resize_prob > 0):  # opt.resized_ratio
+        transforms_list_aug.append(
+            Tv2.RandomApply(
+                [
+                    Tv2.RandomResizedCrop(
+                        size=opt.resize_size,
+                        scale=(5/8, 1.0),
+                        ratio=(opt.resize_ratio, 1.0 / opt.resize_ratio),
+                    )
+                ],
+                opt.resize_prob,
+            )
+        )
+
+    if opt.cmp_prob > 0:
+        transforms_list_aug.append(
+            Tv2.RandomApply(
+                [
+                    Tv2.JPEG(
+                        opt.cmp_qual
+                    )
+                ],
+                opt.cmp_prob,
+            )
+        )
+
+    if len(transforms_list_aug) > 0:
+        return Tv2.Compose(transforms_list_aug)
+    else:
+        return None
+
+
+def make_normalize(opt):
+    transforms_list = list()
+
+    if opt.norm_type == "clip":
+        print("normalize CLIP")
+        transforms_list.append(Tv2.ToTensor())
+        transforms_list.append(
+            Tv2.Normalize(
+                mean=(0.48145466, 0.4578275, 0.40821073),
+                std=(0.26862954, 0.26130258, 0.27577711),
+            )
+        )
+    else:
+        assert False
+
+    return Tv2.Compose(transforms_list)

detectors/CLIP-D/utils/training.py

@@ -0,0 +1,105 @@
+'''                                        
+Copyright 2024 Image Processing Research Group of University Federico
+II of Naples ('GRIP-UNINA'). All rights reserved.
+                        
+Licensed under the Apache License, Version 2.0 (the "License");       
+you may not use this file except in compliance with the License. 
+You may obtain a copy of the License at                    
+                                           
+    http://www.apache.org/licenses/LICENSE-2.0
+                                                      
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,    
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.                         
+See the License for the specific language governing permissions and
+limitations under the License.
+''' 
+
+import os
+import torch
+import numpy as np
+import tqdm
+from networks import create_architecture, count_parameters
+
+class TrainingModel(torch.nn.Module):
+
+    def __init__(self, opt):
+        super(TrainingModel, self).__init__()
+
+        self.opt = opt
+        self.total_steps = 0
+        self.save_dir = os.path.join('checkpoint', opt.name,'weights')
+        self.device = torch.device(opt.device if torch.cuda.is_available() else 'cpu')
+
+        self.model = create_architecture(opt.arch, pretrained=True,  num_classes=1)
+        num_parameters = count_parameters(self.model)
+        print(f"Arch: {opt.arch} with #trainable {num_parameters}")
+
+        self.loss_fn = torch.nn.BCEWithLogitsLoss().to(self.device)
+        parameters = filter(lambda p: p.requires_grad, self.model.parameters())
+        self.optimizer = torch.optim.Adam(parameters, lr=opt.lr, betas=(opt.beta1, 0.999), weight_decay=opt.weight_decay)
+
+        self.model.to(self.device)
+
+    def adjust_learning_rate(self, min_lr=1e-6):
+        for param_group in self.optimizer.param_groups:
+            param_group["lr"] /= 10.0
+            if param_group["lr"] < min_lr:
+                return False
+        return True
+
+    def get_learning_rate(self):
+        for param_group in self.optimizer.param_groups:
+            return param_group["lr"]
+
+    def train_on_batch(self, data):
+        self.total_steps += 1
+        self.model.train()
+        input = data['img'].to(self.device)
+        label = data['target'].to(self.device).float()
+        output = self.model(input)
+        if len(output.shape) == 4:
+            ss = output.shape
+            loss = self.loss_fn(
+                output,
+                label[:, None, None, None].repeat(
+                    (1, int(ss[1]), int(ss[2]), int(ss[3]))
+                ),
+            )
+        else:
+            loss = self.loss_fn(output.squeeze(1), label)
+        self.optimizer.zero_grad()
+        loss.backward()
+        self.optimizer.step()
+        return loss.cpu()
+
+    def save_networks(self, epoch):
+        save_filename = f'{epoch}.pt'
+        save_path = os.path.join(self.save_dir, save_filename)
+
+        # serialize model and optimizer to dict
+        state_dict = {
+            'model': self.model.state_dict(),
+            'optimizer': self.optimizer.state_dict(),
+            'total_steps': self.total_steps,
+        }
+
+        torch.save(state_dict, save_path)
+
+    def predict(self, data_loader):
+        model = self.model.eval()
+        with torch.no_grad():
+            y_true, y_pred, y_path = [], [], []
+            for data in tqdm.tqdm(data_loader):
+                img = data['img']
+                label = data['target'].cpu().numpy()
+                paths = list(data['path'])
+                out_tens = model(img.to(self.device)).cpu().numpy()[:, -1]
+                assert label.shape == out_tens.shape
+
+                y_pred.extend(out_tens.tolist())
+                y_true.extend(label.tolist())
+                y_path.extend(paths)
+
+        y_true, y_pred = np.array(y_true), np.array(y_pred)
+        return y_true, y_pred, y_path

detectors/NPR/README.md

@@ -0,0 +1,49 @@
+# Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection
+
+[![Official Github Repo](https://img.shields.io/badge/Github%20page-222222.svg?style=for-the-badge&logo=github)](https://github.com/chuangchuangtan/NPR-DeepfakeDetection)
+[![Paper](https://img.shields.io/badge/-arXiv-B31B1B.svg?style=for-the-badge)](https://arxiv.org/abs/2312.10461)
+
+Original Paper:
+[Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection](https://arxiv.org/abs/2312.10461).
+
+Authors: Chuangchuang Tan, Huan Liu, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, Yunchao Wei.
+
+## Abstract
+
+Recently, the proliferation of highly realistic synthetic
+images, facilitated through a variety of GANs and Diffu-
+sions, has significantly heightened the susceptibility to mis-
+use. While the primary focus of deepfake detection has tra-
+ditionally centered on the design of detection algorithms,
+an investigative inquiry into the generator architectures has
+remained conspicuously absent in recent years. This paper
+contributes to this lacuna by rethinking the architectures of
+CNN-based generator, thereby establishing a generalized
+representation of synthetic artifacts. Our findings illumi-
+nate that the up-sampling operator can, beyond frequency-
+based artifacts, produce generalized forgery artifacts. In
+particular, the local interdependence among image pixels
+caused by upsampling operators is significantly demon-
+strated in synthetic images generated by GAN or diffusion.
+Building upon this observation, we introduce the concept of
+Neighboring Pixel Relationships(NPR) as a means to cap-
+ture and characterize the generalized structural artifacts
+stemming from up-sampling operations. A comprehensive
+analysis is conducted on an open-world dataset, comprising
+samples generated by 28 distinct generative models. This
+analysis culminates in the establishment of a novel state-of-
+the-art performance, showcasing a remarkable 11.6% im-
+provement over existing methods
+
+
+## Please Cite
+
+```
+@inproceedings{tan2024rethinking,
+title={Rethinking the up-sampling operations in cnn-based generative network for generalizable deepfake detection},
+author={Tan, Chuangchuang and Zhao, Yao and Wei, Shikui and Gu, Guanghua and Liu, Ping and Wei, Yunchao},
+booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+pages={28130--28139},
+year={2024}
+}
+```

detectors/NPR/checkpoint/pretrained/weights/best.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb76594b7304c23249206ab2ac434dbd153c4114660ecc11a23eb82848f0721d
+size 5831180

detectors/NPR/data/__init__.py

@@ -0,0 +1,201 @@
+import torch
+import numpy as np
+from torch.utils.data.sampler import WeightedRandomSampler
+
+from .datasets import dataset_folder
+from torchvision.datasets import DatasetFolder
+import json
+import bisect
+from PIL import Image
+import torchvision.transforms.v2 as Tv2
+'''
+def get_dataset(opt):
+    dset_lst = []
+    for cls in opt.classes:
+        root = opt.dataroot + '/' + cls
+        dset = dataset_folder(opt, root)
+        dset_lst.append(dset)
+    return torch.utils.data.ConcatDataset(dset_lst)
+'''
+
+import os
+# def get_dataset(opt):
+#     classes = os.listdir(opt.dataroot) if len(opt.classes) == 0 else opt.classes
+#     if '0_real' not in classes or '1_fake' not in classes:
+#         dset_lst = []
+#         for cls in classes:
+#             root = opt.dataroot + '/' + cls
+#             dset = dataset_folder(opt, root)
+#             dset_lst.append(dset)
+#         return torch.utils.data.ConcatDataset(dset_lst)
+#     return dataset_folder(opt, opt.dataroot)
+
+# def get_bal_sampler(dataset):
+#     targets = []
+#     for d in dataset.datasets:
+#         targets.extend(d.targets)
+
+#     ratio = np.bincount(targets)
+#     w = 1. / torch.tensor(ratio, dtype=torch.float)
+#     sample_weights = w[targets]
+#     sampler = WeightedRandomSampler(weights=sample_weights,
+#                                     num_samples=len(sample_weights))
+#     return sampler
+
+
+# def create_dataloader(opt):
+#     shuffle = not opt.serial_batches if (opt.isTrain and not opt.class_bal) else False
+#     dataset = get_dataset(opt)
+#     sampler = get_bal_sampler(dataset) if opt.class_bal else None
+
+#     data_loader = torch.utils.data.DataLoader(dataset,
+#                                               batch_size=opt.batch_size,
+#                                               shuffle=shuffle,
+#                                               sampler=sampler,
+#                                               num_workers=int(opt.num_threads))
+#     return data_loader
+
+
+def parse_dataset(settings):
+    gen_keys = {
+        'gan1':['StyleGAN'],
+        'gan2':['StyleGAN2'],
+        'gan3':['StyleGAN3'],
+        'sd15':['StableDiffusion1.5'],
+        'sd2':['StableDiffusion2'],
+        'sd3':['StableDiffusion3'],
+        'sdXL':['StableDiffusionXL'],
+        'flux':['FLUX.1'],
+        'realFFHQ':['FFHQ'],
+        'realFORLAB':['FORLAB']
+    }
+
+    gen_keys['all'] =   [gen_keys[key][0] for key in gen_keys.keys()]
+    # gen_keys['gan'] =   [gen_keys[key][0] for key in gen_keys.keys() if 'gan'   in key]
+    # gen_keys['sd'] =    [gen_keys[key][0] for key in gen_keys.keys() if 'sd'    in key]
+    gen_keys['real'] =  [gen_keys[key][0] for key in gen_keys.keys() if 'real'  in key]
+
+    mod_keys = {
+        'pre':  ['PreSocial'],
+        'fb':   ['Facebook'],
+        'tl':   ['Telegram'],
+        'tw':   ['X'],
+    }
+
+    mod_keys['all'] = [mod_keys[key][0] for key in mod_keys.keys()]
+    mod_keys['shr'] = [mod_keys[key][0] for key in mod_keys.keys() if key in ['fb', 'tl', 'tw']]
+
+    need_real = (settings.task == 'train' and not len([data.split(':')[0] for data in settings.data_keys.split('&') if 'real' in data.split(':')[0]]))
+
+    assert not need_real, 'Train task without real data, this will not get handeled automatically, terminating'
+
+    dataset_list = []
+    for data in settings.data_keys.split('&'):
+        gen, mod = data.split(':')
+        dataset_list.append({'gen':gen_keys[gen], 'mod':mod_keys[mod]})
+    
+    return dataset_list
+
+class TrueFake_dataset(DatasetFolder):
+    def __init__(self, settings):
+        self.data_root = settings.data_root
+        self.split = settings.split
+
+        with open(settings.split_file, "r") as f:
+            split_list = sorted(json.load(f)[self.split])
+        
+        dataset_list = parse_dataset(settings)
+        
+        self.samples = []
+        self.info = []
+        for dict in dataset_list:
+            generators = dict['gen']
+            modifiers = dict['mod']
+
+            for mod in modifiers:
+                for dataset_root, dataset_dirs, dataset_files in os.walk(os.path.join(self.data_root, mod), topdown=True, followlinks=True):
+                    if len(dataset_dirs):
+                        continue
+
+                    (label, gen, sub)  = f'{dataset_root}/'.replace(os.path.join(self.data_root, mod) + os.sep, '').split(os.sep)[:3]
+                    
+                    if gen in generators:
+                        for filename in sorted(dataset_files):
+                            if os.path.splitext(filename)[1].lower() in ['.png', '.jpg', '.jpeg']:
+                                if self._in_list(split_list, os.path.join(gen, sub, os.path.splitext(filename)[0])):
+                                    self.samples.append(os.path.join(dataset_root, filename))
+                                    self.info.append((mod, label, gen, sub))
+
+        if settings.isTrain:
+            crop_func = Tv2.RandomCrop(settings.cropSize)
+        elif settings.no_crop:
+            crop_func = Tv2.Identity()
+        else:
+            crop_func = Tv2.CenterCrop(settings.cropSize)
+
+        if settings.isTrain and not settings.no_flip:
+            flip_func = Tv2.RandomHorizontalFlip()
+        else:
+            flip_func = Tv2.Identity()
+
+        if not settings.isTrain and settings.no_resize:
+            rz_func = Tv2.Identity()
+        else:
+            rz_func = Tv2.Resize((settings.loadSize, settings.loadSize))
+
+        self.transform = Tv2.Compose([
+                            rz_func,
+                            crop_func,
+                            flip_func,
+                            Tv2.ToTensor(),
+                            Tv2.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
+                        ])
+    
+                    
+    def _in_list(self, split, elem):
+        i = bisect.bisect_left(split, elem)
+        return i != len(split) and split[i] == elem
+    
+    def __len__(self):
+        return len(self.samples)
+    
+    def __getitem__(self, index):
+        path = self.samples[index]
+        mod, label, gen, sub = self.info[index]
+
+        image = Image.open(path).convert('RGB')
+        sample = self.transform(image)
+
+        target = 1.0 if label == 'Fake' else 0.0
+        
+        return sample, target, path
+    
+
+def create_dataloader(settings, split=None):
+    if split == "train":
+        settings.split = 'train'
+        is_train=True
+
+    elif split == "val":
+        settings.split = 'val'
+        settings.batch_size = settings.batch_size//4
+        is_train=False
+    
+    elif split == "test":
+        settings.split = 'test'
+        settings.batch_size = settings.batch_size//4
+        is_train=False
+    
+    else:
+        raise ValueError(f"Unknown split {split}")
+
+    dataset = TrueFake_dataset(settings)
+
+    data_loader = torch.utils.data.DataLoader(
+        dataset,
+        batch_size=settings.batch_size,
+        num_workers=int(settings.num_threads),
+        shuffle = is_train,
+        collate_fn=None,
+    )
+    return data_loader

detectors/NPR/data/datasets.py

@@ -0,0 +1,139 @@
+import cv2
+import numpy as np
+import torchvision.datasets as datasets
+import torchvision.transforms as transforms
+import torchvision.transforms.functional as TF
+from random import random, choice
+from io import BytesIO
+from PIL import Image
+from PIL import ImageFile
+from scipy.ndimage.filters import gaussian_filter
+from torchvision.transforms import InterpolationMode
+
+ImageFile.LOAD_TRUNCATED_IMAGES = True
+
+def dataset_folder(opt, root):
+    if opt.mode == 'binary':
+        return binary_dataset(opt, root)
+    if opt.mode == 'filename':
+        return FileNameDataset(opt, root)
+    raise ValueError('opt.mode needs to be binary or filename.')
+
+
+def binary_dataset(opt, root):
+    if opt.isTrain:
+        crop_func = transforms.RandomCrop(opt.cropSize)
+    elif opt.no_crop:
+        crop_func = transforms.Lambda(lambda img: img)
+    else:
+        crop_func = transforms.CenterCrop(opt.cropSize)
+
+    if opt.isTrain and not opt.no_flip:
+        flip_func = transforms.RandomHorizontalFlip()
+    else:
+        flip_func = transforms.Lambda(lambda img: img)
+    if not opt.isTrain and opt.no_resize:
+        rz_func = transforms.Lambda(lambda img: img)
+    else:
+        # rz_func = transforms.Lambda(lambda img: custom_resize(img, opt))
+        rz_func = transforms.Resize((opt.loadSize, opt.loadSize))
+
+    dset = datasets.ImageFolder(
+            root,
+            transforms.Compose([
+                rz_func,
+                # transforms.Lambda(lambda img: data_augment(img, opt)),
+                crop_func,
+                flip_func,
+                transforms.ToTensor(),
+                transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
+            ]))
+    return dset
+
+
+class FileNameDataset(datasets.ImageFolder):
+    def name(self):
+        return 'FileNameDataset'
+
+    def __init__(self, opt, root):
+        self.opt = opt
+        super().__init__(root)
+
+    def __getitem__(self, index):
+        # Loading sample
+        path, target = self.samples[index]
+        return path
+
+
+def data_augment(img, opt):
+    img = np.array(img)
+
+    if random() < opt.blur_prob:
+        sig = sample_continuous(opt.blur_sig)
+        gaussian_blur(img, sig)
+
+    if random() < opt.jpg_prob:
+        method = sample_discrete(opt.jpg_method)
+        qual = sample_discrete(opt.jpg_qual)
+        img = jpeg_from_key(img, qual, method)
+
+    return Image.fromarray(img)
+
+
+def sample_continuous(s):
+    if len(s) == 1:
+        return s[0]
+    if len(s) == 2:
+        rg = s[1] - s[0]
+        return random() * rg + s[0]
+    raise ValueError("Length of iterable s should be 1 or 2.")
+
+
+def sample_discrete(s):
+    if len(s) == 1:
+        return s[0]
+    return choice(s)
+
+
+def gaussian_blur(img, sigma):
+    gaussian_filter(img[:,:,0], output=img[:,:,0], sigma=sigma)
+    gaussian_filter(img[:,:,1], output=img[:,:,1], sigma=sigma)
+    gaussian_filter(img[:,:,2], output=img[:,:,2], sigma=sigma)
+
+
+def cv2_jpg(img, compress_val):
+    img_cv2 = img[:,:,::-1]
+    encode_param = [int(cv2.IMWRITE_JPEG_QUALITY), compress_val]
+    result, encimg = cv2.imencode('.jpg', img_cv2, encode_param)
+    decimg = cv2.imdecode(encimg, 1)
+    return decimg[:,:,::-1]
+
+
+def pil_jpg(img, compress_val):
+    out = BytesIO()
+    img = Image.fromarray(img)
+    img.save(out, format='jpeg', quality=compress_val)
+    img = Image.open(out)
+    # load from memory before ByteIO closes
+    img = np.array(img)
+    out.close()
+    return img
+
+
+jpeg_dict = {'cv2': cv2_jpg, 'pil': pil_jpg}
+def jpeg_from_key(img, compress_val, key):
+    method = jpeg_dict[key]
+    return method(img, compress_val)
+
+
+# rz_dict = {'bilinear': Image.BILINEAR,
+           # 'bicubic': Image.BICUBIC,
+           # 'lanczos': Image.LANCZOS,
+           # 'nearest': Image.NEAREST}
+rz_dict = {'bilinear': InterpolationMode.BILINEAR,
+           'bicubic': InterpolationMode.BICUBIC,
+           'lanczos': InterpolationMode.LANCZOS,
+           'nearest': InterpolationMode.NEAREST}
+def custom_resize(img, opt):
+    interp = sample_discrete(opt.rz_interp)
+    return TF.resize(img, (opt.loadSize,opt.loadSize), interpolation=rz_dict[interp])

detectors/NPR/detect.py

@@ -0,0 +1,106 @@
+import os
+import sys
+import time
+import yaml
+import torch
+from PIL import Image
+import torchvision.transforms as transforms
+import argparse
+
+# Add project root to path for imports
+project_root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+sys.path.append(project_root)
+from support.detect_utils import format_result, save_result, get_device
+import networks.resnet as resnet
+from networks.resnet import resnet50
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(description='NPR single image detector')
+    parser.add_argument('--image', type=str, required=True, help='Path to input image')
+    parser.add_argument('--model', type=str, default='checkpoint/best.pt', help='Path to model checkpoint')
+    parser.add_argument('--output', type=str, help='Path to save detection result JSON')
+    parser.add_argument('--device', type=str, help='Device to run on (e.g., cuda:0, cuda:1, cpu)')
+    parser.add_argument('--config', type=str, default='configs/npr.yaml', help='Path to config file')
+    return parser.parse_args()
+
+def load_config(config_path):
+    """Load configuration from YAML file."""
+    with open(config_path, 'r') as f:
+        return yaml.safe_load(f)
+
+def load_image(image_path, size=224):
+    """Load and preprocess image."""
+    if not os.path.exists(image_path):
+        raise FileNotFoundError(f"Image not found: {image_path}")
+        
+    image = Image.open(image_path).convert('RGB')
+    transform = transforms.Compose([
+        transforms.Resize(size),
+        transforms.CenterCrop(size),
+        transforms.ToTensor(),
+        transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
+    ])
+    return transform(image).unsqueeze(0)
+
+def main():
+    args = parse_args()
+    
+    # Setup device
+    device = torch.device(args.device) if args.device else get_device()
+    
+        # Load model
+    try:
+        # Initialize model
+        model = resnet50(num_classes=1).to(device)
+        load_path = f'./detectors/NPR/checkpoint/{args.model}/weights/best.pt'
+        
+        if not os.path.exists(load_path):
+            raise FileNotFoundError(f"Model weights not found at: {load_path}")
+            
+        checkpoint = torch.load(load_path, map_location=device)
+
+        model.load_state_dict(checkpoint, strict=True)
+
+        model.eval()
+    except Exception as e:
+        print(f"Error loading model: {e}")
+        return
+
+    # Load and preprocess image
+    try:
+        image_tensor = load_image(args.image).to(device)
+    except Exception as e:
+        print(f"Error loading image: {e}")
+        return
+
+    # Run detection
+    start_time = time.time()
+    with torch.no_grad():
+        try:
+            score = model(image_tensor)
+            prediction = torch.sigmoid(score)
+            
+            confidence = prediction.item()
+            
+            result = format_result(
+                'fake' if confidence>0.5 else 'real',
+                confidence,
+                time.time() - start_time
+            )
+            
+            # Print result
+            print(f"Prediction: {result['prediction']}")
+            print(f"Confidence: {result['confidence']:.4f}")
+            print(f"Time: {result['elapsed_time']:.3f}s")
+            
+            # Save result if output path provided
+            if args.output:
+                save_result(result, args.output)
+                
+        except Exception as e:
+            print(f"Error during detection: {e}")
+            return
+
+if __name__ == '__main__':
+    main()