kricko commited on
Commit
ceab863
·
verified ·
1 Parent(s): c8a49c7

Add clean Model Card and Inference Documentation

Browse files
Files changed (1) hide show
  1. README.md +84 -0
README.md ADDED
@@ -0,0 +1,84 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Complete Multi-Task Image Auditor - Model Card
2
+
3
+ This document provides a quick overview and instruction guide on how to load and run inference on the trained multi-task adversarial image auditor model.
4
+
5
+ ## Model Description
6
+ The Complete Multi-Task Auditor is based on a `ResNet101` backbone augmented with a text encoder and specialized prediction heads. It is designed to evaluate images against six safety categories while providing interpretability metrics like seam quality and text faithfulness.
7
+
8
+ ### Output Metrics
9
+ - **Global Safety Score:** Overall safety probability (1.0 = Safe, 0.0 = Unsafe).
10
+ - **Category Probabilities:** Softmax probabilities across 6 classes: `Safe`, `Violence`, `Sexual`, `Hate`, `Illegal Activity`, and `Sensitive IP`.
11
+ - **Faithfulness Score:** Evaluates how faithfully the image represents the given text prompt.
12
+ - **Seam Quality:** Detects inpainting or generation artifacts (0.0 = Poor, 1.0 = flawless).
13
+ - **Binary Map:** Adversarial/hotspot activation heatmap (numpy array).
14
+
15
+ ---
16
+
17
+ ## Quick Start: How to Use the Model
18
+
19
+ ### 1. Requirements
20
+ Ensure you have the required dependencies and the tokenization/model definition loaded. You need the `auditor_inference.py` script accessible to import the architecture.
21
+
22
+ ```bash
23
+ pip install torch torchvision numpy Pillow
24
+ ```
25
+
26
+ ### 2. Loading the Model Weights
27
+ Once you or your team have downloaded the model from the Hugging Face repository, you can instantiate the architecture and load the checkpoint.
28
+
29
+ ```python
30
+ import torch
31
+ from PIL import Image
32
+
33
+ # Import the model architecture and required globals from the standalone script
34
+ from auditor_inference import CompleteMultiTaskAuditor, SimpleTokenizer, inference
35
+
36
+ # Initialize the model (Ensure NUM_CLASSES=6 matches your training)
37
+ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
38
+ tokenizer = SimpleTokenizer(vocab_size=50000)
39
+
40
+ model = CompleteMultiTaskAuditor(num_classes=6, vocab_size=50000)
41
+ model.to(device)
42
+
43
+ # Load the trained weights downloaded from HF
44
+ checkpoint_path = "complete_auditor_best.pth"
45
+ model.load_state_dict(torch.load(checkpoint_path, map_location=device))
46
+
47
+ # Set to evaluation mode
48
+ model.eval()
49
+ print("Model loaded successfully!")
50
+ ```
51
+
52
+ ### 3. Running Inference
53
+ You can use the built-in `inference` function added to the standalone script to easily benchmark and extract all metrics.
54
+
55
+ ```python
56
+ # Load your image
57
+ image_path = "path/to/your/test_image.jpg"
58
+ image = Image.open(image_path).convert("RGB")
59
+
60
+ # Provide the prompt used to generate the image (if applicable)
61
+ prompt = "A photo of..."
62
+
63
+ # Run the inference function
64
+ results = inference(model=model, tokenizer=tokenizer, image=image, prompt=prompt)
65
+
66
+ # Print the metrics
67
+ print("\n--- Auditor Results ---")
68
+ print(f"Global Safety Score: {results['global_safety_score']:.4f}")
69
+ print(f"Seam Quality: {results['seam_quality']:.4f}")
70
+ print(f"Faithfulness Score: {results['faithfulness_score']:.4f}")
71
+
72
+ print("\n--- Category Probabilities ---")
73
+ for category, prob in results['category_probabilities'].items():
74
+ print(f"{category:>16}: {prob:.4f}")
75
+
76
+ # The binary heatmap mask is available for visualization
77
+ heatmap_array = results['binary_map']
78
+ ```
79
+
80
+ ## Interpreting Results
81
+ * **Safety Score > 0.5**: Image is generally evaluated as safe.
82
+ * **Category High Probability**: The model strongly suspects the image contains content from the respective category.
83
+ * **Seam Quality < 0.5**: Indicates strong probability of visual anomalies, deepfake seams, or poor image quality.
84
+ * **Faithfulness**: Helps determine if the adversarial content was strictly directed by the user's prompt or originated spontaneously.