Aditya11031/deepfake-detector-models

Deepfake Detector Models

A Hugging Face model repository organizing deepfake-detection model artifacts into three architecture families: CNN, single-channel Vision Transformer (ViT), and nine-channel ViT with an auxiliary forensic agent.

The repository focuses on exploring complementary signals for manipulated-image detection: spatial appearance, frequency-domain patterns, and noise-residual evidence.

Model-card scope: The diagrams below document the architecture specifications supplied for this project. They are not proof that every component is implemented, trained, or validated. Consult the individual model directories for the files actually available.

Repository layout

deepfake-detector-models/
├── CNN/
├── VIT/
├── VIT-9-Channel/
├── images/
│   ├── 01-unified-preprocessing-pipeline.png
│   ├── 02-single-channel-vit.png
│   ├── 03-nine-channel-vit-forensic-agent.png
│   └── 04-tri-branch-multimodal-cnn.png
├── .gitattributes
└── README.md

• CNN/ — artifacts for the tri-branch multimodal convolutional architecture.
• VIT/ — artifacts for a single-channel Vision Transformer.
• VIT-9-Channel/ — artifacts for a nine-channel Vision Transformer and the proposed auxiliary forensic scoring path.
• images/ — architecture diagrams referenced in this model card.

Folder names describe the intended model family. Check each folder for the actual checkpoint, configuration, and supporting files; do not assume a particular file format is present.

At a glance

Model family	Input design	Feature extraction	Output design
CNN	RGB, FFT, and SRM representations	ConvNeXt-Tiny, ResNet-34, modified ResNet-18	Gated feature fusion and sigmoid score
VIT	One channel at 224 × 224	ViT-Small/16	Classification score
VIT-9-Channel	Nine channels at 224 × 224	ViT-Small/16 plus auxiliary forensic MLP	Fused score, subject to validation

1. Unified preprocessing pipeline

The intended workflow is:

1. Validate image files and labels.
2. Standardize image dimensions and define consistent channel/value-range handling.
3. Apply augmentation to the training split only, according to the training configuration.
4. Generate the representations required by the selected model: RGB, FFT, and/or SRM/noise residuals.
5. Apply model-specific normalization and arrange the tensors in the expected shape.

The exact resizing, normalization, augmentation, FFT implementation, and SRM filters must match the configuration used to train the corresponding checkpoint.

2. CNN — tri-branch multimodal architecture

This design extracts features from three complementary representations in parallel.

Branch	Representation	Backbone	Specified feature size
RGB	Original color image	ConvNeXt-Tiny	512-D
FFT	Frequency-domain representation	ResNet-34	512-D
SRM	Noise-residual representation	Modified ResNet-18	512-D

The three vectors are concatenated into a 1,536-dimensional representation. A gated fusion module projects this representation to 512 dimensions, followed by the proposed classification head:

512 → 256 → 64 → 1 → sigmoid

Design motivation: Spatial appearance, frequency patterns, and noise residuals may provide different evidence about image manipulation. Their usefulness must be established by evaluation and ablation testing.

3. VIT — single-channel Vision Transformer

The single-channel configuration processes one representation per forward pass. The intended input can be a grayscale image, one FFT-derived channel, or one SRM/noise channel, depending on the model configuration.

Component	Specification
Input	1 × 224 × 224
Patch size	16 × 16
Patch grid	14 × 14 = 196 patches
CLS token	1 additional token
Sequence length	197 tokens
Embedding dimension	384
Transformer blocks	12
Attention heads	6
Prediction head	Linear layer and sigmoid score

The patch embeddings and CLS token are processed by the transformer encoder. The CLS representation is passed to the classification head to produce a real/fake score.

These specifications describe the intended ViT-Small/16 design. Confirm the implementation's input stem, pretrained weights, pooling strategy, and classifier before assuming exact compatibility.

4. VIT-9-Channel — multimodal ViT with forensic agent

The proposed nine-channel input is formed by concatenating three representations:

Input representation	Channels
RGB	3
FFT	3
SRM / noise residual	3
Total	9

The combined tensor has the intended shape 9 × 224 × 224. The ViT branch generates a classification score, while an auxiliary multilayer perceptron processes nine forensic features.

The specified MLP dimensions are:

9 → 32 → 16 → 1

The proposed score fusion is:

$$p_{\text{final}}=\alpha p_{\text{ViT}}+(1-\alpha )sp\_\{\backslash text\{final\}\}\; =\; \backslash alpha\; p\_\{\backslash text\{ViT\}\}\; +\; (1-\backslash alpha)s$$

where:

• $p_{\text{ViT}}$ is the score from the ViT classifier.
• $s$ is the auxiliary forensic score.
• $\alpha$ controls the contribution of each score. This expression is a proposed fusion rule, not evidence of calibrated probability. Before use, document the nine features, the range and meaning of (s), how (\alpha) is selected, and whether the fused score is calibrated on held-out data. If the scores are not on compatible scales, a weighted average may be misleading.

Intended use

These artifacts are intended for research and experimentation involving:

• Deepfake and manipulated-image classification.
• Comparisons between convolutional and transformer-based approaches.
• Spatial, frequency-domain, and noise-residual representations.
• Multimodal feature fusion.
• Evaluation of auxiliary forensic features.

They should not be treated as a standalone forensic verdict or as definitive proof of authenticity.

Evaluation

No performance figures are claimed in this model card. Evaluate each available checkpoint on held-out data before reporting results.

Recommended metrics:

• Accuracy, precision, recall, and F1-score.
• ROC-AUC and confusion matrix.
• False-positive and false-negative rates.
• Cross-dataset generalization and robustness to compression or resizing.
• Ablations comparing RGB-only, FFT-only, SRM-only, multimodal fusion, and the auxiliary forensic path where applicable.

Keep test data separate from training and model selection. Report the evaluation dataset, split strategy, decision threshold, preprocessing, and checkpoint version with every result.

Limitations

• Deepfake detectors can fail on manipulation methods or datasets not represented during training.
• Compression, resizing, post-processing, and distribution shift can alter useful image cues.
• Dataset leakage or source-specific artifacts can inflate evaluation scores.
• FFT and SRM features depend on their exact implementation and normalization.
• A sigmoid output is not automatically a calibrated probability.
• The nine-channel model requires a compatible nine-channel input stem; a standard three-channel ViT checkpoint may not accept this input without an appropriate architectural adaptation.

Files and inference

See CNN/, VIT/, and VIT-9-Channel/ for the artifacts currently published. This README intentionally does not prescribe a loading command because the actual filenames, serialization formats, and inference interfaces must be verified from those directories.

Before publishing an inference example, document:

1. The checkpoint and configuration filename.
2. The required preprocessing and expected tensor shape.
3. The label mapping and score interpretation.
4. The software dependencies and tested environment.
5. A reproducible inference example.

Citation

If you use these artifacts, cite this Hugging Face repository and identify the exact model directory, revision, and checkpoint used. Add a publication citation if a paper describing the work becomes available.