LTF-v3.3 / README.md

Use non-overlapping full blocks with boundary-only completion

a2a2e29 verified 1 day ago

6.67 kB

---
language:
- en
library_name: pytorch
pipeline_tag: image-classification
tags:
- ltf
- ai-generated-image-detection
- clip
- multiple-instance-learning
metrics:
- accuracy
---
# LTF — AI-generated image detection

[Paper](https://arxiv.org/abs/2611.04529) · [Project page](https://huiguanlab.github.io/LTF/) · [Code](https://github.com/HuiGuanLab/LTF)

**[Try the online demo](https://huggingface.co/spaces/MuskAI/LTF-Demo)** — upload
an original image and click Run Analysis. The demo uses the Fusion branch at threshold
0.5 by default.

The demo also shows exact tile boundaries, L1 per-head tile attention, L2 view
weights, top-four attention crops, within-tile MIL patch attention and read-only parameters.
Attention shows relative model focus and is not a pixel-level forgery probability.

This repository contains the **epoch 20 checkpoint** used in the historical
GenImage evaluation `20260326_012209_ViT-L-14_genimage_v3_v3_3`.

LTF uses a frozen **CLIP ViT-L/14** backbone, two-level multiple-instance
learning (MIL), cross-shuffle during training, and pool-logit gated fusion.
The repository includes the custom PyTorch runtime and the frozen backbone.
It does not require conversion to a Transformers architecture.

## Historical results

Fixed threshold **0.5**, 8 GenImage subsets, **100,000 images**. Accuracy is the
unweighted mean across subsets, as recorded in the original rounded logs:

| Branch | Accuracy |
| --- | ---: |
| L1 | 95.26% |
| L2 | 97.39% |
| Fusion | **98.39%** |

Per-subset metrics are in `results/genimage8_logged_metrics.csv`.
These are historical logged results; uploading this repository did not rerun
the complete benchmark. SD1.4 official validation images were used during
training validation, so that subset overlaps model validation. These results
do not represent eight entirely held-out generators.

## Download and run

Use **Python 3.11**. The commands below use this model repository.

```bash
python -m pip install -U huggingface_hub
hf download MuskAI/LTF-v3.3 --local-dir ./LTF-v3.3
cd LTF-v3.3
python -m pip install -r requirements.txt
python inference.py --input /path/to/image.jpg --device cpu
```

For GPU inference, use `--device cuda:0`. For larger images, lower the CLIP
forward chunk size without removing tiles:

```bash
python inference.py --input /path/to/image.jpg --device cuda:0 --chunk-size 8
```

## Python API

Run from the downloaded repository directory:

```python
from ltf_final import LTFDetector
detector = LTFDetector(device="auto", chunk_size=8)
result = detector.predict("image.jpg", branch="fusion", threshold=0.5)
print(result["prediction"], result["score_fake"])
print(result["probabilities"])
```

The image is decoded as RGB. Non-overlapping full 224×224 local blocks, with right/bottom boundary completion,
plus a global view use CLIP preprocessing. Input files are not
recompressed. Test tiling depends on image size. Labels are **0=real, 1=fake**.
The API returns L1/L2/fusion scores, the selected label and the view count.
Loading the model does not download external weights after this repository
has been downloaded. It uses `backbones/ViT-L-14.pt`.

## Run the web demo locally

The tested Gradio UI is included in `demo/`. From the downloaded model directory:

```bash
python -m pip install -r demo/requirements.txt
LTF_MODEL_DIR="$PWD" python demo/app.py
```

Open `http://localhost:7860`. For a remote server, forward port 7860 over SSH.
Upload an original file, click **Run Analysis**, and inspect fusion, L1 and L2 scores.
The preview is for display only; inference uses the original file's bytes.

The demo also supports deployment as a separate Gradio Space. Set
`LTF_MODEL_REPO` to this model ID and `LTF_MODEL_REVISION` to a commit SHA.
Creating the online compute Space requires an eligible Hugging Face account.
Publishing the model repository alone does not start a hosted inference service.

## Checkpoint identity and provenance

`weights/epoch_20.pth` is the original core state_dict (not `best.pth`):
```text
SHA256 32e9fa2e38e04d6d0221c844d650f0d45f7235bbf898b5ad665543d8be580031
```
Frozen CLIP backbone:
```text
SHA256 b8cca3fd41ae0c99ba7e8951adf17d267cdb84cd88be6f7c2e0eca1737a03836
```
`release_manifest.json` records checksums of every runtime file. The original
training and evaluation directories did not save training-day source snapshots.
The released core was recovered from git commit
`65c18ee5868ced87a27d048261e237632ad5d704`, with network naming adjusted to LTF.
This verifies checkpoint identity and runtime compatibility, not bit-for-bit
reproduction of the original training process. This Hub adaptation adds
tensor-only checkpoint loading, preserves the model root in symlink-based Hub
caches, and removes a machine-specific dataset path.

Source comments and docstrings are in English. This documentation update preserves
the executable AST and checkpoint parameter files.

## Limitations

Scores are not calibrated authenticity probabilities. Predictions can change
with image content, generator, editing and compression. This model is a
research detector; an individual score does not prove image provenance.

The public demo uses configurable image-size limits for its available compute;
oversized uploads are rejected, not silently resized. CPU inference can be slow
for images containing many local tiles.

## License and citation

The official [source repository](https://github.com/HuiGuanLab/LTF) is released
under the [MIT license](https://github.com/HuiGuanLab/LTF/blob/main/LICENSE).
Example images retain the terms of their original datasets and rightsholders.
Bundled CLIP components retain the upstream
[OpenAI CLIP license](https://github.com/openai/CLIP/blob/main/LICENSE).

Please cite the LTF paper:

```bibtex
@inproceedings{chen2026ltf,
  title={Learning to Falsify: Hierarchical Multiple Instance Learning for High Resolution AI-Generated Image Detection},
  author={Chen, Haoran and Gao, Daiqi and Li, Xinyuan and Wang, Yongwei and Zhu, Lei and Cai, Rui and Dong, Jianfeng},
  booktitle={Proceedings of the 34th ACM International Conference on Multimedia},
  year={2026},
  doi={10.1145/3767308.3836365},
  eprint={2611.04529},
  archivePrefix={arXiv}
}
```

## Tiling policy

Full 224×224 blocks are taken without overlap. If pixels remain at the right
or bottom, a final block is aligned to that boundary; only boundary blocks
may overlap. For width 500, horizontal starts are 0, 224, 276. The global view
is separate. This corrects earlier uniformly overlapping tiling; weights are
unchanged, but predictions can change. Historical benchmark results have not
been recomputed with this preprocessing.

	---
	language:
	- en
	library_name: pytorch
	pipeline_tag: image-classification
	tags:
	- ltf
	- ai-generated-image-detection
	- clip
	- multiple-instance-learning
	metrics:
	- accuracy
	---

	# LTF — AI-generated image detection

	[Paper](https://arxiv.org/abs/2611.04529) · [Project page](https://huiguanlab.github.io/LTF/) · [Code](https://github.com/HuiGuanLab/LTF)

	[Try the online demo](https://huggingface.co/spaces/MuskAI/LTF-Demo) — upload
	an original image and click Run Analysis. The demo uses the Fusion branch at threshold
	0.5 by default.

	The demo also shows exact tile boundaries, L1 per-head tile attention, L2 view
	weights, top-four attention crops, within-tile MIL patch attention and read-only parameters.
	Attention shows relative model focus and is not a pixel-level forgery probability.

	This repository contains the epoch 20 checkpoint used in the historical
	GenImage evaluation `20260326_012209_ViT-L-14_genimage_v3_v3_3`.

	LTF uses a frozen CLIP ViT-L/14 backbone, two-level multiple-instance
	learning (MIL), cross-shuffle during training, and pool-logit gated fusion.
	The repository includes the custom PyTorch runtime and the frozen backbone.
	It does not require conversion to a Transformers architecture.

	## Historical results

	Fixed threshold 0.5, 8 GenImage subsets, 100,000 images. Accuracy is the
	unweighted mean across subsets, as recorded in the original rounded logs:

	\| Branch \| Accuracy \|
	\| --- \| ---: \|
	\| L1 \| 95.26% \|
	\| L2 \| 97.39% \|
	\| Fusion \| 98.39% \|

	Per-subset metrics are in `results/genimage8_logged_metrics.csv`.
	These are historical logged results; uploading this repository did not rerun
	the complete benchmark. SD1.4 official validation images were used during
	training validation, so that subset overlaps model validation. These results
	do not represent eight entirely held-out generators.

	## Download and run

	Use Python 3.11. The commands below use this model repository.

	```bash
	python -m pip install -U huggingface_hub
	hf download MuskAI/LTF-v3.3 --local-dir ./LTF-v3.3
	cd LTF-v3.3
	python -m pip install -r requirements.txt
	python inference.py --input /path/to/image.jpg --device cpu
	```

	For GPU inference, use `--device cuda:0`. For larger images, lower the CLIP
	forward chunk size without removing tiles:

	```bash
	python inference.py --input /path/to/image.jpg --device cuda:0 --chunk-size 8
	```

	## Python API

	Run from the downloaded repository directory:

	```python
	from ltf_final import LTFDetector

	detector = LTFDetector(device="auto", chunk_size=8)
	result = detector.predict("image.jpg", branch="fusion", threshold=0.5)
	print(result["prediction"], result["score_fake"])
	print(result["probabilities"])
	```

	The image is decoded as RGB. Non-overlapping full 224×224 local blocks, with right/bottom boundary completion,
	plus a global view use CLIP preprocessing. Input files are not
	recompressed. Test tiling depends on image size. Labels are 0=real, 1=fake.
	The API returns L1/L2/fusion scores, the selected label and the view count.
	Loading the model does not download external weights after this repository
	has been downloaded. It uses `backbones/ViT-L-14.pt`.

	## Run the web demo locally

	The tested Gradio UI is included in `demo/`. From the downloaded model directory:

	```bash
	python -m pip install -r demo/requirements.txt
	LTF_MODEL_DIR="$PWD" python demo/app.py
	```

	Open `http://localhost:7860`. For a remote server, forward port 7860 over SSH.
	Upload an original file, click Run Analysis, and inspect fusion, L1 and L2 scores.
	The preview is for display only; inference uses the original file's bytes.

	The demo also supports deployment as a separate Gradio Space. Set
	`LTF_MODEL_REPO` to this model ID and `LTF_MODEL_REVISION` to a commit SHA.
	Creating the online compute Space requires an eligible Hugging Face account.
	Publishing the model repository alone does not start a hosted inference service.

	## Checkpoint identity and provenance

	`weights/epoch_20.pth` is the original core state_dict (not `best.pth`):

	```text
	SHA256 32e9fa2e38e04d6d0221c844d650f0d45f7235bbf898b5ad665543d8be580031
	```

	Frozen CLIP backbone:

	```text
	SHA256 b8cca3fd41ae0c99ba7e8951adf17d267cdb84cd88be6f7c2e0eca1737a03836
	```

	`release_manifest.json` records checksums of every runtime file. The original
	training and evaluation directories did not save training-day source snapshots.
	The released core was recovered from git commit
	`65c18ee5868ced87a27d048261e237632ad5d704`, with network naming adjusted to LTF.
	This verifies checkpoint identity and runtime compatibility, not bit-for-bit
	reproduction of the original training process. This Hub adaptation adds
	tensor-only checkpoint loading, preserves the model root in symlink-based Hub
	caches, and removes a machine-specific dataset path.

	Source comments and docstrings are in English. This documentation update preserves
	the executable AST and checkpoint parameter files.

	## Limitations

	Scores are not calibrated authenticity probabilities. Predictions can change
	with image content, generator, editing and compression. This model is a
	research detector; an individual score does not prove image provenance.

	The public demo uses configurable image-size limits for its available compute;
	oversized uploads are rejected, not silently resized. CPU inference can be slow
	for images containing many local tiles.

	## License and citation

	The official [source repository](https://github.com/HuiGuanLab/LTF) is released
	under the [MIT license](https://github.com/HuiGuanLab/LTF/blob/main/LICENSE).
	Example images retain the terms of their original datasets and rightsholders.
	Bundled CLIP components retain the upstream
	[OpenAI CLIP license](https://github.com/openai/CLIP/blob/main/LICENSE).

	Please cite the LTF paper:

	```bibtex
	@inproceedings{chen2026ltf,
	title={Learning to Falsify: Hierarchical Multiple Instance Learning for High Resolution AI-Generated Image Detection},
	author={Chen, Haoran and Gao, Daiqi and Li, Xinyuan and Wang, Yongwei and Zhu, Lei and Cai, Rui and Dong, Jianfeng},
	booktitle={Proceedings of the 34th ACM International Conference on Multimedia},
	year={2026},
	doi={10.1145/3767308.3836365},
	eprint={2611.04529},
	archivePrefix={arXiv}
	}
	```

	## Tiling policy

	Full 224×224 blocks are taken without overlap. If pixels remain at the right
	or bottom, a final block is aligned to that boundary; only boundary blocks
	may overlap. For width 500, horizontal starts are 0, 224, 276. The global view
	is separate. This corrects earlier uniformly overlapping tiling; weights are
	unchanged, but predictions can change. Historical benchmark results have not
	been recomputed with this preprocessing.