- - 1Academia Sinica, - 2Microsoft, - 3National Taiwan University, -
- 4National Taiwan University of Science and Technology -
-
- 
- -
- Yue-Hua Han
- 1,3,4
-
- Tai-Ming Huang
- 1,3,4
-
- Kai-Lung Hua
- 2,4
-
- Jun-Cheng Chen
- 1
-
-
- 1Academia Sinica,
- 2Microsoft,
- 3National Taiwan University,
-
- 4National Taiwan University of Science and Technology
-
-
-
- 
-
| Compression |
|---|
| c23 |
| c0 |
| AUC |
|---|
| 92.4 |
| 96.5 |
| AUC |
|---|
| 74.2 |
| 76.3 |
| AUC |
|---|
| 98.5 |
| 98.9 |
| AUC |
|---|
| 74.6 |
| 77.6 |
| AUC |
|---|
| 90.0 |
| 94.1 |
| AUC |
|---|
| 96.9 |
| 97.7 |
-
-
-
-
-
-
-> **Abstract.** In recent years, DeepFake technology has achieved unprecedented success in high-quality video synthesis, but these methods also pose potential and severe security threats to humanity. DeepFake can be bifurcated into entertainment applications like face swapping and illicit uses such as lip-syncing fraud. However, lip-forgery videos, which neither change identity nor have discernible visual artifacts, present a formidable challenge to existing DeepFake detection methods. Our preliminary experiments have shown that the effectiveness of the existing methods often drastically decrease or even fail when tackling lip-syncing videos.
-> In this paper, for the first time, we propose a novel approach dedicated to lip-forgery identification that exploits the inconsistency between lip movements and audio signals. We also mimic human natural cognition by capturing subtle biological links between lips and head regions to boost accuracy. To better illustrate the effectiveness and advances of our proposed method, we create a high-quality LipSync dataset, AVLips, by employing the state-of-the-art lip generators. We hope this high-quality and diverse dataset could be well served the further research on this challenging and interesting field. Experimental results show that our approach gives an average accuracy of more than 95.3% in spotting lip-syncing videos, significantly outperforming the baselines. Extensive experiments demonstrate the capability to tackle deepfakes and the robustness in surviving diverse input transformations. Our method achieves an accuracy of up to 90.2% in real-world scenarios (e.g., WeChat video call) and shows its powerful capabilities in real scenario deployment.
-
-
-
-
-
-## 🔥 AVLips: A high-quality audio-visual dataset for LipSync detection
-
-To the best of our knowledge, the majority of public DeepFake datasets consist solely of videos or images, with no specialized one specifically dedicated to LipSync detection available. To fill this gap, we construct a high-quality **A**udio-**V**isual **Lip**-syncing Dataset, **AVLips**, which contains up to 340,000 audio-visual samples generated by several SOTA LipSync methods. The workflow is demonstrated below.
-
-**High quality.** We employed a combination of static MakeItTalk and dynamic Wav2Lip, TalkLip, SadTalker generation methods to simulate realistic lip movements. These methods are widely recognized as high-quality work, capable of generating high-resolution videos while ensuring accurate lip movements. We applied a noise reduction algorithm to all audio samples before synthesis to reduce irrelevant background noise, ensuring the models can focus on speech content.
-
-**Diversity.** Our dataset encompasses a wide range of scenarios, covering not only well-known public datasets but also real-world data. Our aim is for this collection to act as a catalyst for advancing real-time forgery detection. To better simulate the nuances of real-world conditions, we have employed six perturbation techniques — saturation, contrast, compression, Gaussian noise, Gaussian blur, and pixelation — at various degrees, thus ensuring the dataset's realism and practical relevance.
-
-**Download Link: [AVLips v1.0](https://drive.google.com/file/d/1fEiUo22GBSnWD7nfEwDW86Eiza-pOEJm/view?usp=share_link)**
-
-
-
-
-#### Update (Aug 25, 2020)
-
-This update adds:
- * A new `include_top` (default: `True`) option ([#208](https://github.com/lukemelas/EfficientNet-PyTorch/pull/208))
- * Continuous testing with [sotabench](https://sotabench.com/)
- * Code quality improvements and fixes ([#215](https://github.com/lukemelas/EfficientNet-PyTorch/pull/215) [#223](https://github.com/lukemelas/EfficientNet-PyTorch/pull/223))
-
-#### Update (May 14, 2020)
-
-This update adds comprehensive comments and documentation (thanks to @workingcoder).
-
-#### Update (January 23, 2020)
-
-This update adds a new category of pre-trained model based on adversarial training, called _advprop_. It is important to note that the preprocessing required for the advprop pretrained models is slightly different from normal ImageNet preprocessing. As a result, by default, advprop models are not used. To load a model with advprop, use:
-```python
-model = EfficientNet.from_pretrained("efficientnet-b0", advprop=True)
-```
-There is also a new, large `efficientnet-b8` pretrained model that is only available in advprop form. When using these models, replace ImageNet preprocessing code as follows:
-```python
-if advprop: # for models using advprop pretrained weights
- normalize = transforms.Lambda(lambda img: img * 2.0 - 1.0)
-else:
- normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],
- std=[0.229, 0.224, 0.225])
-```
-This update also addresses multiple other issues ([#115](https://github.com/lukemelas/EfficientNet-PyTorch/issues/115), [#128](https://github.com/lukemelas/EfficientNet-PyTorch/issues/128)).
-
-#### Update (October 15, 2019)
-
-This update allows you to choose whether to use a memory-efficient Swish activation. The memory-efficient version is chosen by default, but it cannot be used when exporting using PyTorch JIT. For this purpose, we have also included a standard (export-friendly) swish activation function. To switch to the export-friendly version, simply call `model.set_swish(memory_efficient=False)` after loading your desired model. This update addresses issues [#88](https://github.com/lukemelas/EfficientNet-PyTorch/pull/88) and [#89](https://github.com/lukemelas/EfficientNet-PyTorch/pull/89).
-
-#### Update (October 12, 2019)
-
-This update makes the Swish activation function more memory-efficient. It also addresses pull requests [#72](https://github.com/lukemelas/EfficientNet-PyTorch/pull/72), [#73](https://github.com/lukemelas/EfficientNet-PyTorch/pull/73), [#85](https://github.com/lukemelas/EfficientNet-PyTorch/pull/85), and [#86](https://github.com/lukemelas/EfficientNet-PyTorch/pull/86). Thanks to the authors of all the pull requests!
-
-#### Update (July 31, 2019)
-
-_Upgrade the pip package with_ `pip install --upgrade efficientnet-pytorch`
-
-The B6 and B7 models are now available. Additionally, _all_ pretrained models have been updated to use AutoAugment preprocessing, which translates to better performance across the board. Usage is the same as before:
-```python
-from efficientnet_pytorch import EfficientNet
-model = EfficientNet.from_pretrained('efficientnet-b7')
-```
-
-#### Update (June 29, 2019)
-
-This update adds easy model exporting ([#20](https://github.com/lukemelas/EfficientNet-PyTorch/issues/20)) and feature extraction ([#38](https://github.com/lukemelas/EfficientNet-PyTorch/issues/38)).
-
- * [Example: Export to ONNX](#example-export)
- * [Example: Extract features](#example-feature-extraction)
- * Also: fixed a CUDA/CPU bug ([#32](https://github.com/lukemelas/EfficientNet-PyTorch/issues/32))
-
-It is also now incredibly simple to load a pretrained model with a new number of classes for transfer learning:
-```python
-model = EfficientNet.from_pretrained('efficientnet-b1', num_classes=23)
-```
-
-
-#### Update (June 23, 2019)
-
-The B4 and B5 models are now available. Their usage is identical to the other models:
-```python
-from efficientnet_pytorch import EfficientNet
-model = EfficientNet.from_pretrained('efficientnet-b4')
-```
-
-### Overview
-This repository contains an op-for-op PyTorch reimplementation of [EfficientNet](https://arxiv.org/abs/1905.11946), along with pre-trained models and examples.
-
-The goal of this implementation is to be simple, highly extensible, and easy to integrate into your own projects. This implementation is a work in progress -- new features are currently being implemented.
-
-At the moment, you can easily:
- * Load pretrained EfficientNet models
- * Use EfficientNet models for classification or feature extraction
- * Evaluate EfficientNet models on ImageNet or your own images
-
-_Upcoming features_: In the next few days, you will be able to:
- * Train new models from scratch on ImageNet with a simple command
- * Quickly finetune an EfficientNet on your own dataset
- * Export EfficientNet models for production
-
-### Table of contents
-1. [About EfficientNet](#about-efficientnet)
-2. [About EfficientNet-PyTorch](#about-efficientnet-pytorch)
-3. [Installation](#installation)
-4. [Usage](#usage)
- * [Load pretrained models](#loading-pretrained-models)
- * [Example: Classify](#example-classification)
- * [Example: Extract features](#example-feature-extraction)
- * [Example: Export to ONNX](#example-export)
-6. [Contributing](#contributing)
-
-### About EfficientNet
-
-If you're new to EfficientNets, here is an explanation straight from the official TensorFlow implementation:
-
-EfficientNets are a family of image classification models, which achieve state-of-the-art accuracy, yet being an order-of-magnitude smaller and faster than previous models. We develop EfficientNets based on AutoML and Compound Scaling. In particular, we first use [AutoML Mobile framework](https://ai.googleblog.com/2018/08/mnasnet-towards-automating-design-of.html) to develop a mobile-size baseline network, named as EfficientNet-B0; Then, we use the compound scaling method to scale up this baseline to obtain EfficientNet-B1 to B7.
-
-
- 
- |
-
- 
- |
-
-
-
-#### Update (Aug 25, 2020)
-
-This update adds:
- * A new `include_top` (default: `True`) option ([#208](https://github.com/lukemelas/EfficientNet-PyTorch/pull/208))
- * Continuous testing with [sotabench](https://sotabench.com/)
- * Code quality improvements and fixes ([#215](https://github.com/lukemelas/EfficientNet-PyTorch/pull/215) [#223](https://github.com/lukemelas/EfficientNet-PyTorch/pull/223))
-
-#### Update (May 14, 2020)
-
-This update adds comprehensive comments and documentation (thanks to @workingcoder).
-
-#### Update (January 23, 2020)
-
-This update adds a new category of pre-trained model based on adversarial training, called _advprop_. It is important to note that the preprocessing required for the advprop pretrained models is slightly different from normal ImageNet preprocessing. As a result, by default, advprop models are not used. To load a model with advprop, use:
-```python
-model = EfficientNet.from_pretrained("efficientnet-b0", advprop=True)
-```
-There is also a new, large `efficientnet-b8` pretrained model that is only available in advprop form. When using these models, replace ImageNet preprocessing code as follows:
-```python
-if advprop: # for models using advprop pretrained weights
- normalize = transforms.Lambda(lambda img: img * 2.0 - 1.0)
-else:
- normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],
- std=[0.229, 0.224, 0.225])
-```
-This update also addresses multiple other issues ([#115](https://github.com/lukemelas/EfficientNet-PyTorch/issues/115), [#128](https://github.com/lukemelas/EfficientNet-PyTorch/issues/128)).
-
-#### Update (October 15, 2019)
-
-This update allows you to choose whether to use a memory-efficient Swish activation. The memory-efficient version is chosen by default, but it cannot be used when exporting using PyTorch JIT. For this purpose, we have also included a standard (export-friendly) swish activation function. To switch to the export-friendly version, simply call `model.set_swish(memory_efficient=False)` after loading your desired model. This update addresses issues [#88](https://github.com/lukemelas/EfficientNet-PyTorch/pull/88) and [#89](https://github.com/lukemelas/EfficientNet-PyTorch/pull/89).
-
-#### Update (October 12, 2019)
-
-This update makes the Swish activation function more memory-efficient. It also addresses pull requests [#72](https://github.com/lukemelas/EfficientNet-PyTorch/pull/72), [#73](https://github.com/lukemelas/EfficientNet-PyTorch/pull/73), [#85](https://github.com/lukemelas/EfficientNet-PyTorch/pull/85), and [#86](https://github.com/lukemelas/EfficientNet-PyTorch/pull/86). Thanks to the authors of all the pull requests!
-
-#### Update (July 31, 2019)
-
-_Upgrade the pip package with_ `pip install --upgrade efficientnet-pytorch`
-
-The B6 and B7 models are now available. Additionally, _all_ pretrained models have been updated to use AutoAugment preprocessing, which translates to better performance across the board. Usage is the same as before:
-```python
-from efficientnet_pytorch import EfficientNet
-model = EfficientNet.from_pretrained('efficientnet-b7')
-```
-
-#### Update (June 29, 2019)
-
-This update adds easy model exporting ([#20](https://github.com/lukemelas/EfficientNet-PyTorch/issues/20)) and feature extraction ([#38](https://github.com/lukemelas/EfficientNet-PyTorch/issues/38)).
-
- * [Example: Export to ONNX](#example-export)
- * [Example: Extract features](#example-feature-extraction)
- * Also: fixed a CUDA/CPU bug ([#32](https://github.com/lukemelas/EfficientNet-PyTorch/issues/32))
-
-It is also now incredibly simple to load a pretrained model with a new number of classes for transfer learning:
-```python
-model = EfficientNet.from_pretrained('efficientnet-b1', num_classes=23)
-```
-
-
-#### Update (June 23, 2019)
-
-The B4 and B5 models are now available. Their usage is identical to the other models:
-```python
-from efficientnet_pytorch import EfficientNet
-model = EfficientNet.from_pretrained('efficientnet-b4')
-```
-
-### Overview
-This repository contains an op-for-op PyTorch reimplementation of [EfficientNet](https://arxiv.org/abs/1905.11946), along with pre-trained models and examples.
-
-The goal of this implementation is to be simple, highly extensible, and easy to integrate into your own projects. This implementation is a work in progress -- new features are currently being implemented.
-
-At the moment, you can easily:
- * Load pretrained EfficientNet models
- * Use EfficientNet models for classification or feature extraction
- * Evaluate EfficientNet models on ImageNet or your own images
-
-_Upcoming features_: In the next few days, you will be able to:
- * Train new models from scratch on ImageNet with a simple command
- * Quickly finetune an EfficientNet on your own dataset
- * Export EfficientNet models for production
-
-### Table of contents
-1. [About EfficientNet](#about-efficientnet)
-2. [About EfficientNet-PyTorch](#about-efficientnet-pytorch)
-3. [Installation](#installation)
-4. [Usage](#usage)
- * [Load pretrained models](#loading-pretrained-models)
- * [Example: Classify](#example-classification)
- * [Example: Extract features](#example-feature-extraction)
- * [Example: Export to ONNX](#example-export)
-6. [Contributing](#contributing)
-
-### About EfficientNet
-
-If you're new to EfficientNets, here is an explanation straight from the official TensorFlow implementation:
-
-EfficientNets are a family of image classification models, which achieve state-of-the-art accuracy, yet being an order-of-magnitude smaller and faster than previous models. We develop EfficientNets based on AutoML and Compound Scaling. In particular, we first use [AutoML Mobile framework](https://ai.googleblog.com/2018/08/mnasnet-towards-automating-design-of.html) to develop a mobile-size baseline network, named as EfficientNet-B0; Then, we use the compound scaling method to scale up this baseline to obtain EfficientNet-B1 to B7.
-
-|
-
- |
-
-
- |
-
- 
- 
- 
- - We introduce a novel method for deepfake video detection that utilizes pixel‑wise temporal frequency spectra. Unlike previous approaches that stack 2D frame‑wise spatial frequency spectra, we extract pixel‑wise temporal frequency by performing a 1D Fourier transform on the time axis per pixel, effectively identifying temporal artifacts. We also propose an Attention Proposal Module (APM) to extract regions of interest for detecting these artifacts. Our method demonstrates outstanding generalizability and robustness in various challenging deepfake video detection scenarios. -
-
- 
- 
- 
- We will investigate temporal‑frequency regularization to mitigate compression‑induced degradation.
-
- @InProceedings{Kim_2025_ICCV,
- author = {Kim, Taehoon and Choi, Jongwook and Jeong, Yonghyun and Noh, Haeun and Yoo, Jaejun and Baek, Seungryul and Choi, Jongwon},
- title = {Beyond Spatial Frequency: Pixel-wise Temporal Frequency-based Deepfake Video Detection},
- booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
- month = {October},
- year = {2025},
- pages = {11198-11207}
-}
- This work was partly supported by the IITP grant funded by the Korea government (MSIT): No. RS-2025-02263841: Development of a Real-time Multimodal Framework for Comprehensive Deepfake Detection Incorporating Common Sense Error AnalysisRS-2021-II211341: Artificial Intelligence Graduate School Program (Chung-Ang University) No. RS-2020-II201336: AIGS program (UNIST)
-