Self-supervised Deepfake detection with Local Feature Exploration

Deepfake detection remains a challenging and critical task in security. No single model excels across all types of manipulated faces. This research aims to discover the importance of different parts of the face in the deepfake detection task using Self-Blended Images (SBI). The SBI approach involves generating a mask from a face, manipulating it, and blending it back to create a fake image. We extend this technique by generating distinct masks for the eyes, nose, and mouth. Then we apply the SBI method per model and train three models on more complex tasks. In our implementation, we extract the input of the last fully connected layer in the EfficientNet-04 for different mask generators. Next, we define three principal component analyses (PCAs) and fine-tune a multi-layer perceptron (MLP) for each type. In this approach, the model's ability is leveraged to focus on specific facial regions, which has the potential to enhance the discriminative power of the model. Our findings indicate that while the extended SBI approach does not universally improve performance across all datasets, it shows notable improvement for one dataset. This highlights the potential of our method in certain contexts and suggests areas for further refinement and optimization.