Machine Learning-based Image Forgery Detection

Image manipulation tools are constantly improving. Most recently, the productization of generative models in popular software like Adobe Photoshop provided a whole new range of possibilities. Though many applications might be harmless, image forgery is not. Tampered images can spread false information, manipulate opinions, and erode trust in media. Therefore, being able to detect fake images is of the utmost importance. The majority of Image Forgery Detection models consist of specialized architectures, often trained with limited data and computational resources. In contrast, image segmentation has found substantial interest and investment. In this work, I explore the capabilities of state-of-the-art general image segmentation models to adapt to the task of Image Forgery Detection to leverage the extensive resources and advancements in this field. I assess their performance on the detection of classical Photoshop manipulation like splicing. Further, I extend the scope to the detection of AI-inpainted images, i.e. images that were manipulated using deep generative models. I show that image segmentation models can keep up with state-of-the-art forgery detection tools. Moreover, the models can detect AI-inpainted regions by identifying the characteristic frequency signature of the generative models.