Enhancing interpretability in Visual Anomaly Detection through Concept Bottleneck Models

Visual Anomaly Detection (VAD) algorithms identify unusual patterns in images that differ from expected data. These algorithms find several real-world applications, including industrial inspection, medical imaging, and security surveillance. Together with achieving impressive performance on benchmarks, state-of-the-art algorithms provide visual explanations by highlighting the area of the image that causes it to be classified as anomalous. While useful, these visual insights lack a more direct and meaningful interpretation for the human operator. To tackle this liability, this work explores the possibility of adapting Concept Bottleneck Models (CBMs) to the context of Industrial VAD: CBMs are an innovative type of neural network, designed to improve transparency and interpretability by predicting outcomes based on intermediate, human-interpretable concepts that are learned directly by the network. The application of CBMs to the Visual Anomaly Detection scenario is currently unexplored, so no dataset with concept annotation is available. Thus, the first part of the study explores how to extract concepts from industrial images, leveraging the capabilities of Vision Language Models (VLMs), which are employed to devise a fully automated pipeline for the choice of meaningful concepts and dataset annotation. Following, three learning strategies for Concept Bottleneck Models are tested, and the results are analyzed in terms of prediction accuracy and quality of the predicted concepts, suggesting promising possibilities for future extensions in the field. The other key advantage of CBMs is the possibility of manually intervening on the predicted concepts, thus allowing for a closer human-model interaction that enhances the performance of the algorithm: this will be briefly explored for the scenario at hand, together with the integration of an unsupervised VAD algorithm that allows for the extraction of an anomaly heatmap, thus allowing for visual explanations as well.