Semi-supervised Deep Learning methods for Video Anomaly Detection applied to sticking identification during steelmaking continuous casting process

Detecting video anomalies is a challenging task, as anomalies represent rare and unbounded data. Many works show how unsupervised or semi-supervised Deep Learning algorithms can be used to detect anomalies. In particular, in the case of the video anomaly detection task, these methods can be divided into reconstruction-based and prediction-based. The idea is that badly reconstructed samples or wrong predictions can be considered anomalies. However, often the task might be to detect a certain type of anomalous data, then an unsupervised method might not perform well as it finds all anomalies among the data. This thesis proposes an approach that combines unsupervised, or semi-supervised, methods with supervised methods to predict specific anomalies among the data. The general idea is to first find all anomalies and then use supervised models to predict them. The framework is applied for sticking detection during steelmaking continuous casting process. Sticking occurs when casting steel adheres to the mold surface and this can cause several problems, such as ``breakout'', which is one of the most expensive and dangerous problems during continuous casting. In particular, stickings are detected using the mold surface temperatures over time as a heat map video. Three Deep Learning models are proposed for predicting stickings among the anomalous data: a Video DenseNet, a Convolutional LSTM model, and a Video Vision Transformer model.