Beyond Detection: Black-Box Characterization of AI Video Surveillance Systems. Operational Evaluation, Multimodal Retrieval and Camera - Display Compensation

The increasing integration of artificial intelligence techniques into modern CCTV systems has introduced advanced functionalities such as video analytics, intelligent event detection, and multimodal semantic retrieval. While these technologies offer significant operational advantages, their performance characterisation in real-world conditions remains challenging, particularly when their internal models, parameters, decision thresholds, and training datasets are not accessible. This thesis investigates the black-box evaluation of intelligent video surveillance systems at three complementary levels. Structured field tests were designed to characterise camera-level event detection under real residential conditions. A dedicated protocol and evaluation tool were developed to assess the multimodal semantic retrieval functionality of an NVR through graded relevance judgements and ranking-oriented metrics. Finally, a calibration and compensation method was developed to reduce the alterations introduced by a camera-display setup before video reacquisition. The experimental activities identified critical operating conditions for camera-level detection and clarified the relationship between retrieval coverage, ranking quality and similarity scores. The proposed compensation pipeline reduced the alterations introduced by the camera-display chain and improved the downstream detection metrics under the evaluated configuration. Overall, the thesis applies established evaluation and calibration principles to the operational assessment of commercial black-box CCTV systems. The resulting methodologies provide structured and reusable procedures for characterizing system behaviour within explicitly defined experimental conditions.