Self-supervised learning for colonoscopy: a study on polyp counting and model generalization

All endoscopy procedures are now guided by a video stream, which allows for a less invasive operation, reduced risks, and faster recovery. However, the endoscopy datasets available today represent only a very small fraction of all colonoscopies performed. For this reason, there has been a growing need to leverage all available endoscopy data, including unannotated data, which is relatively easier to collect and significantly cheaper than data manually annotated by expert endoscopists. The recent success of self-supervised pre-training strategies has not gone unnoticed in the medical computer vision community, which has quickly adopted them to address the problem of data and annotation scarcity. However, unlike self-supervised models on natural images—which can draw on hundreds of millions, if not billions, of examples—in colonoscopy the numbers usually reach only hundreds of thousands, at best a few million. This naturally raises an important question: are these general-purpose models robust enough to be applied to tasks different from those illustrated in the pre-training data? Can they also generalize to new datasets? The aim of this thesis is to analyze and compare this type of models on the task of polyp counting, which requires the ability to determine the number of polyps encountered throughout the video. Beyond this comparative study, this thesis will explore how representation learning with different objectives can affect downstream tasks.