Study on Vision Transformer Ensemble for medical image Segmentation

Deep learning has emerged as a powerful tool for medical image segmentation, crucial for accurate diagnosis and treatment planning. This thesis explores the utilization of Vision Transformer (ViT) ensembles, in addition to models used in standard approaches like Convolutional Neural Networks (CNN). We investigate the effect of combining various networks for the challenging task of medical images segmentation, in particular working on datasets focusing on samples of polyp images. The study also questions the effectiveness of differentiating the loss function during training, in relation to the strategies used. In addition to the evaluation of the impact of standard data augmentation, we have adopted a very recent and promising approach, involving the use of the Segment Anything (SAM) model, by Meta. This is used to pre-process adding insightful information by exploiting the features that this model offers given its excellent ability in generalized segmentation. Through extensive experimentation, and with due considerations, our approach demonstrates comparable performance to most of the latest methods, giving insights for more detailed studies regarding medical segmentation.