Exploring SAM-Augmented Ensembles For Image Segmentation Tasks

This work presents novel augmentation methods designed to integrate into the original images spatial and semantic information extracted from SAM, a general-purpose segmentation model. The proposed algorithms are first tested in a standalone manner using different state-of-the-art segmentation models across diverse domains, aiming to assess their generalization capabilities. Afterward, exploiting the different strengths of each proposed method, this work presents AuxMix, an ensemble strategy that combines different base learners trained with different augmentation algorithms. The reported results demonstrate how this ensemble strategy, by combining complementary information from each augmentation, leads to a robust and improved segmentation performance. In particular, this study demonstrates how the proposed framework can be applied to diverse architectures and across a broad range of domains, including, but not limited to, medical images.