Enhancing Deep Ensembles for Image Classification

Ensembles are almost always preferable to a single model in real-world settings due to their enhanced reliability and scalability. Nonetheless, creating deep ensembles is not as straightforward as it may seem. In this work, I explore several previously successful ensembling techniques used with residual networks, applying them to various image transformer architectures, including Co-Scale Conv-Attentional Image Transformers, Pyramid Vision Transformer, Cross Image Transformer, and Swin Transformer. The results, along with the proposed creative approaches, are presented in detail.