Comparative Study of a Combined EVE-RAdam-SGD Optimizer Versus the Adam Optimizer on ResNet-50

Deep learning has significantly advanced image classification tasks, with architectures like ResNet-50 achieving state-of-the-art performance. However, the choice of optimization algo- rithm is crucial for balancing convergence speed, accuracy, and generalization. In this thesis, we introduce AERS (Adaptive Eve-RAdam-SGD), a novel optimizer that integrates RAdams adaptive momentum correction, SGDs momentum-based updates, and an adaptive loss-scaling mechanism to improve training stability. We compare AERS with Adama widely used adaptive optimizerby training ResNet-50 on a custom dataset and evaluating performance based on training, validation, and test accuracies, test loss, and the Area Under the ROC Curve (AUC). Our results indicate that Adam achieves higher test accuracy (97.27% vs. 96.36%) and faster convergence, whereas AERS attains better validation accuracy (95.45% vs. 91.82%), lower test loss (0.2095 vs. 0.4234), and a slightly higher mean AUC (0.9973 vs. 0.9951). Moreover, AERS exhibits lower mean error under ROC (0.0027 vs. 0.0049) and a better worst-class AUC, suggesting stronger performance on more challenging classes. However, these differences are not statistically significant (Wilcoxon Signed-Rank Test, p = 0.8750), implying that both Adam and AERS remain viable optimizers. Ultimately, the choice hinges on whether to prioritize raw accuracy and speed (favoring Adam) or improved generalization and robustness (favoring AERS).