Training Techniques for Federated Continual Learning

Federated Continual Learning (FCL) aims to address the challenge of adapting the model to class distribution shifts over time while learning from non-IID data distributed across multiple clients. Building upon the FedSpace framework, this thesis explores the integration of Hyperbolic Prototypes (HP), Nearest Mean Prototype (NMP) Classifier, and Knowledge Distillation (KD) in an attempt to enhance classification accuracy and mitigate catastrophic forgetting in FCL. Hyperbolic prototypes are leveraged to capture the hierarchical structure of data representations, enabling a more compact and expressive representation of class relationships. The NMP classifier, known for its efficiency in continual learning, is employed to enhance classification performance by utilizing the class-wise mean of embeddings. Additionally, Knowledge Distillation is introduced to facilitate knowledge transfer between past and current models, aiming to retain previously learned information while accommodating new data. Experimental results reveal that the integration of hyperbolic prototypes and the NMP classifier did not lead to improvements in classification accuracy. These methods, while theoretically beneficial, struggled to provide benefits under the challenging conditions of FCL. In contrast, Knowledge Distillation demonstrated a slight improvement in performance, suggesting its potential in mitigating forgetting in FCL settings. This study provides a comprehensive analysis of these methods within the FedSpace framework, exploring the limitations and potential of hyperbolic spaces, prototype-based classifiers, and knowledge transfer techniques in advancing FCL systems.