Latent Replay for Continual Learning on Edge devices with Efficient Architectures

Due to the limited computational capabilities, low memory and limited energy budget, training deep neural networks on edge devices is very challenging. On the other hand, privacy and data limitations, lack of network connection, as well as the need for rapid model adaptation, make real-time training on the device crucial. Standard artificial neural networks suffer from the issue of catastrophic forgetting, making learning difficult. Continual learning shifts this paradigm to networks that can continuously accumulate knowledge on different tasks without the need to retrain from scratch. In this work, a Continual Learning technique called Latent Replay is employed, in which the activations of intermediate layers are stored and used to integrate training data for each new task. This approach reduces the computation time and memory required, facilitating training on the limited resources of edge devices. In addition, a new efficient architecture, known as PhiNets, was used for the first time in the context of Continual Learning. An intensive study was conducted to compare PhiNets with efficient architectures already tested in this context, such as MobileNet. Several metrics were considered, such as computation time, inference time, memory used, and accuracy. In addition, the variation of these metrics based on factors such as the layer at which Latent Replay is applied was analysed. Tests were performed on well-known computer vision datasets, evaluating them as a stream of classes.