Improvement of time-series forecasting algorithms in low-power embedded systems by efficient online/incremental learning

The systems have several sensors inside, forecasting the signals coming from the sensors could be advantageous in a variety of issues such as controlling the system. Neural networks and specifically deep neural networks are powerful models to address dilemmas including nonlinearity. Seeing that electrical appliances have microcontrollers with tiny space for memory for controlling the device, with low power, sometimes deep neural networks can’t be fitted. Hence, a shallow network would be well suited for this scheme. Extracting some features before feeding the signal to the model is useful. Then, the computed features will go through the shallow network to calculate the output. Applying a shallow network for signal forecasting might not guarantee the best performance. As a consequence, applying online learning at the edge could be worthwhile to improve the shallow network as the data is streaming. Finally, the performance of the shallow network by using online learning at the device in the aspects of inference time and memory usage will be compared to the shallow network without online learning. There exists various approaches for incremental learning. Here, TinyTL will be utilized. TinyTL is one of the techniques for incremental learning which freeze the weights and train the biases of the network.