Forecasting with Drift and Uncertainty: A self-adaptive prediction model in the energy grid context

The European power grid is undergoing a transition towards a continually increasing share of renewable energy sources, targeting a net zero emissions state. This shift introduces greater variability and uncertainty in energy supply because renewable sources, such as wind and solar, are naturally inconsistent. As the grid incorporates these sources, management and operation become more complex. Consequently, accurately forecasting energy production, load, and market price fluctuations becomes more challenging and important. Here, changes in the underlying data distribution (concept drifts) and uncertainty in input features (feature uncertainty) are two major issues that compromise the accuracy of conventional machine learning models. This thesis introduces a lightweight model based on adaptive regression and meta-learning to address the concept drift and feature uncertainty issues. We structured the framework as a two-layer system, composed of a primary model that is used to generate the prediction and a secondary meta-model that dynamically adjusts the primary model's parameters in response to drift signals, feature uncertainty scores, and recent performance metrics. Unlike traditional methods, this approach does not require constant retraining or continuous online parameter updates; instead, a meta-learner dynamically tunes the primary model based on real-time indicators. This system could significantly reduce computational cost and provide a more robust model that can adapt to changes in the data landscape and provide continuous predictions as output.