Robustness and Adaptation of TD-MPC2

Reinforcement learning enables robotic controllers to be learned through interaction with the environment, reducing the need for manually engineered control strategies. In model-based approaches, a learned world model predicts the consequences of actions and supports planning, potentially reducing the experience needed to learn effective behaviour. However, both learned skills and model predictions may become less reliable when operating conditions change. Actuator degradation, changes in mechanical properties or different contact conditions can affect task performance, raising the question of whether existing knowledge remains useful and how the controller can adapt through further interaction. This thesis investigates these questions through TD-MPC2, a model-based reinforcement learning algorithm combining a learned world model with model predictive control. Experiments consider five configurations of a simulated bipedal robot in a running task, with different actuator strengths and physical properties. The study examines zero-shot robustness, the contribution of planning and policy proposals and adaptation through continual training. Replay buffer resets and selective neural network resets are also evaluated, considering their effects on recovery, target performance and partial forgetting of previously acquired capabilities. The results show that strong performance under the training dynamics does not guarantee successful transfer. Training under symmetrically modified dynamics provides the strongest transfer across most tested targets, while more extensive modifications make a configuration difficult to transfer to, despite supporting moderate performance when transferring from it to other environments. Planning improves performance and remains useful under the tested changes, although longer planning horizons do not consistently improve results. Continual learning enables substantial recovery without requiring an explicit signal of the dynamics change. Reset strategies affect recovery and retention but offer no clear, sustained advantage in final target performance. These findings suggest that previously learned knowledge can provide a useful basis for adaptation. The thesis therefore identifies both the potential of continual learning under changing dynamics and the difficulty of retaining earlier capabilities, leaving their balance as a direction for future research in robotic learning adaptation.