A digital twin-based framework for developing new reinforcement learning algorithms for type 1 diabetes management

Type 1 Diabetes Mellitus (T1DM) is a chronic condition that requires constant monitoring and management of blood glucose levels, typically through insulin therapy. Despite advancements in insulin delivery technologies, such as insulin pumps and continuous glucose monitoring (CGM) systems, maintaining optimal glycemic control remains a significant challenge. This thesis presents a novel framework that integrates Digital Twins with Reinforcement Learning (RL) algorithms to enhance the management of T1DM. The proposed framework leverages real-time data from CGM devices and digital twins to simulate patient-specific glucose dynamics, enabling the RL algorithms to learn optimal insulin dosing strategies. The digital twin functions as a virtual model, replicating the physiological behavior of the patient, while the RL component optimizes glucose regulation by continuously adjusting insulin delivery based on predicted glucose trends. This work explores the application of Q-learning, a model-free RL technique, within this framework and compares it with the existing replay modality of the ReplayBG Framework. Using the OhioT1DM dataset, the developed RL models were trained on one patient day and tested on various others to evaluate their effectiveness in maintaining blood glucose levels within the target range. The results suggest that the digital twin-enhanced RL approach may improve Time in Range (TIR) and reduce the occurrence of hypo- and hyperglycemia events compared to traditional methods in some patients. Additionally, the thesis outlines the potential of this framework for future applications in personalized diabetes management.