Development of a reinforcement learning-based meal insulin bolus calculator for pediatric type 1 diabetes under multiple daily injection therapy

Type 1 Diabetes Mellitus (T1DM) is a chronic disease characterized by insulin deficiency due to autoimmune β-cell destruction. This condition leads to the need for T1DM patients to constantly monitor and manage blood glucose levels. Typically, this is achieved using Continuous Glucose Monitoring (CGM) devices and insulin therapy, which is often delivered in a Multiple Daily Injections (MDI) regimen. The insulin dose is usually calculated using a standard formula that, even if personalized using Carbohydrate Ratio (CR) and Correction Factor (CF), remains generic and does not fully account for the patient's unique physiology and metabolic dynamics. In this thesis, a novel framework is presented for developing a meal insulin bolus calculator leveraging digital twin methodology and reinforcement learning, to further personalize and optimize insulin dosing. In particular, the digital twin provides a virtual model of the pediatric patients, allowing the reinforcement learning agent to learn optimal insulin doses for different glucose levels and meal patterns. The digital twin software used in this thesis is ReplayBG, while the reinforcement learning algorithm chosen is Q-Learning, which is a model-free and quite explainable algorithm. The dataset used to train the RL agents comes from the Tidepool Big Data Donation Project and consists of five pediatric patients affected by T1DM. The RL agents were trained on seven days and tested on fourteen days of data. Some metrics such as Time in Range (TIR), Time Below Range (TBR) and Time Above Range (TAR) were measured and compared to a baseline where doses were computed using the standard formula. The results of this thesis show the potential of using an integration of digital twin and reinforcement learning to create a personalized insulin bolus calculator for T1DM patients under multiple daily injection therapy.