Scheduling of Electric Vehicle Charging Stations in CityLearn: Reinforcement Learning versus Model-based Control

The growing use of Electric Vehicles (EVs) is creating new challenges for energy management in smart-grid systems, especially in scenarios where charging demand, electricity prices, and battery constraints must be handled simultaneously. This thesis studies EV charging scheduling in the CityLearn 2.5.0 environment and compares different control approaches for managing EV charging behavior under realistic scheduling constraints. The work focuses on three controllers: a no-control baseline, a rule-based controller, and a tabular Q-learning agent. The experiments evaluate how these approaches perform in terms of charging efficiency, electricity consumption, and the ability to satisfy EV departure charging requirements. Particular attention is given to the relationship between the reward function and the actual charging objectives of the system. The results show that, although the Q-learning agent is able to improve cumulative reward in some cases, the rule-based controller performs more reliably in meeting departure charging targets. The study also highlights how reward design and state representation strongly affect the behavior of reinforcement learning agents in EV charging applications. Overall, this thesis provides an experimental analysis of EV charging control in CityLearn and discusses the practical strengths and limitations of reinforcement learning methods for smart-grid energy management.