Reinforcement Learning algorithms for IoT communications over uncoordinated access channels

This thesis focuses on massive remote monitoring applications, where thousands of devices send time stamped updates over a wireless channel to a common receiver. An uncoordinated communication protocol based on slotted ALOHA is adopted, with the overall objective to keep an up-to-date perception at the receiver, in terms of data freshness - Age of Information (AoI). Under the paradigm of Reinforcement Learning, this thesis proposes and evaluates a Q-learning Multi-Agent algorithm, the AoI-Q-ALOHA method, where the users develop ad-hoc strategies for the minimisation of their local mean AoI. Without any communication with the others, except for the central receiver, the agents will learn on a binary success/collision feedback. Team behaviour is encouraged through a tailored individual reward designation, without any assumption on the network population. We compare the performance in terms of the overall AoI, Throughput and Fairness index, to the standard slotted-ALOHA protocol, and to the threshold-ALOHA policy, a benchmark protocol that resorts to a central optimization of the access parameters. Interesting insights on the distributed setup are derived, as well an exhaustive survey on the properties and robustness of the algorithm.