Standard and Riemannian-Inspired Reward Shaping for MADDPG--MPC-CBF Navigation

The development of compact and sustainable micro-factories for Micro-Electromechanical Systems (MEMS) motivates the study of cooperative robotic systems capable of operating safely and precisely in constrained workspaces. In this context, teams of autonomous mobile robots must combine coordination, task efficiency, and collision avoidance while remaining compatible with high-precision manipulation and visual-servoing tasks. Multi-Agent Reinforcement Learning (MARL) offers a promising framework for learning cooperative behaviours, but purely learning-based policies generally do not provide formal safety guarantees. Conversely, model-based safety-critical control methods, such as Control Barrier Functions (CBFs), can enforce safety constraints but are not designed to learn complex cooperative strategies from interaction data. This thesis proposes a hierarchical learning-based planning and model-based safety-filtering architecture for cooperative multi-robot navigation. The framework combines Multi-Agent Deep Deterministic Policy Gradient (MADDPG) with Model Predictive Control subject to Control Barrier Function constraints. Within a Sense-Plan-Act structure, MADDPG is used as a high-level nominal planner for target reaching and cooperative landmark coverage, while the downstream MPC-CBF layer modifies the nominal motion commands to enforce safety-related constraints during execution. This separation allows the learning component to focus on coordination and task performance, while the model-based controller handles safety-critical corrections. A central contribution of the thesis is the design and simulation-based evaluation of reward-shaping strategies for cooperative multi-agent navigation. A standard Euclidean reward formulation is compared with a Riemannian-inspired geometry-aware reward, designed to shape the learning signal through configuration-dependent geometric quantities. The objective is to encourage smoother and more coordinated nominal behaviours while preserving compatibility with the downstream CBF-based safety filter. The reward design, training procedure, hyperparameters, curriculum learning strategy, and the complete MADDPG--MPC-CBF pipeline are evaluated in simulation using a Multi-Agent Particle Environment based on the OpenAI Gym interface. The proposed architecture is then implemented in ROS2 and experimentally assessed as a proof of concept at the Fuchiwaki Laboratory, Yokohama National University. The overall pipeline is designed to be compatible with custom-made Holonomic Beetle robots intended for sub-millimetric precision applications in MEMS-oriented micro-factory environments. For the proof-of-concept experiments presented in this thesis, however, the physical validation is conducted using RoverC.Pro omnidirectional mobile robots, which provide a practical platform for testing the Sense-Plan-Act workflow, global vision feedback, nominal planning, safety filtering, and command execution. The results show that the proposed architecture provides a coherent framework for combining learned cooperative planning with model-based safety filtering. Simulation studies support the role of reward shaping in improving nominal multi-agent behaviours, while the ROS2 implementation and proof-of-concept experiments demonstrate the feasibility of deploying the complete pipeline on physical omnidirectional robots. Overall, this thesis contributes toward safe and coordinated multi-robot systems for future high-precision applications, including automated micro-assembly and visual-servoing tasks in MEMS-oriented micro-factories.