Multi-agent reinforcement learning for autonomous vehicles (use case : highway on-Ramp merging)

Abstract

We focuses on the application of multi-agent reinforcement learning (MARL) to address the complex task of on-ramp merging for autonomous vehicles (AVs). The main objective is to develop a collaborative policy that enables AVs on both the ramp and highway to effectively merge while avoiding collisions and minimizing traffic trip time delay. A centralized MARL framework is implemented, utilizing a single-agent reinforcement learning (RL) framework as the foundation. The framework is designed to handle dynamic traffic scenarios and employs centralized training techniques and global rewards to foster inter-agent cooperation. The reward function is carefully designed to encourage safe merging, maintain desired speeds, and discourage illegal or unsafe actions for vehicles on both the ramp and the highway. To support experimentation, an open-sourced gym-like simulation environment is created, operating on the SUMO simulator and capable of synchronizing with the CARLA simulator. The environment enables the simulation of various random spawn positions for vehicles and facilitates multiagent simulations. Through comprehensive experiments and evaluations, we have demonstrated the advantages of our MARL framework by comparing it with independent learning methods.