Cooperative adaptive cruise control and energy management for hevs usign reingorcement learning

The increased number of cars on the road, combined with stricter emissions regulations, necessitates a solution that can improve traffic flow, vehicle stability, passenger comfort, and fuel economy. For a hybrid electric vehicle (HEV), we propose a multi-objective cooperative adaptive cruise control (CACC) based on deep reinforcement learning (DRL). Specifically, as the CACC control actions and energy management system (EMS) power-splitting decisions are interdependent, we envision a multi-agent DRL (MADRL-CACC) solution for the CACC and EMS of HEVs. The framework consists of two layers: the upper layer has a DRL agent, which determines throttle or brake pedal commands considering headway, vehicle stability, passenger comfort, and fuel economy; the lower layer utilizes another DRL agent that estimates the equivalence factor of the equivalent consumption minimization strategy (ECMS), an energy management system. Based on the equivalence factor, ECMS calculates the optimal power split between the internal combustion engine and the electric machine. The proposed controller is compared to a Proportional Feedback controller tuned by a genetic algorithm for ACC and a single agent DRL for CACC applications, respectively. Both controllers use ECMS as their EMS. In simulation, the performance of the controllers is evaluated for urban and highway driving scenarios. In both driving scenarios, the proposed controller outperforms in terms of traffic efficiency, passenger comfort, vehicle stability, and fuel economy. The MADRL-CACC can improve fuel economy by 7.9% in the urban driving cycle, whereas in the highway driving scenario, it improves headway by 39.9% and passenger comfort by 43.5%.