Reinforcement Learning for charging scheduling in a renewable powered Battery Swapping Station

Battery Swap (BS) technology represents a promising solution to overcome the main obstacles to a widespread adoption of electric vehicles (EVs) in a urban environment, like the limited range of EVs and the long battery charging time. Furthermore, with respect to traditional charging stations, it offers higher flexibility in dynamically managing the EV electricity demand to prevent the risk of power grid overload. Nevertheless, proper scheduling of the battery charge process is crucial to offer effective e-mobility services, trading off cost, Quality of Service and feasibility constraints. In this paper we consider a renewable powered multi-socket Battery Swapping Station (BSS) and design two algorithms based on Approximate Dynamic Programming (ADP) and Reinforcement Learning (RL) to dynamically adapt the scheduling of the battery charging process to the stochastic nature of the system. Both approaches are proved to be effective in remarkably enhancing the service quality in terms of increased capability to satisfy the customer demand for EV battery charging, at a lower cost with respect to benchmark approaches, with RL outperforming ADP under any budget constraint. In particular, under RL the probability of not satisfying the EV demand can be decreased by up to more than 40% with respect to benchmark approaches, and a significant cost reduction of almost 20% can be achieved, jointly with a greener system operation. Furthermore, our results show that a fine tuning of hyper-parameters is fundamental to properly trade off cost and Quality of Service constraints according to varying business needs. Finally, we analyse how the proposed strategies may affect the battery health due to their impact on battery degradation, hence influencing the BSS management cost.