An online reinforcement learning approach for HVAC control

Heating, Ventilation and Air Conditioning (HVAC) optimization for energy consumption reduction is becoming ever more a topic of the utmost environmental and energetic concerns. The two most employed methodologies for optimizing HVAC systems are Model Predictive Control (MPC) and Reinforcement Learning (RL). This paper compares three different RL approaches to HVAC optimization: one based on a black-box system identification model trained on historical data, one based on a white-box model of a building and one online method based on an imitation learning pretraining phase on historical data. The three approaches are compared with a literature baseline and an EnergyPlus baseline. Results show that the overall best method in terms of energy consumption reduction (65% decrease) and thermal comfort increase (25% increase) is the approach based on the white-box model. However, the proposed methodology, based on online and imitation learning, demonstrates remarkable efficiency, achieving comparable improvements in energy consumption after just a few months of online training, while maintaining thermal comfort at around the same level as the baseline. These results prove a direct online RL approach, which avoid the use of costly simulations, can provide a reliable and inexpensive solution to the problem of HVAC optimization.