The growing penetration of renewable energy sources holds great potential for decarbonizing the building energy sector. However, the intermittent and unpredictable nature of renewable generation poses significant challenges to grid stability and energy integration. Demand-side management (DSM) has emerged as a promising solution, leveraging demand flexibility to align energy consumption with periods of peak renewable generation and mitigate grid instability. To fully harness this flexibility, energy coordination across multiple buildings is essential, enabling participation in flexibility markets and optimizing energy management at district level. This paper introduces attention-actor-critic multi-agent deep reinforcement learning (AAC-MADRL), an actor-critic algorithm built upon the centralized training with decentralized execution (CTDE) framework, enhanced with attention mechanisms with the aim of enabling scalable, coordinated, and autonomous DSM in residential districts. A parameterized reward structure allows systematic testing under different cooperation scenarios – fully cooperative, competitive, and mixed – highlighting the conditions where AAC-MADRL outperforms other deep reinforcement learning (DRL) approaches, including decentralized and non-attention-based cooperative methods. Evaluated through winter and summer scenarios in districts across Alameda County, California (73 buildings) and Texas County (100 buildings) using the CityLearn platform, AAC-MADRL demonstrates substantial improvements. AAC-MADRL achieves energy cost reductions of up to 18 % in Texas and 12.5 % in California compared to the rule-based controller. Additionally, it improves self-sufficiency by 6 %–10.5 % during periods of limited solar generation and significantly reduces peak demand. The algorithm also exhibited superior computational efficiency, with deployment times 40.5 % faster than decentralized DRL and 62.5 % faster than cooperative non-attention-based DRL approaches on average.

A scalable demand-side energy management control strategy for large residential districts based on an attention-driven multi-agent DRL approach / Savino, Sabrina; Minella, Tommaso; Nagy, Zoltán; Capozzoli, Alfonso. - In: APPLIED ENERGY. - ISSN 0306-2619. - 393:(2025). [10.1016/j.apenergy.2025.125993]

A scalable demand-side energy management control strategy for large residential districts based on an attention-driven multi-agent DRL approach

Sabrina Savino;Alfonso Capozzoli
2025

Abstract

The growing penetration of renewable energy sources holds great potential for decarbonizing the building energy sector. However, the intermittent and unpredictable nature of renewable generation poses significant challenges to grid stability and energy integration. Demand-side management (DSM) has emerged as a promising solution, leveraging demand flexibility to align energy consumption with periods of peak renewable generation and mitigate grid instability. To fully harness this flexibility, energy coordination across multiple buildings is essential, enabling participation in flexibility markets and optimizing energy management at district level. This paper introduces attention-actor-critic multi-agent deep reinforcement learning (AAC-MADRL), an actor-critic algorithm built upon the centralized training with decentralized execution (CTDE) framework, enhanced with attention mechanisms with the aim of enabling scalable, coordinated, and autonomous DSM in residential districts. A parameterized reward structure allows systematic testing under different cooperation scenarios – fully cooperative, competitive, and mixed – highlighting the conditions where AAC-MADRL outperforms other deep reinforcement learning (DRL) approaches, including decentralized and non-attention-based cooperative methods. Evaluated through winter and summer scenarios in districts across Alameda County, California (73 buildings) and Texas County (100 buildings) using the CityLearn platform, AAC-MADRL demonstrates substantial improvements. AAC-MADRL achieves energy cost reductions of up to 18 % in Texas and 12.5 % in California compared to the rule-based controller. Additionally, it improves self-sufficiency by 6 %–10.5 % during periods of limited solar generation and significantly reduces peak demand. The algorithm also exhibited superior computational efficiency, with deployment times 40.5 % faster than decentralized DRL and 62.5 % faster than cooperative non-attention-based DRL approaches on average.
File in questo prodotto:
File Dimensione Formato  
1-s2.0-S0306261925007238-main_compressed-1.pdf

accesso aperto

Tipologia: 2a Post-print versione editoriale / Version of Record
Licenza: Creative commons
Dimensione 1.57 MB
Formato Adobe PDF
1.57 MB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11583/3000402