Investigating Deep Reinforcement Learning Advantages and Response to Different Forecast Error Distribution in Energy Applications

Deep reinforcement learning (DRL) algorithms have demonstrated significant advantages over traditional optimization methods for energy management systems operating under conditions of uncertainty. However, the variability of forecast errors in electricity markets raises critical questions about the generalizability of DRL performance across different uncertainty conditions, as DRL responses may vary substantially depending on the nature and distribution of forecasting error inputs. This study compares the effectiveness of Proximal Policy Optimization (PPO) and Particle Swarm Optimization (PSO) for battery energy storage system arbitrage under three distinct forecast error distributions: random, normal, and Ornstein-Uhlenbeck. These distributions represent diverse real-world forecasting scenarios with varying error structures and temporal correlations. Using a one-year dataset of 8,760 hourly time steps from a 4 MWh/2 MW battery system, we demonstrate that DRL consistently outperformed PSO across all tested scenarios. PPO achieved profit improvements ranging from 7.2% under time-correlated errors to 55.4% under random errors. This variation in performance across error distributions shows that DRL behavior significantly adapts to the characteristics of input uncertainty, with the greatest advantages emerging under the most challenging random error conditions. These findings validate that DRL advantages extend across diverse uncertainty regimes and quantify how algorithm performance responds to realistic variations in forecast error behavior.