Debunking Reinforcement Learning for Bandwidth Estimation in Real-Time Communications: When a Simple Regressor Suffices

Bandwidth estimation (BWE) serves as a pivotal mechanism in real-time communications (RTC), supporting congestion control (CC) by governing traffic sending rate and thereby optimizing network performance. In recent years, reinforcement learning (RL) has surged to prominence, transcending conventional approaches as a superior bandwidth estimator. However, as the hype continues to escalate, the relentless pursuit of increasingly sophisticated algorithms eclipses the investigation into the core principles of CC, while the inherent flaws of RL appear to be overlooked. In this work, we rethink the necessity of RL in RTC, and contend that BWE itself, rather than reward-driven learning, is the deterministic factor, thus rendering a regressor sufficient. We first underscore the paramount importance of BWE accuracy and propose a simple feedforward neural network-based regressor with dual training stages: an offline stage imitating the perfect estimator and an online stage accommodating traffic dynamics. Utilizing an advanced RTC platform, we benchmark our solution against multiple state-of-the-art RL-based BWE algorithms. Conclusively, our regressor achieves superior performance, potentially charting a new course for BWE and CC in RTC.