Multi-Armed Bandit Learning for ISAC Systems in Time-Varying Environments

Integrated sensing and communication (ISAC) systems aim to jointly perform data transmission and environmental sensing, leading to coupled and often conflicting design objectives. This paper investigates a lightweight online learning framework for ISAC based on multi-armed bandit (MAB) algorithms. A finite beam codebook is used for sequential beam selection, while the sensing–communication trade-off is captured through a normalized reward combining communication rate and a CRLB-based sensing metric. To address time-varying environments, a sliding-window UCB strategy is adopted for beam selection and the trade-off parameter is adapted online on a slower timescale. Numerical results under abrupt changes, gradual drift, and multiple change points show that the proposed approach tracks environmental variations and improves dynamic-regret performance over fixed-α and stationary baselines.