ROAR: Routing Packets in P4 Switches With Multi-Agent Decisions Logic

The soaring complexity of networks has led to more complex methods to efficiently manage and orchestrate the multitude of network environments. Recent advances in machine learning (ML) have opened new opportunities for network management automation, exploiting existing advances in software-defined infrastructures. Advanced routing strategies have been proposed to accommodate the traffic demand of interactive systems, where the common architecture is composed of a data-driven network management schema collecting network data that feed a reinforcement learning (RL) algorithm. However, the overhead introduced by the SDN controller and its operations can be mitigated if the networking architecture is redesigned. In this paper, we propose ROAR, a novel architectural solution that implements Deep Reinforcement Learning (DRL) inside P4 programmable switches to perform adaptive routing policies based on network conditions and traffic patterns. The network devices act independently in a multi-agent reinforcement learning (MARL) framework but are able to learn cooperative behaviors to reduce the queuing time of transmitting packets. Experimental results show that for an increasing amount of traffic in the network, there is both a throughput and delay improvement in the transmission compared to traditional approaches.