Dealing With Challenged IoT Networks in Hierarchical Federated Learning

Federated Learning has revolutionized the way in which mobile devices and IoT can share common knowledge in data analytics. However, some challenges arise when dealing with heterogeneous and challenged networks, especially in gradient synchronization. For example, some clients (referred to as stragglers) may take much longer to report their output than other nodes. Current solutions addressing the straggling problems either propose a distributed coordination (but introduce new synchronization issues) or deadline-based approaches to discard clients after a fixed deadline (but introduce the problem of determining a suitable deadline). To this end, we propose to set a dynamic deadline in which the central server selects the best IoT nodes via an online learning approach based on predicting the response time of each client. Moreover, to further mitigate synchronization and scalability issues, we also consider a hierarchical approach in which clients send model parameters to intermediate aggregation edge servers. Our results demonstrate that this approach can lower network overhead by 78% compared to the widely adopted FedAvg and 49% to the best alternative. At the same time, the model accuracy is preserved, and the training time in challenged networks is reduced by 52% w.r.t. FedAvg and 32% w.r.t. recent solutions.