Statistical heterogeneity in Federated Learning (FL) often leads to client drift and biased local solutions. Prior work in the literature shows that client drift particularly affects the parameters of the classification layer, hindering both convergence and accuracy. While Personalized FL (PFL) addresses this by allowing client-specific models, it can overlook valuable global knowledge. This paper introduces Federated Recursive Ridge Regression (Fed3R), a fast and efficient method to construct a closed-form classifier that effectively incorporates global knowledge while being inherently robust to statistical heterogeneity. Fed3R leverages a pre-trained feature extractor and a recursive ridge regression formulation to achieve exact aggregation of local classifiers and recover the centralized solution. We demonstrate that Fed3R serves as a robust initialization for further fine-tuning with various FL and PFL algorithms, accelerating convergence and boosting performance. Furthermore, we propose Only Local Labels (OLL), a novel PFL technique that simplifies local classifiers by focusing only on locally relevant classes, preventing misclassifications and improving efficiency. Our empirical evaluation on real-world cross-device datasets shows that Fed3R, combined with OLL, significantly improves performance and reduces training costs in heterogeneous FL and PFL scenarios.

Resource-Efficient Personalization in Federated Learning with Closed-Form Classifiers / Fani, Eros; Camoriano, Raffaello; Caputo, Barbara; Ciccone, Marco. - In: IEEE ACCESS. - ISSN 2169-3536. - 13:(2025), pp. 61928-61957. [10.1109/ACCESS.2025.3556587]

Resource-Efficient Personalization in Federated Learning with Closed-Form Classifiers

Fani, Eros;Camoriano, Raffaello;Caputo, Barbara;Ciccone, Marco
2025

Abstract

Statistical heterogeneity in Federated Learning (FL) often leads to client drift and biased local solutions. Prior work in the literature shows that client drift particularly affects the parameters of the classification layer, hindering both convergence and accuracy. While Personalized FL (PFL) addresses this by allowing client-specific models, it can overlook valuable global knowledge. This paper introduces Federated Recursive Ridge Regression (Fed3R), a fast and efficient method to construct a closed-form classifier that effectively incorporates global knowledge while being inherently robust to statistical heterogeneity. Fed3R leverages a pre-trained feature extractor and a recursive ridge regression formulation to achieve exact aggregation of local classifiers and recover the centralized solution. We demonstrate that Fed3R serves as a robust initialization for further fine-tuning with various FL and PFL algorithms, accelerating convergence and boosting performance. Furthermore, we propose Only Local Labels (OLL), a novel PFL technique that simplifies local classifiers by focusing only on locally relevant classes, preventing misclassifications and improving efficiency. Our empirical evaluation on real-world cross-device datasets shows that Fed3R, combined with OLL, significantly improves performance and reduces training costs in heterogeneous FL and PFL scenarios.
2025
File in questo prodotto:
File Dimensione Formato  
Resource-Efficient_Personalization_in_Federated_Learning_With_Closed-Form_Classifiers.pdf

accesso aperto

Tipologia: 2a Post-print versione editoriale / Version of Record
Licenza: Creative commons
Dimensione 12.9 MB
Formato Adobe PDF
12.9 MB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11583/2998609