Rapid development of hardware goes hand-in-hand with the advancement of modern computer vision (CV) algorithms. In a typical machine learning operations (MLOps) flow, this continuous evolution of hardware and software is coupled with an active growth in data collected for training. These three pillars of MLOps continue their parallel con- tinuous integration and improvement after an iteration of the deployment has been released. Ideally, the data chosen to improve the next iteration of the deployment is tailored for the future software solution and the future hardware capabilities which enable it. However, here we have a causality problem, where data needs to be collected for a future algorithm from a fleet of deployments which are still running the last iteration of software and hardware. In this paper, we prove that models of previous MLOps iterations are capable surrogates for choosing data for future network architectures running on more capable hardware. We show that surrogate models for the DeepLabv3+ architecture using a ResNet-50 backbone provide a +3.2 p.p. mIoU improvement on average using uncertainty scores over randomly selecting data to train the deployment model on the CityScapes dataset. Further, we show that the type of surrogate has a huge impact on the prediction capability of the deployment model. For instance, the prediction capability of a deployment model, DeepLabv3+, using a MobileNetV3 backbone, can vary by up to +2.4 p.p. on the CityScapes dataset.

Back to the Future: Models as Active Learning Surrogates for Next Generation ML Deployments / Frickenstein, Lukas; Thoma, Moritz; Mori', Pierpaolo; Sampath, Shambhavi Balamuthu; Fasfous, Nael; Vemparala, Manoj-Rohit; Frickenstein, Alexander; Unger, Christian; Passerone, Claudio; Stechele, Walter. - ELETTRONICO. - 1:(2024), pp. 685-699. (Intervento presentato al convegno Proceedings of the 2024 Intelligent Systems Conference (IntelliSys) tenutosi a Amsterdam (The Netherlands) nel 5 and 6 September 2024) [10.1007/978-3-031-66329-1_44].

Back to the Future: Models as Active Learning Surrogates for Next Generation ML Deployments

Pierpaolo, Mori;Claudio, Passerone;
2024

Abstract

Rapid development of hardware goes hand-in-hand with the advancement of modern computer vision (CV) algorithms. In a typical machine learning operations (MLOps) flow, this continuous evolution of hardware and software is coupled with an active growth in data collected for training. These three pillars of MLOps continue their parallel con- tinuous integration and improvement after an iteration of the deployment has been released. Ideally, the data chosen to improve the next iteration of the deployment is tailored for the future software solution and the future hardware capabilities which enable it. However, here we have a causality problem, where data needs to be collected for a future algorithm from a fleet of deployments which are still running the last iteration of software and hardware. In this paper, we prove that models of previous MLOps iterations are capable surrogates for choosing data for future network architectures running on more capable hardware. We show that surrogate models for the DeepLabv3+ architecture using a ResNet-50 backbone provide a +3.2 p.p. mIoU improvement on average using uncertainty scores over randomly selecting data to train the deployment model on the CityScapes dataset. Further, we show that the type of surrogate has a huge impact on the prediction capability of the deployment model. For instance, the prediction capability of a deployment model, DeepLabv3+, using a MobileNetV3 backbone, can vary by up to +2.4 p.p. on the CityScapes dataset.
2024
978-3-031-66328-4
File in questo prodotto:
File Dimensione Formato  
Volume 1.pdf

accesso riservato

Tipologia: 2a Post-print versione editoriale / Version of Record
Licenza: Non Pubblico - Accesso privato/ristretto
Dimensione 870.76 kB
Formato Adobe PDF
870.76 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
_Accept__IntelliSys24___Back_2_the_Future.pdf

embargo fino al 31/07/2025

Tipologia: 2. Post-print / Author's Accepted Manuscript
Licenza: Pubblico - Tutti i diritti riservati
Dimensione 595.04 kB
Formato Adobe PDF
595.04 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11583/2992260