Structural Health Monitoring (SHM) is a critical task for ensuring the safety and reliability of civil infrastructures, typically realized on bridges and viaducts by means of vibration monitoring. In this paper, we propose for the first time the use of Transformer neural networks, with a Masked Auto-Encoder architecture, as Foundation Models for SHM. We demonstrate the ability of these models to learn generalizable representations from multiple large datasets through self-supervised pre-training, which, coupled with task-specific fine-tuning, allows them to outperform state-of-the-art traditional methods on diverse tasks, including Anomaly Detection (AD) and Traffic Load Estimation (TLE). We then extensively explore model size versus accuracy trade-offs and experiment with Knowledge Distillation (KD) to improve the performance of smaller Transformers, enabling their embedding directly into the SHM edge nodes. We showcase the effectiveness of our foundation models using data from three operational viaducts. For AD, we achieve a near-perfect 99.9% accuracy with a monitoring time span of just 15 windows. In contrast, a state-of-the-art method based on Principal Component Analysis (PCA) obtains its first good result (95.03% accuracy), only considering 120 windows. On two different TLE tasks, our models obtain state-of-the-art performance on multiple evaluation metrics (R2 score, MAE% and MSE%). On the first benchmark, we achieve an R2 score of 0.97 and 0.90 for light and heavy vehicle traffic, respectively, while the best previous approach (a Random Forest) stops at 0.91 and 0.84. On the second one, we achieve an R2 score of 0.54 versus the 0.51 of the best competitor method, a Long-Short Term Memory network.

Foundation Models for Structural Health Monitoring / Benfenati, Luca; Pagliari, Daniele Jahier; Zanatta, Luca; Velez, Yhorman Alexander Bedoya; Acquaviva, Andrea; Poncino, Massimo; Macii, Enrico; Benini, Luca; Burrello, Alessio. - In: IEEE TRANSACTIONS ON SUSTAINABLE COMPUTING. - ISSN 2377-3782. - ELETTRONICO. - (2025), pp. 1-14. [10.1109/tsusc.2025.3592097]

Foundation Models for Structural Health Monitoring

Benfenati, Luca;Pagliari, Daniele Jahier;Zanatta, Luca;Acquaviva, Andrea;Poncino, Massimo;Macii, Enrico;Benini, Luca;Burrello, Alessio
2025

Abstract

Structural Health Monitoring (SHM) is a critical task for ensuring the safety and reliability of civil infrastructures, typically realized on bridges and viaducts by means of vibration monitoring. In this paper, we propose for the first time the use of Transformer neural networks, with a Masked Auto-Encoder architecture, as Foundation Models for SHM. We demonstrate the ability of these models to learn generalizable representations from multiple large datasets through self-supervised pre-training, which, coupled with task-specific fine-tuning, allows them to outperform state-of-the-art traditional methods on diverse tasks, including Anomaly Detection (AD) and Traffic Load Estimation (TLE). We then extensively explore model size versus accuracy trade-offs and experiment with Knowledge Distillation (KD) to improve the performance of smaller Transformers, enabling their embedding directly into the SHM edge nodes. We showcase the effectiveness of our foundation models using data from three operational viaducts. For AD, we achieve a near-perfect 99.9% accuracy with a monitoring time span of just 15 windows. In contrast, a state-of-the-art method based on Principal Component Analysis (PCA) obtains its first good result (95.03% accuracy), only considering 120 windows. On two different TLE tasks, our models obtain state-of-the-art performance on multiple evaluation metrics (R2 score, MAE% and MSE%). On the first benchmark, we achieve an R2 score of 0.97 and 0.90 for light and heavy vehicle traffic, respectively, while the best previous approach (a Random Forest) stops at 0.91 and 0.84. On the second one, we achieve an R2 score of 0.54 versus the 0.51 of the best competitor method, a Long-Short Term Memory network.
File in questo prodotto:
File Dimensione Formato  
fomo_SHM_fullpaper_compressed.pdf

accesso riservato

Tipologia: 2. Post-print / Author's Accepted Manuscript
Licenza: Non Pubblico - Accesso privato/ristretto
Dimensione 4.52 MB
Formato Adobe PDF
4.52 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
fomo_SHM_fullpaper_open.pdf

accesso aperto

Tipologia: 1. Preprint / submitted version [pre- review]
Licenza: Creative commons
Dimensione 5.38 MB
Formato Adobe PDF
5.38 MB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11583/3003612