In recent years, a wide range of data type representations have been employed for training and storing the parameters of Deep Neural Networks (DNNs). The decision to employ a particular data type over another is influenced by various requirements, including the desire to enhance training accuracy or reduce data size to minimize memory usage, energy and power consumption. However, opting for one data type over another inevitably impacts the reliability of the model. This work studies the impact of different data representations on the reliability of LeNet-5, a popular Convolutional Neural Network (CNN) used for image classification tasks.An investigation is performed to evaluate the efficacy of the Average Bit-Flip Distance (ABFD) in predicting the criticality of bit positions in the data representation. The data type under analysis are FP32, POSIT32, POSIT16 and INT8. Together with the widely adopted metrics, this work proposes a new metric, called Soft SDC-n, to measure the percentage of faults that cause a change in the order of the top-n output elements. Experimental results shows that POSIT is not as reliable as FP32, while indicating that the most reliable data type is INT8. Furthermore, the same results confirm the presence of a relationship between the ABFD and the criticality of a bit in all the data representations under analysis.
On the resilience of representative and novel data formats in CNNs / Gavarini, G.; Ruospo, A.; Sanchez, E.. - (2023). (Intervento presentato al convegno International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems tenutosi a Juan-Les-Pins (FR) nel 03-05 October 2023) [10.1109/DFT59622.2023.10313551].
On the resilience of representative and novel data formats in CNNs
Gavarini G.;Ruospo A.;Sanchez E.
2023
Abstract
In recent years, a wide range of data type representations have been employed for training and storing the parameters of Deep Neural Networks (DNNs). The decision to employ a particular data type over another is influenced by various requirements, including the desire to enhance training accuracy or reduce data size to minimize memory usage, energy and power consumption. However, opting for one data type over another inevitably impacts the reliability of the model. This work studies the impact of different data representations on the reliability of LeNet-5, a popular Convolutional Neural Network (CNN) used for image classification tasks.An investigation is performed to evaluate the efficacy of the Average Bit-Flip Distance (ABFD) in predicting the criticality of bit positions in the data representation. The data type under analysis are FP32, POSIT32, POSIT16 and INT8. Together with the widely adopted metrics, this work proposes a new metric, called Soft SDC-n, to measure the percentage of faults that cause a change in the order of the top-n output elements. Experimental results shows that POSIT is not as reliable as FP32, while indicating that the most reliable data type is INT8. Furthermore, the same results confirm the presence of a relationship between the ABFD and the criticality of a bit in all the data representations under analysis.File | Dimensione | Formato | |
---|---|---|---|
On_the_resilience_of_representative_and_novel_data_formats_in_CNNs.pdf
accesso riservato
Tipologia:
2a Post-print versione editoriale / Version of Record
Licenza:
Non Pubblico - Accesso privato/ristretto
Dimensione
1.53 MB
Formato
Adobe PDF
|
1.53 MB | Adobe PDF | Visualizza/Apri Richiedi una copia |
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/11583/2982812