The growing interest in Internet of Things (IoT) and mobile biomedical applications is pushing the investigation on approaches that can be used to reduce the energy consumption while acquiring data. Compressed Sensing (CS) is a technique that allows to reduce the energy required for the acquisition and compression of a sparse signal, transferring the complexity to the reconstruction stage. Many works leverage the use of Deep Neural Networks (DNNs) for signal reconstruction and, assuming that also this operation has to be performed on a IoT device, it is necessary for the DNN architecture to fit in small and low-energy devices. Pruning techniques, that can reduce the size of DNNs by removing unnecessary parameters and thus decreasing storage requirements, can be of great help in this effort. In this work, a novel Multiply and Max&Min (MAM²) map-reduce paradigm trained with the vanishing contributes technique and then pruned with the activation rate method is proposed. The result is a naturally and aggressively pruned DNN layer structure. This structure is used to reduce the complexity of a DNN-based CS reconstructor and its performance is verified. As an example, MAM²-based layers still retain the baseline accuracy of the CS decoder with 94% of the parameters pruned against 25% when using classic MAC-based layers only.
Aggressively prunable MAM²-based Deep Neural Oracle for ECG acquisition by Compressed Sensing / Bich, Philippe; Prono, Luciano; Mangia, Mauro; Pareschi, Fabio; Rovatti, Riccardo; Setti, Gianluca. - STAMPA. - (2022), pp. 163-167. (Intervento presentato al convegno 2022 IEEE Biomedical Circuits and System Conference (BioCAS2022) tenutosi a Taipei, Taiwan nel October 13-15, 2022) [10.1109/BioCAS54905.2022.9948676].
Aggressively prunable MAM²-based Deep Neural Oracle for ECG acquisition by Compressed Sensing
Bich, Philippe;Prono, Luciano;Pareschi, Fabio;Setti, Gianluca
2022
Abstract
The growing interest in Internet of Things (IoT) and mobile biomedical applications is pushing the investigation on approaches that can be used to reduce the energy consumption while acquiring data. Compressed Sensing (CS) is a technique that allows to reduce the energy required for the acquisition and compression of a sparse signal, transferring the complexity to the reconstruction stage. Many works leverage the use of Deep Neural Networks (DNNs) for signal reconstruction and, assuming that also this operation has to be performed on a IoT device, it is necessary for the DNN architecture to fit in small and low-energy devices. Pruning techniques, that can reduce the size of DNNs by removing unnecessary parameters and thus decreasing storage requirements, can be of great help in this effort. In this work, a novel Multiply and Max&Min (MAM²) map-reduce paradigm trained with the vanishing contributes technique and then pruned with the activation rate method is proposed. The result is a naturally and aggressively pruned DNN layer structure. This structure is used to reduce the complexity of a DNN-based CS reconstructor and its performance is verified. As an example, MAM²-based layers still retain the baseline accuracy of the CS decoder with 94% of the parameters pruned against 25% when using classic MAC-based layers only.File | Dimensione | Formato | |
---|---|---|---|
biocas2022-MAMbasedCS.pdf
accesso aperto
Descrizione: Author's version
Tipologia:
2. Post-print / Author's Accepted Manuscript
Licenza:
Pubblico - Tutti i diritti riservati
Dimensione
449.2 kB
Formato
Adobe PDF
|
449.2 kB | Adobe PDF | Visualizza/Apri |
Aggressively_prunable_MAM-based_Deep_Neural_Oracle_for_ECG_acquisition_by_Compressed_Sensing.pdf
accesso riservato
Descrizione: Editorial version
Tipologia:
2a Post-print versione editoriale / Version of Record
Licenza:
Non Pubblico - Accesso privato/ristretto
Dimensione
1.95 MB
Formato
Adobe PDF
|
1.95 MB | Adobe PDF | Visualizza/Apri Richiedi una copia |
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/11583/2973319