The increasing need for small and low-power Deep Neural Networks (DNNs) for edge computing applications involves the investigation of new architectures that allow good performance on low-resources/mobile devices. To this aim, many different structures have been proposed in the literature, mainly targeting the reduction in the costs introduced by the Multiply and Accumulate (MAC) primitive. In this work, a DNN layer based on the novel Sum and Max (SAM) paradigm is proposed. It does not require either the use of multiplications or the insertion of complex non-linear operations. Furthermore, it is especially prone to aggressive pruning, thus needing a very low number of parameters to work. The layer is tested on a simple classification task and its cost is compared with a classic DNN layer with equivalent accuracy based on the MAC primitive, in order to assess the reduction of resources that the use of this new structure could introduce.

A Non-conventional Sum-and-Max based Neural Network layer for Low Power Classification / Prono, Luciano; Mangia, Mauro; Pareschi, Fabio; Rovatti, Riccardo; Setti, Gianluca. - STAMPA. - (2022), pp. 712-716. (Intervento presentato al convegno 2022 International Symposium on Circuits and Systems tenutosi a Austin, Texas nel May 28 - June 1, 2022) [10.1109/ISCAS48785.2022.9937576].

A Non-conventional Sum-and-Max based Neural Network layer for Low Power Classification

Prono, Luciano;Pareschi, Fabio;Setti, Gianluca
2022

Abstract

The increasing need for small and low-power Deep Neural Networks (DNNs) for edge computing applications involves the investigation of new architectures that allow good performance on low-resources/mobile devices. To this aim, many different structures have been proposed in the literature, mainly targeting the reduction in the costs introduced by the Multiply and Accumulate (MAC) primitive. In this work, a DNN layer based on the novel Sum and Max (SAM) paradigm is proposed. It does not require either the use of multiplications or the insertion of complex non-linear operations. Furthermore, it is especially prone to aggressive pruning, thus needing a very low number of parameters to work. The layer is tested on a simple classification task and its cost is compared with a classic DNN layer with equivalent accuracy based on the MAC primitive, in order to assess the reduction of resources that the use of this new structure could introduce.
2022
978-1-6654-8485-5
File in questo prodotto:
File Dimensione Formato  
iscas2022-samnmax.pdf

accesso aperto

Descrizione: Author's version
Tipologia: 2. Post-print / Author's Accepted Manuscript
Licenza: Pubblico - Tutti i diritti riservati
Dimensione 345.52 kB
Formato Adobe PDF
345.52 kB Adobe PDF Visualizza/Apri
A_Non-conventional_Sum-and-Max_based_Neural_Network_layer_for_Low_Power_Classification.pdf

accesso riservato

Descrizione: Editorial version
Tipologia: 2a Post-print versione editoriale / Version of Record
Licenza: Non Pubblico - Accesso privato/ristretto
Dimensione 356.34 kB
Formato Adobe PDF
356.34 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11583/2973317