The increasing need for small and low-power Deep Neural Networks (DNNs) for edge computing applications involves the investigation of new architectures that allow good performance on low-resources/mobile devices. To this aim, many different structures have been proposed in the literature, mainly targeting the reduction in the costs introduced by the Multiply and Accumulate (MAC) primitive. In this work, a DNN layer based on the novel Sum and Max (SAM) paradigm is proposed. It does not require either the use of multiplications or the insertion of complex non-linear operations. Furthermore, it is especially prone to aggressive pruning, thus needing a very low number of parameters to work. The layer is tested on a simple classification task and its cost is compared with a classic DNN layer with equivalent accuracy based on the MAC primitive, in order to assess the reduction of resources that the use of this new structure could introduce.
A Non-conventional Sum-and-Max based Neural Network layer for Low Power Classification / Prono, Luciano; Mangia, Mauro; Pareschi, Fabio; Rovatti, Riccardo; Setti, Gianluca. - STAMPA. - (2022), pp. 712-716. (Intervento presentato al convegno 2022 International Symposium on Circuits and Systems tenutosi a Austin, Texas nel May 28 - June 1, 2022) [10.1109/ISCAS48785.2022.9937576].
A Non-conventional Sum-and-Max based Neural Network layer for Low Power Classification
Prono, Luciano;Pareschi, Fabio;Setti, Gianluca
2022
Abstract
The increasing need for small and low-power Deep Neural Networks (DNNs) for edge computing applications involves the investigation of new architectures that allow good performance on low-resources/mobile devices. To this aim, many different structures have been proposed in the literature, mainly targeting the reduction in the costs introduced by the Multiply and Accumulate (MAC) primitive. In this work, a DNN layer based on the novel Sum and Max (SAM) paradigm is proposed. It does not require either the use of multiplications or the insertion of complex non-linear operations. Furthermore, it is especially prone to aggressive pruning, thus needing a very low number of parameters to work. The layer is tested on a simple classification task and its cost is compared with a classic DNN layer with equivalent accuracy based on the MAC primitive, in order to assess the reduction of resources that the use of this new structure could introduce.File | Dimensione | Formato | |
---|---|---|---|
iscas2022-samnmax.pdf
accesso aperto
Descrizione: Author's version
Tipologia:
2. Post-print / Author's Accepted Manuscript
Licenza:
PUBBLICO - Tutti i diritti riservati
Dimensione
345.52 kB
Formato
Adobe PDF
|
345.52 kB | Adobe PDF | Visualizza/Apri |
A_Non-conventional_Sum-and-Max_based_Neural_Network_layer_for_Low_Power_Classification.pdf
non disponibili
Descrizione: Editorial version
Tipologia:
2a Post-print versione editoriale / Version of Record
Licenza:
Non Pubblico - Accesso privato/ristretto
Dimensione
356.34 kB
Formato
Adobe PDF
|
356.34 kB | Adobe PDF | Visualizza/Apri Richiedi una copia |
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
https://hdl.handle.net/11583/2973317