Temporal Convolutional Networks (TCNs) are promising Deep Learning models for time-series processing tasks. One key feature of TCNs is time-dilated convolution, whose optimization requires extensive experimentation. We propose an automatic dilation optimizer, which tackles the problem as a weight pruning on the time-axis, and learns dilation factors together with weights, in a single training. Our method reduces the model size and inference latency on a real SoC hardware target by up to 7.4× and 3×, respectively with no accuracy drop compared to a network without dilation. It also yields a rich set of Pareto-optimal TCNs starting from a single model, outperforming hand-designed solutions in both size and accuracy.

Pruning in Time (PIT): A Lightweight Network Architecture Optimizer for Temporal Convolutional Networks / Risso, M.; Burrello, A.; JAHIER PAGLIARI, Daniele; Conti, F.; Lamberti, L.; Macii, E.; Benini, L.; Poncino, M.. - ELETTRONICO. - 2021 58th ACM/IEEE Design Automation Conference (DAC):(2021), pp. 1015-1020. ((Intervento presentato al convegno 58th ACM/IEEE Design Automation Conference, DAC 2021 tenutosi a USA nel 2021 [10.1109/DAC18074.2021.9586187].

Pruning in Time (PIT): A Lightweight Network Architecture Optimizer for Temporal Convolutional Networks

Risso M.;Burrello A.;Pagliari Jahier Daniele;MacIi E.;Poncino M.
2021

Abstract

Temporal Convolutional Networks (TCNs) are promising Deep Learning models for time-series processing tasks. One key feature of TCNs is time-dilated convolution, whose optimization requires extensive experimentation. We propose an automatic dilation optimizer, which tackles the problem as a weight pruning on the time-axis, and learns dilation factors together with weights, in a single training. Our method reduces the model size and inference latency on a real SoC hardware target by up to 7.4× and 3×, respectively with no accuracy drop compared to a network without dilation. It also yields a rich set of Pareto-optimal TCNs starting from a single model, outperforming hand-designed solutions in both size and accuracy.
File in questo prodotto:
File Dimensione Formato  
Pruning_In_Time_PIT_A_Lightweight_Network_Architecture_Optimizer_for_Temporal_Convolutional_Networks.pdf

non disponibili

Descrizione: Articolo principale
Tipologia: 2a Post-print versione editoriale / Version of Record
Licenza: Non Pubblico - Accesso privato/ristretto
Dimensione 1.62 MB
Formato Adobe PDF
1.62 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
pit_open.pdf

accesso aperto

Tipologia: 2. Post-print / Author's Accepted Manuscript
Licenza: PUBBLICO - Tutti i diritti riservati
Dimensione 1.7 MB
Formato Adobe PDF
1.7 MB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

Caricamento pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/11583/2959926