Optimal deployment of deep neural networks (DNNs) on state-of-the-art Systems-on-Chips (SoCs) is crucial for tiny machine learning (TinyML) at the edge. The complexity of these SoCs makes deployment non-trivial, as they typically contain multiple heterogeneous compute cores with limited, programmer-managed memory to optimize latency and energy efficiency. We propose HTVM - a compiler that merges TVM with DORY to maximize the utilization of heterogeneous accelerators and minimize data movements. HTVM allows deploying the MLPerfT Tiny suite on DIANA, an SoC with a RISC-V CPU, and digital and analog compute-in-memory AI accelerators, at 120x improved performance over plain TVM deployment.

HTVM: Efficient Neural Network Deployment On Heterogeneous TinyML Platforms / Van Delm, Josse; Vandersteegen, Maarten; Burrello, Alessio; Sarda, Giuseppe Maria; Conti, Francesco; Pagliari, Daniele Jahier; Benini, Luca; Verhelst, Marian. - (2023). (Intervento presentato al convegno Design Automation Conference (DAC) 2023 tenutosi a San Francisco, CA (USA) nel 09-13 July 2023) [10.1109/dac56929.2023.10247664].

HTVM: Efficient Neural Network Deployment On Heterogeneous TinyML Platforms

Burrello, Alessio;Pagliari, Daniele Jahier;
2023

Abstract

Optimal deployment of deep neural networks (DNNs) on state-of-the-art Systems-on-Chips (SoCs) is crucial for tiny machine learning (TinyML) at the edge. The complexity of these SoCs makes deployment non-trivial, as they typically contain multiple heterogeneous compute cores with limited, programmer-managed memory to optimize latency and energy efficiency. We propose HTVM - a compiler that merges TVM with DORY to maximize the utilization of heterogeneous accelerators and minimize data movements. HTVM allows deploying the MLPerfT Tiny suite on DIANA, an SoC with a RISC-V CPU, and digital and analog compute-in-memory AI accelerators, at 120x improved performance over plain TVM deployment.
2023
979-8-3503-2348-1
File in questo prodotto:
File Dimensione Formato  
Conf___TVM_DORY_Compiler.pdf

accesso aperto

Tipologia: 1. Preprint / submitted version [pre- review]
Licenza: PUBBLICO - Tutti i diritti riservati
Dimensione 3.26 MB
Formato Adobe PDF
3.26 MB Adobe PDF Visualizza/Apri
HTVM_Efficient_Neural_Network_Deployment_On_Heterogeneous_TinyML_Platforms.pdf

non disponibili

Tipologia: 2a Post-print versione editoriale / Version of Record
Licenza: Non Pubblico - Accesso privato/ristretto
Dimensione 3.96 MB
Formato Adobe PDF
3.96 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11583/2991620