A Suitability analysis of software based testing strategies for the on-line testing of artificial neural networks applications in embedded devices

Electronic devices based on artificial intelligence solutions are pervading our everyday life. Nowadays, human decision processes are supported by real-time data gathered from intelligent systems. Artificial Neural Networks (ANNs) are one of the most used deep learning predictive models due to their outstanding computational capabilities. However, assessing their reliability is still an open issue faced by both the academic and industrial worlds, especially when ANNs are deployed on safety-critical systems, such as self-driving cars in the automotive world. In these systems, a strategy for identifying hardware faults is required by industry standards (e.g., ISO26262 for automotive, and DO254 for avionics). Among the existing in-field test strategies, the periodic scheduling of on-line Software Test Library (STL) is a wide strategy adopted; STL allows to reach an acceptable fault coverage without the need for additional hardware. However, when dealing with ANN-based applications, the execution of on-line tests interleaving the ANN inferences may jeopardise the strive for performance maximization. The paper presents a comprehensive analysis of six possible scenarios concerning the execution of on-line self-test programs in embedded devices running ANN-based applications. In the proposed scenarios, the impact of the STL execution on the ANN performance is analyzed; in particular, the execution times of an inference and the Fault Detection Time (FDT) of the STL are discussed and compared. Experimental analyses are provided by relying on: an open-source RISC-V platform running two different convolutional neural networks; a STL for RISC-V cores with a maximum achievable fault coverage of 90%.