AI Eye Charts: measuring the visual acuity of Neural Networks with test images

In the last decade, neural networks (NNs) have established themselves as the foundation of modern computer vision, achieving remarkable performances in a consistent number of tasks, including safety-critical applications such as autonomous driving and healthcare devices. As a consequence, existing devices, such as Graphics Processing Units (GPUs), have evolved and new, specialized AI hardware has been designed to better support the ever-growing demand for computational resources. Since the adoption of NNs in safety-critical tasks requires to ensure that hardware faults are not negatively affecting the application's performance, the use of self-test routines to continuously verify the device's functionality during operation is crucial. However, conventional self-testing methods such as Built-in Self Test (BIST) and Software Test Libraries (STLs) are not ideal candidates for this task, since they require to completely halt the application's operation at test time. For this reason, various methods have been proposed that employ the NN itself to test the hardware it is running on by launching the inference of one or more test images. The aim of this paper is to examine the current state-of-the-art regarding test images, as well as draw a comparison of the methods that have been proposed to choose and/or generate test images.