First steps towards micro-benchmarking the Lava-Loihi neuromorphic ecosystem

Neuromorphic computing has proved to be capable of remarkable gains in energy efficiency over traditional architectures. With the continuous development of new software and hardware tools, benchmarking plays a crucial role in the measurement of technological advancement in the field. Over the last few years, some benchmarking efforts for neuromorphic computing have been proposed focusing on specific tasks like noise suppression or gesture recognition, while micro-benchmarking approaches have not been widely investigated. In this paper, we present our proposal to fill this gap: the Lava micro-benchmarking suite, a set of tests specifically designed for the Lava neuromorphic framework and the Loihi 2 neuromorphic architecture. Tests are divided into two broad categories: those aimed at evaluating Lava's message-passing implementation, that are partially inspired by MPI benchmarks, and those specifically designed to test the Loihi 2 architecture and the Lava compilation process. The suite is still a work in progress that needs to be extended to cover more functionalities of Lava and Loihi 2, but in its current state is functional and includes tests covering the three main hardware backends of interest: host CPU, Loihi 2 embedded CPU and Loihi 2 neuron cores. We present the general software design of the suite, the methods we have used to implement the tests and collect measurements and some examples of results obtained from running the tests. We expect that the suite can help Lava / Loihi 2 developers and users with meaningful insights that can be used to improve the state of this neuromorphic ecosystem.