All you need is higher accuracy? On the quest for minimum acceptable accuracy for medical artificial intelligence

In this paper we will discuss the concept of the minimum level of accuracy an Artificial Intelligence system must exhibit in medical settings to be fit to its intended use and improve the daily practice of its intended users, the medical doctors. We will consider simple binary classification tasks in both diagnostic and prognostic ambit (like to discriminate between normal/abnormal case, and improvement/no improvement prospects). We will make the point that the common ways to determine this minimum acceptable accuracy are fraught with many conceptual and practical troubles. We will report about a small user study conducted to elicit the discriminative requirements from a sample of medical doctors, stratified both in general practitioners and specialists. Finally, we will present a simple nomogram by which to determine the minimum accuracy of a technological aid, once the human average performance and the desired level of accuracy are known. The nomogram is to be intended as a provocative simple tool to recognize that the technological tool is less important than a sound protocol in which to use it, responsibly and paying due attention to the role of the human decision makers.