How accurate do you want it? Defining minimum required accuracy for medical artificial intelligence

Artificial intelligence (AI) is becoming a more and more common component of biomedical engineering solutions, and these latter systems are getting promising results in terms of diagnostic and prognostic accuracy. Medical AI (MAI) is then reaching the maturity level for its appropriate use in clinical practice, but to this end, its efficacy needs to be demonstrated first. Currently, this efficacy is proven in terms of the reported accuracy of the algorithm, especially for diagnostic tasks. But also in this case, how much accurate is "enough" accurate? To address this question means to define the minimum required accuracy for a system to be valid, that is fit to its intended use. To this aim, we propose a risk-based approach to the definition of adequate accuracy, in accordance with a risk-based regulatory classification. We investigated whether the current state of the art is already compliant with this standard-based approach, by performing a literature review in four application domains, one for each of the four risk classes we identified: the diagnosis of psoriasis, of knee osteoarthritis, the screening of breast cancer screening, and the detection of influenza outbreaks. The evaluation of the literature review highlighted that this approach is still not widely adopted, but that there is a partial presence of an implicit, conventional scheme that is similar to our proposal, especially in the high-impact literature. We also provide some guideline to assess the minimum required accuracy but also sheds light on the need for further official guidelines that ensure the wider application of the regulatory risk-based approach by the scholarly community of MAI.