Decision concordance with incomplete expert rankings in manufacturing applications

The manufacturing field encompasses a number of problems in which some experts formulate their rankings of a set of objects, which should be aggregated into a collective judgment. E.g., consider the aggregation of (i) the opinions of designers on alternative design concepts, (ii) the opinions of reliability/safety engineers on the criticality of a set of failures, (iii) the perceptions of a panel of customers on alternative aesthetic features of a product, etc.. For these problems, the Kendall’s concordance coefficient (W) can be used to express the degree of agreement between experts in a simple and practical way. Unfortunately, this indicator is applicable to complete rankings only, while experts often find it more practical to formulate incomplete rankings, e.g., identifying only the most/less relevant objects and/or deliberately excluding some of them, if they are not sufficiently relevant or well known. This research aims at extending the use of the traditional W to incomplete rankings, preserving its practical meaning and simplicity. In a nutshell, the proposed methodological approach associates a so-called “midrank” to all objects, even the ones that are not easily comparable with the other ones; subsequently, W can be applied to these midranks. The description is supported by several pedagogical examples.