Gender disparities in research evaluation in Italy

We present a gendered analysis of the results of two assessment processes adopted by the Italian Ministry of University and Research over the last decade, to allocate research funds and identify researchers qualifying for career progression respectively. The processes are known as Evaluation of Research Quality (VQR, 2004-2010), and National Scientific Qualification (ASN, 2012-2013). The few criteria chosen for the assessments were different in the two cases, both based on the scientific production of the researchers. In both cases we observe, particularly in bibliometric research fields, that the criteria benefited the male gender. More precisely, in the VQR each structure was evaluated by summing the scores of three selected publications of all its researchers in each research field. Although overall such choice turns out to slightly advantage men, the presence of a vertical segregation is very evident when disaggregating the scores by role (researcher, associate professor and full professor): in each role, women average score outperforms that of men for most research field. Despite such segregation, however, the ASN qualifications did advantage again men in career progression. Indeed, in this case aspirants to the role had to overcome three medians calculated on quantitative features of the publications of researchers already in service. We found that such medians were on average higher of about 10% for men, with a presumed much larger impact in terms of actual promotions. The present analysis confirms that bibliometrics, like other seemingly neutral forms of algorithmic assessment, can produce gender inequalities. While the comparison between the two evaluation processes shows how the choice of different indicators in the two cases has led to a different evaluation of even the same researchers.