For nearly a decade, several national exercises have been implemented for assessing the Italian research performance, from the viewpoint of universities and other research institutions. The penultimate one – i.e., the VQR 2004–2010, which adopted a hybrid evaluation approach based on bibliometric analysis and peer review – suffered heavy criticism at a national and international level. The architecture of the subsequent exercise – i.e., the VQR 2011–2014, still in progress – is partly similar to that of the previous one, except for a few presumed improvements. Nevertheless, this other exercise is suffering heavy criticism too. This paper presents a structured discussion of the VQR 2011–2014, collecting and organizing some critical arguments so far emerged, and developing them in detail. Some of the major vulnerabilities of the VQR 2011–2014 are: (1) the fact that evaluations cover a relatively small fraction of the scientific publications produced by the researchers involved in the evaluation, (2) incorrect and anachronistic use of the journal metrics (i.e., ISI Impact Factor and similar ones) for assessing individual papers, and (3) conceptually misleading criteria for normalizing and aggregating the bibliometric indicators in use.

Critical remarks on the Italian research assessment exercise VQR 2011–2014 / Franceschini, Fiorenzo; Maisano, DOMENICO AUGUSTO FRANCESCO. - In: JOURNAL OF INFORMETRICS. - ISSN 1751-1577. - STAMPA. - 11:2(2017), pp. 337-357. [10.1016/j.joi.2017.02.005]

Critical remarks on the Italian research assessment exercise VQR 2011–2014

FRANCESCHINI, FIORENZO;MAISANO, DOMENICO AUGUSTO FRANCESCO
2017

Abstract

For nearly a decade, several national exercises have been implemented for assessing the Italian research performance, from the viewpoint of universities and other research institutions. The penultimate one – i.e., the VQR 2004–2010, which adopted a hybrid evaluation approach based on bibliometric analysis and peer review – suffered heavy criticism at a national and international level. The architecture of the subsequent exercise – i.e., the VQR 2011–2014, still in progress – is partly similar to that of the previous one, except for a few presumed improvements. Nevertheless, this other exercise is suffering heavy criticism too. This paper presents a structured discussion of the VQR 2011–2014, collecting and organizing some critical arguments so far emerged, and developing them in detail. Some of the major vulnerabilities of the VQR 2011–2014 are: (1) the fact that evaluations cover a relatively small fraction of the scientific publications produced by the researchers involved in the evaluation, (2) incorrect and anachronistic use of the journal metrics (i.e., ISI Impact Factor and similar ones) for assessing individual papers, and (3) conceptually misleading criteria for normalizing and aggregating the bibliometric indicators in use.
File in questo prodotto:
File Dimensione Formato  
INFORMETRICS VQR Revised_JOI_2016_273.R1 (08-02-2017) no Yellow.pdf

Open Access dal 09/02/2019

Descrizione: INFORMETRICS VQR Revised_JOI_2016_273.R1 (08-02-2017) no Yellow
Tipologia: 2. Post-print / Author's Accepted Manuscript
Licenza: Creative commons
Dimensione 420.63 kB
Formato Adobe PDF
420.63 kB Adobe PDF Visualizza/Apri
INFORMETRICS v.11 n.2, 2017 pp.337-357 VQR 11-14 (FF DM).pdf

non disponibili

Descrizione: INFORMETRICS v.11 n.2, 2017 pp.337-357 VQR 11-14 (FF DM)
Tipologia: 2a Post-print versione editoriale / Version of Record
Licenza: Non Pubblico - Accesso privato/ristretto
Dimensione 1.63 MB
Formato Adobe PDF
1.63 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11583/2666081
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo