09 March 2018
An article by Marlo M Vernon, E Andrew Balas, and Shaher Momani, ‘Are university rankings useful to improve performance: A systematic review’, has just been published in PLOS ONE. The article reports a review of 13 ranking systems in order to determine their usefulness for measuring research performance. The validity of the rankings was determined by comprehensiveness, consistency, transparency and resistance to bias.
The authors argue that administrators and stakeholders “should look for rankings which are consistent over time, cover multiple areas of measurement and are less reliant on peer reputation” and that rankings that include self-reported data and reputation surveys are less helpful.
They suggest that the Leiden Ranking, Scimago and Clarivate Analytics Most Innovative Universities can be used for evaluation of research performance. They note that U-Multirank does not allow for comparison over time.
It was reported that there is no correlation between research expenditure and research quality, that peer surveys are biased towards elite universities and that indicators such as Nobel prize winners are “luxury”indicators that are of little relevance to most institutions.
In conclusion the authors report that the rankings now available are:
“inadequate to accurately evaluate research outcomes and should be supplemented and expanded to meet standardized criteria. We suggest that future research evaluate three dimensions of research outcomes: scientific impact, economic outcomes, and public health impact for evaluating research performance within an academic institutional environment.”