An analysis of existing literature highlights a prevailing consensus that utilizing university rankings in research assessment or policy guidance is not advisable. Transitioning from this academic consensus to reducing reliance on rankings requires a significant cultural shift, which poses considerable challenges.
The study titled “University rankings in the context of research evaluation: A state-of-the-art review,” published on June 7 in the SocArXiv archive, systematically reviewed academic and grey literature. It focused on English-language sources indexed in the Web of Science and Russian academic literature.
Dmitry Kochetkov, associate professor at RUDN University and PhD candidate at Leiden University, intentionally focused on Russia due to rankings’ pivotal role in the Project 5–100 excellence initiative (2013-2020).
The study primarily discusses rankings such as ARWU, QS World University Rankings, Times Higher Education World University Rankings, Leiden Ranking, and U-Multirank. Kochetkov distinguishes between composite indicator rankings, used for league tables, and non-composite indicators like Leiden Ranking and U-Multirank, which are more suitable for monitoring and benchmarking rather than assessment.
Academic Consensus and Criticisms
Most scholarly articles critique rankings on several fronts: methodological errors, lack of transparency, incomplete indicator coverage, territorial biases, and conflicts of interest. For instance, many global rankings are compiled by profit-seeking organizations that offer additional services to universities, potentially influencing rankings.
A minority of articles support rankings or adopt a neutral stance. Some argue that rankings like ARWU provide transparent tools for assessing research performance based solely on objective data.
Challenges in Withdrawing from Rankings
Kochetkov underscores the cultural and reputational risks universities face if they opt out of rankings. Despite criticisms and withdrawals by prestigious institutions from certain ranking systems, rankings retain significant marketing functions and influence.
Recommendations and Future Directions
The study calls for a shift away from using ranking indicators in academic evaluations and strategic university decisions. It advocates for evaluating universities based on their unique missions and contributions rather than rankings. It also advises against using rankings in national strategies and discourages universities from paying for consulting services from ranking compilers due to conflicts of interest.
Expert Opinions and Alternative Approaches
Experts like Ellen Hazelkorn emphasize the flaws of rankings in research evaluation, advocating for alternative metrics and frameworks such as the San Francisco Declaration on Research Assessment and the Leiden Manifesto for Research Metrics.
Conclusion
While rankings provide insights into university reputations and attract students, they also pose limitations and biases. The study encourages a collective reevaluation of rankings at all levels of academia and policymaking, promoting a nuanced approach to assessing research and educational quality beyond ranking systems.
