University leaders pay close attention to comparative rankings such as those offered by Times Higher Education, ShanghaiRanking Consultancy and others. Rankings influence student matriculation numbers, attract talented faculty and justify donations from wealthy donors. University leaders rail against them, and some schools “withdraw” from them, but rankings are influential.
A radical shift in the data underlying rankings is about to upend the rankings world – largely in favor of China’s position.
For instance, in early 2024, the Leiden University Center for Science and Technology Studies CWTS group issued new university rankings that add open-data sources to the traditional curated list of elite journals that has been the standard. The results show a world turned upside down for university rankings.
Where once the list of universities with the highest scientific impact would have been dominated by U.S. and U.K. schools including Cambridge, Stanford, Harvard and MIT, the new top 10 list of universities with high scientific impact includes six universities from China.
What does this transformation mean for understanding scholarly excellence? I study the global research system and its contribution to social welfare. China’s swift progress in science and technology, propelled by investments in research and university strength, has alarmed the United States and other nations. Concerns are mounting that the U.S. may be losing its competitive advantage to an assertive rival, with potential implications for national security, economic standing and global influence. These new rankings will likely raise even more alarm.
Broader range of more sources
The rankings programs draw heavily upon quantitative assessments called “indicators.” A glance at the influential ShanghaiRanking criteria shows the inputs to its assessment include “papers indexed in major citation indices.” The popular indices draw from a highly curated set of scholarly journals such as Cell, The Lancet and Chemical Reviews. The most reputed index collecting information on these and other journals is the Web of Science’s Science Citation Index, or SCI, a product of careful standardization and data enrichment by Clarivate.
SCI represents only a fraction of the work published worldwide, though. Among other critiques, many people decry the SCI’s exclusivity and its perceived Western bias.
But careful curation makes it the gold standard of academic indexing and one that journals and authors aspire to join. Its value is in its replicability: It is possible to dip into it multiple times using different search strategies and produce comparable results.
Reliance on curated databases is about to end with the introduction of rankings based on open data like that collected by OpenAlex. OpenAlex claims to include over 100,000 journals – of highly varying quality and editorial practices – compared with SCI’s 9,200. All data in OpenAlex has been released into the public domain with the laudable goal of making research freely available to all. The downside is that this wider net sweeps in predatory journals that exploit researchers and undermine the quality and integrity of scholarly communication.
CFOTO/Future Publishing via Getty Images
Reflecting China’s research productivity
The volume of scholarly articles represented in the open databases has a mighty influence on China’s position in the open-source rankings. Chinese scholars produce a vast body of written work, some in English, some in Chinese; estimates of percentage shares for language range widely, but hover around 50-50. As China has invested in education and grown its science and engineering capacity, many more people turn out scholarly articles.
From a very small number in the 1980s, China had 2.2 million scientists and engineers by 2023, based on UNESCO data. China’s scholarly output of scientific and engineering articles shows a very rapid rise since the 1990s, with growth outpacing all other nations. Quality has lagged quantity, but China is outproducing the United States in the total number of scientific publications in the Web of Science, by my count – a shift in leadership not seen since the U.S. overtook the U.K. in 1948.
Although the numbers are dated, when I counted China’s scholarly publishing in 2010, my colleague and I estimated that between 2000 and 2009, China published around 1 million scientific papers that were not captured by the Web of Science. That means they didn’t “count” toward traditional rankings. These publications are counted in the new open databases. Many of the papers included in open-source or open-access journals will not be considered of high quality; nonetheless, they become part of the written record.
Open-access publishing services have grown rapidly and offer fast publication times, but there are questions about the quality of their journals. Open publishing services such as MDPI and Frontiers have an outsized number of Chinese contributors compared to those from other countries.
The open-access services often include content from potential paper mills, businesses manufacturing what look like scholarly manuscripts for sale. Despite concerns about the reputation and editorial practices of these publishers and editors, there’s little oversight. These services are flooding the publishing world with vast numbers of lower-quality articles.
Chinese researchers and their sponsoring institutions put a huge premium on publishing in international journals, even those hosted by questionable publishers. Citation stacking practices – when authors cite the works of co-nationals to raise their citation profiles – skew counts to enhance China’s performance.
China is attempting to address malign practices. To its credit, China’s government recently announced the retraction of 17,000 articles with a Chinese author or co-author. Efforts are underway to enhance quality. Governmental payments to researchers for articles in ranked journals are being sunsetted.
Despite the quality questions, the numbers alone will push China up the rankings lists. This rapid shift will enhance China’s position relative to the rest of the world. In itself, the rise does not reflect a change in quality, status or output, but it will continue to stoke the fires of those alarmed by the rise of China in world science, technology and innovation circles, and perhaps put rankings further into question.
The Conversation has updated the headline and the table to correctly report the CWTS Leiden Ranking Open Edition 2023 numbers.
