Lin Zhang and Gunnar Sivertsen ask whether reforms can balance global excellence with local impact
In February, China launched a radical reform of research evaluation and funding.
Two policy documents published by the Ministry of Science and Technology and the Ministry of Education outlined a move away from the current strong focus on bibliometric indicators based on the Web of Science (WoS) database. In their place comes a more balanced combination of measurements and peer review, with a greater emphasis on publishing in domestic journals.
Universities are being urged to implement the policy by the end of July. How to do it, and the possible consequences, have aroused intense discussion among Chinese academics and gained worldwide attention.
The move does not come out of the blue. In 2016, President Xi Jinping called for a more comprehensive evaluation system for individual researchers.
This was followed in 2018 by a joint document from three ministries and two national institutions that proposed moving away from what it called the “four only” approach to evaluation, based on “only papers, only titles, only diplomas and only awards”.
Among these, “only papers” has received the most attention, and for good reason. At all levels in the Chinese research system, there was, and still is, a strong focus on the WoS’s measures of journal impact factors, journal citation reports and highly cited papers.
These indicators have been used as a universal currency in decisions on evaluation, hiring, promotion, awards, rankings and funding. Cash incentives for publishing in WoS-listed journals are widespread. The reforms aim to turn China away from this system.
No easy answers
China’s new policy mirrors initiatives in other parts of the world, such as the San Francisco Declaration on Research Assessment, the Leiden Manifesto for research metrics, and the EU Policy for Responsible Research and Innovation.
None of these initiatives has provided easy solutions. Other countries will therefore recognise both China’s aims and some of the problems they face.
In our view, China has underestimated the need for national coordination and services to replace the WoS as a standard. Here are three problems and their possible solutions:
First, by moving away from basing national research evaluation and funding on an international database, China is empowering its own academic communities, research institutions and funding organisations in defining the principles, criteria and protocols for evaluation.
Essentially, the country is moving from a system based on a commercial product to something self-determined and self-organised. But achieving this will require as-yet uncreated integrated national systems for research information and journal evaluation.
Second, it seems China intends to use methods for individual-level research assessment as a model for evaluation at more aggregated levels, such as institutions, thematic programmes and research sectors.
For example, evaluation at all levels is to be based on a small number of representative publications, of which at least a third should come from domestic journals. We think the new policy needs to differentiate more clearly between appropriate methods at different levels of aggregation.
Depending on the purpose, metrics can be useful at aggregate levels, while data and indicators need not be limited to those in the WoS. Requirements for domestic publishing will also need to vary according to field, type and purpose of research.
Global vs local
Third, until now Chinese researchers have been encouraged to think global in their publishing. Some of them are happy to leave this policy of globalisation behind.
But international peer review has improved the country’s research and international collaboration has increased. Young researchers generally publish more internationally than older colleagues and are more concerned that support for collaborating and publishing abroad will be reduced.
There has been concern in China about locally useful research being published in remote English-language journals. Chinese researchers, for example, published one of the first scientific articles warning of coronavirus in a Western international journal, before the epidemic was widely known in China.
This sparked a more general discussion of where the nation’s scientific results should be published, and became one of the main talking points on social media in the early stages of the outbreak. Those against international publishing argued that it hindered access to knowledge that China needed to control the epidemic.
Such a public reaction is understandable, but time has shown that global exchange of information and advice is crucial to stop the pandemic. China’s challenge is to find a balance between local relevance and the globalisation of research.
See also Scholarly Assessment Reports DOI: 10.29024/sar.15 (2020)
Lin Zhang is a professor of information management at Wuhan University, China. Gunnar Sivertsen is a research professor at the Nordic Institute for Studies in Innovation, Research and Education in Oslo.
A version of this article also appeared in Research Europe