Year-long competition will highlight the research staff that publications overlook, says Simon Hettrick
I study how software is used in research. Recently, I started looking at submissions to the 2014 Research Excellence Framework (REF).
Given that about 70 per cent of research relies on software, I expected that the 191,000 outputs submitted would include a few thousand pieces of software.
In fact, there were 38.
This is disappointing for people like me who care about software’s role in research, but it’s a travesty for the people who create the software. Their expertise is being overlooked and undervalued. How can we expect to maintain a world-leading research environment if we ignore the vital contribution of large sections of the research community?
To redress the balance a little, this month saw the launch of the hidden REF, a year-long competition to celebrate all research outputs and the people who make them possible.
Publish or perish
In REF2014, 97 per cent of all research outputs submitted were publications. The percentage has increased steadily with successive Research Assessment Exercises and into the REF.
Publications, of course, are fundamental to the conduct of research and to academia’s contract with the public. The problem arises when they become the only measure of research success.
The research community’s myopic fixation on publications as the metric of research excellence means that the REF overlooks many skilled people who are essential to the conduct of research but do not get authorship of papers.
The REF itself is not at fault. The categories are there—not only for software, but for exhibitions, performances, artworks and so on. The problem is the conservatism of the committees choosing which outputs to submit.
In the course of writing guidance for the next REF in 2021, I talked to a number of researchers and REF managers. They painted a picture of university REF committees under incredible pressure to maximise their score and funding.
They understand publications, and they’ve seen their dominance in previous assessments. Anything else looks risky.
Hidden figures
This is not just a problem with software. I’ve seen technicians, data stewards, imaging scientists and medical statisticians suffer the same lack of recognition. From the arts to zoology, you can bet there are many more of these important, yet hidden, research roles.
I have also seen firsthand that recognising everyone’s contribution improves research. In 2013, the Software Sustainability Institute began a campaign to recognise the contribution of research software engineers. Thanks partly to our efforts, 28 universities now host research software engineering groups, with five new groups set up in 2019 alone.
These groups give thousands of researchers access to people who understand both the research process and software engineering. This is of huge benefit to UK research and it occurred naturally once the importance of the research software engineer was recognised.
The problem with supporting hidden roles is that you have to uncover them. Even in one’s own discipline, this is not easy. It gets harder still if you attempt it across all of research. This is where the hidden REF comes in.
In the competition’s first phase, which runs up to July this year, anyone can suggest a category of research output that is currently unrecognised. There are no limitations to the outputs, so there are no limitations to the roles involved in their creation. Every month, we will announce the new categories and seek feedback about which should be adopted into the competition.
The second phase starts in July when the newly minted categories will open for submissions. These will be judged by expert panels drawn from the research community, and the winners announced in November 2020. The aim is to raise awareness of unrecognised roles in research, not replicate the burden of the REF, so for most categories a 300-word submission will suffice.
The hidden REF has been fortunate to gain the support of a number of organisations, each of which supports communities of research professionals who have lacked recognition: BioimagingUK, the Data Curation Centre, the Imaging Scientist, the Society of Research Software Engineering, the Software Sustainability Institute and the Technician Commitment. Supporters are currently skewed towards the sciences, but we hope this will change as word of the hidden REF gets out and new organisations join us.
Research is about more than publications. By raising awareness of the full range of different and vital outputs, I hope the hidden REF will lead to greater recognition of the variety of people that make research possible. Ultimately, we’re running this competition because we believe a fairer research environment is a more effective one.
A version of this article also appeared in Research Fortnight