Tackling the persistent problem of gender bias in peer review
It should be a straightforward topic for an academic to explore: how do funding agencies carry out peer review of applications they receive?
But researcher Emre Özel is frustrated by the barriers he has repeatedly encountered when trying to answer this question.
In particular, he has found funders unwilling to share the kind of data that would enable researchers like him to look at gender bias in the system. “Gender really matters in all parts of academia, and peer review is one of the most important parts of it,” he says. “We need data to see if there is a bias in the peer review process.”
It is an unavoidable fact that women are persistently less well represented at senior levels across the research sector than at junior levels, and that winning funding is the gateway to climbing the academic ladder.
This has led to increasing focus on women’s participation in successful bids. In the EU’s Horizon 2020 R&D programme, which ran from 2014 to 2020, 36 per cent of research staff on funded projects were women. For its successor programme Horizon Europe, the EU has ruled that most kinds of organisations applying for funding must have gender equality plans in place.
Until last year, Özel was a postdoctoral researcher at the University of Strasbourg, France, working on a project about gender bias in grant allocation funded by the ANR, France’s national research agency. He is the founder of Respublit, an organisation that provides research evaluation and peer review services.
He says that funders, including the Commission and the European Research Council, have repeatedly rebuffed his requests for data on peer reviewer comments on the basis of confidentiality. “It’s all about data transparency,” he says. “If you don’t know what happens during the process, you cannot reduce gender bias.”
An ERC spokesperson told Research Europe that gender balance issues have always been high on the agenda of the funder’s governing body, and that confidentiality of reviewer comments “is key to ensure the integrity” of its grant-making process.
They also said that funders need to comply with the EU’s General Data Protection Regulation. The Commission declined to comment.
Despite such hurdles, in 2022 Özel and colleagues published a study analysing peer reviewer comments in which they wrote of “compelling evidence” of implicit gender bias in a major European funding programme.
The study looked at the Eurocores scheme for collaborative research projects, which ran from 2003 to 2015 and gave out €150 million to 47 multinational research projects. Eurocores was run by the European Science Foundation, which provided data on over 10,000 applicants and the peer review materials they generated.
“We found systematically unfavourable evaluations for consortia with a higher proportion of female [principal investigators],” the study concluded.
Consortia with more women received lower scores in evaluations, but importantly “reviewers did not perceive female scientists as being less competent in their comments”.
This discrepancy between how reviewers talked about projects with a higher share of women and the lower scores that were given to those projects suggests implicit bias was at play, Özel believes.
Despite having worked in peer review management for seven years, Özel says there are still many unknowns about how major funders operate, such as how they pick reviewers and deal with conflicts of interest.
He feels that as many funders give out public money, “they should be transparent”.
“How they allocate these research funds is one of the most important parts of this peer review process, and we have little idea about this,” he says.
According to Özel, the first thing funders do to tackle gender disparities is put more women on funding panels. But this alone will not fix the problem, he says. “It is important, but not enough.”
A more drastic approach advocated by Özel and his colleagues is to tell panel members and reviewers if any biases have been found in their decisions. This might increase evaluators’ awareness of the issue and change future behaviour, they believe.
Özel thinks that it would be possible for funders to do this internally, but one reason they might not is because of pushback from individuals involved in peer review processes.
“You need to be educated to understand if you are unintentionally biased against something,” Özel says. “To be honest, when you talk to professors, the reviewers and the panel members, they don’t want to be educated.”
The other side
Nicolas Walter, chief executive of the European Science Foundation, feels reviewers do want feedback, even if it is critical. “When you are putting time into writing a review, you want to be fair to the applicant,” he says.
Walter was “not surprised” by the results of Özel’s analysis of the Eurocores data, as gender bias in research “is something that has been well documented” and is “known to exist in all assessment processes”.
“We know that there is a gender imbalance in the research landscape,” he says. “It has to come from somewhere—it doesn’t happen by chance—and there are many elements that result in this situation.”
He would like to see more research carried out on recent data, as he feels the situation has improved in the years since the Eurocores data were collected. “There has been positive action on raising awareness and it would be interesting to see whether there is evolution here,” he says.
Ultimately, Walter wants to see more scrutiny of how decisions on funding awards are made. “We need research on research,” he says. “Gender bias is something that is systemic. How do we do peer review? How do we understand the processes and the impact of those processes?”
Hope for the future
There are reasons to be hopeful that European research funders are willing to tackle problems over bias.
Helene Schiffbaenker from the Austrian research company Joanneum Research is co-leading an EU-funded project on gender inequalities in grant allocation processes.
Having worked in the field for some time, Schiffbaenker says she has seen awareness increasing among research funders of their role in shaping careers and building an inclusive science community.
Getting data for her project from funders “was easier than we expected”, with national funders in Austria, Ireland, Poland, Slovakia and Sweden willing to provide written reports from reviewers and grant applications.
“We have funders that are advanced in awareness on exclusion and gender inequality, and we have funders who have just started,” Schiffbaenker says.
But although many funders may be cognisant of problems with gender bias, change can be slow.
“Even when there is a policy [on mitigating gender bias], it’s not clear yet that it arrives in practice,” Schiffbaenker says. “This is a complex and demanding issue that we need to look more at.”
Change may be slow, but there is hope that researchers like Özel will increasingly find it easier to carry out their work. And that could help finally bring unconscious bias against women out into the open, where it can be tackled.
Research assessment is under-going major reform in Europe. The Agreement on Reforming Research Assessment commits signatories to actions such as avoiding the use of metrics and not using institutional rankings when assessing researchers.
One of the suggestions focuses on narrative CVs, which allow candidates applying for research funding to highlight their experience and skills rather than just listing journals they have been published in.
Nicolas Walter, chief executive of the European Science Foundation, believes that the European reforms on research assessment will have an impact on gender bias—particularly narrative CVs. “The idea of narrative CVs is something that is very interesting because it could help to set out the picture of a researcher, of an academic, rather than a man or a woman,” he says.
A version of this article appeared in Research Europe