Major Physics Publisher Goes Double Blind
On a sultry day last year in Mumbai, Mandar Deshmukh, an experimental physicist at the Tata Institute of Fundamental Research, opened the referee reports he had just received for his paper on quantum nanodevices. He was incensed. What got under his skin wasn’t the rejection itself, but the belittling tone—one reviewer called his work a “Friday-afternoon experiment.” “Would the referee have used this language if I had been a US scientist working at MIT?” asks Deshmukh.
Similar suspicions of unfair bias are often voiced in the scientific community, and publishers across the globe have been toying with different options to tackle the problem. Now, IOP Publishing (IOPP), a prominent publisher of physics research, has announced a major shift in its peer-review methods, which, they say, could offer better chances of impartial evaluation. By the end of 2021, IOPP journals will make their default peer-review option “double-blind.” In double-blind review, neither reviewers nor authors know each other’s identities.
Most scientific journals operate in single-blind mode: Reviewers know who has written the paper they are scrutinizing, but the authors don’t know who the reviewers are. Knowing the authors’ identities could be useful to reviewers—for instance, by allowing them to see a new result within the context of previous work. But critics of single-blind review say that this knowledge can also negatively bias a reviewer, who may disfavor lesser-known researchers and institutions, specific nations, or minority groups.
Kim Eggleton, who manages integrity and inclusion initiatives at IOPP, says that this bias, which she believes to be mostly unconscious and unintentional, has cropped up in several studies. “There is a growing body of evidence of bias both in gender and geography,” she says. Some small-scale studies in economics and in biomedical sciences indicate, for instance, that changing author affiliation from a low- to a high-income country significantly improves how a paper is viewed and that referees might set a higher bar for papers authored by women. Another study found that reviews are influenced by homophily, the tendency of people to favor people similar to themselves. This bias bodes well for male authors from North America and Europe, since they are overrepresented in referee pools.
Physics consulted several researchers outside of these regions to seek their perspectives. “Like many Latin American scientists, I felt subtle forms of prejudice throughout my career, although the situation used to be much worse,” says Herbert Vinck Posada, a quantum optics researcher from the National University of Colombia, Bogota.” Deshmukh agrees. “There is certainly some bias, and it becomes more evident as one moves up to the top journals,” he says. “The system favors groups with better exposure,” says Shuyun Zhou, a condensed-matter physicist at Tsinghua University in Beijing.
While the extent of this bias may be difficult to quantify, Eggleton believes that “there are sufficient data to act.” In 2017, IOPP ran a pilot project, offering double-blind peer review as an option for two of its journals, Materials Research Express and Biomedical Physics & Engineering Express. The publisher later extended the program to three more journals. Over time, the uptake of the double-blind option in these journals rose from 10% to over 35%. A survey conducted by the IOPP revealed that even though the rate of acceptance was lower for authors that chose double-blind review, these authors were more likely to say that they were treated fairly compared with authors that chose the traditional system. The same authors also found that the reports that they received were more thorough and clearer, Eggleton says.
Based on this experience, IOP decided to go all the way and make double blind the default review mode for all 53 of its wholly owned journals by the end of 2021. That’s an important step, explains Eggleton, because it avoids potential bias against those authors who choose double-blind review.
Other physics publishers have experimented with double blind review before. The American Physical Society (APS, the publisher of Physics) started offering the option for its journals in 1980. But it ended the program in 2002 because few authors opted for it. Editor in Chief Michael Thoennessen says that diversity, inclusion, and equity are essential parts of APS’ approach to peer review and journals publishing. “We are keenly aware of the importance of these issues,” Thoennessen says, adding that, for example, the journals are promoting the use of gender-neutral language in referee reports, author responses, and the manuscripts themselves.
Nature journals have offered double blind as an option since 2015 and will continue to leave the choice in the hands of the authors, says Andrea Taroni, chief editor of Nature Physics. “But rather than more anonymity, we are exploring transparency as another way to improve peer review,” he says. For example, in Nature Communications, and now also in Nature, authors have the option to publish the peer-review history of their paper, including reviewer comments and rebuttal letters. (Eggleton says that IOPP will also offer this feature for some journals.)
Both Thoennessen and Taroni say that a difficulty with implementing the double-blind process is that it’s hard to conceal author identities, in particular with widespread use of preprints on arXiv. But Eggleton says that she hopes reviewers won’t make an explicit effort to unmask authors, and even if they do, the papers most likely to benefit from a double-blind system tend to be more difficult to identify.
Penny Gowland, a biomedical physicist from the University of Nottingham in the UK, says that implementing the double-blind process “is not just the right thing to do, but it will also benefit science.” In her view, double-blind review will help small, unsung groups publish paradigm-shifting ideas that a single-blind system may not allow to emerge.
–Matteo Rini
Matteo Rini is the Editor of Physics Magazine.