How to Spot a Milestone from a Mile Away
The development of the Higgs mechanism in 1964 was a major milestone in physics. This theoretical model provided an explanation for how particles acquire mass, a prediction confirmed by the discovery of the Higgs boson in 2012. And yet a paper by one of its architects—the model’s namesake Peter Higgs—scores a relatively low “disruptiveness” rating on the basis of a commonly used metric for identifying breakthrough results.
A new study has uncovered the reason why the Higgs paper—and others like it—receive low disruptiveness scores [1]. “We found that a single citation can drastically swing the metric values,” explains Yong-Yeol Ahn from the University of Virginia. He and his colleagues have devised a new metric, called the embedding disruptiveness measure (EDM), which uses a global-network approach to identify breakthroughs. The researchers show that EDM values are consistently high for acclaimed papers, such as those associated with Nobel Prizes, implying that the new measure does a better job than previous metrics at spotting milestones.
Papers are often judged by how many papers cite them. But this approach has limitations, Ahn says. Some influential papers don’t garner many citations because they belong to a relatively small field. And some review papers attract a lot of citations even though they aren’t presenting any new results. In 2017, researchers introduced a new citation metric called the consolidation–disruption (CD) index [2]. This now commonly used approach separates citations into two categories: papers that cite the target paper exclusively and papers that cite both the target paper and the references in that paper. A paper scores a high CD-index value—and is deemed disruptive—if it has more of the first category, meaning it serves as a focal point for most of the papers that come after it.
“The CD index was a really neat idea,” Ahn says, noting that it has been instrumental in studying the drivers of innovation. For example, studies have found that small teams tend to produce more disruptive papers than large teams, and that disruptiveness has declined in recent years.
But the CD index can also give unexpected results. In the case of the Higgs mechanism, the paper by Higgs [3] came out in the same year as another paper by François Englert and Robert Brout [4], who independently derived the theory. The Higgs paper cited the Englert-Brout paper (but not vice versa), and this caused it to have a remarkably low CD-index value, one that lies in the bottom 0.1% of the CD distribution. By contrast, the Englert-Brout paper ranks in the top 2.8%.
In their study, Ahn and colleagues found similar cases of undervalued papers. They identified a set of breakthroughs on the basis of two sources: works linked to Nobel Prizes and “Milestone” papers [5] listed by the American Physical Society (APS, which publishes Physics Magazine). Calculating the CD index of these breakthrough papers, the researchers found that most had high values, but a significant fraction (around 10%) had unexpectedly low values.
With more digging, the team found that most of these undervalued breakthroughs came out at the same time as other papers presenting similar results. “We found that a lot of them were simultaneous discovery cases,” says Munjung Kim, graduate student at the University of Virginia. When two similar papers come out at the same time, they often share citation patterns, and thus their CD-index values tend to go down.
Manolis Antonoyiannakis, a bibliostatistics analyst at APS, finds these results fascinating. “The single act of citing a simultaneous discovery by a disruptive paper can ‘punish’ it by designating it as nondisruptive.” He notes that journal editors often encourage researchers to cite simultaneous discoveries.
Ahn, Kim, and their colleague Sadamori Kojaku from Binghamton University in New York developed the new EDM metric to rectify these problems. Rather than focusing on direct citation relationships, the EDM approach uses machine-learning tools to uncover longer-range connections within the network of citations.
The researchers used two sets of papers, one from the Web of Science and the other from the APS journals archive. Their machine-learning algorithm places each paper in a multidimensional citation space. The meanings of these dimensions are not predefined. Instead, the algorithm adjusts the space so that papers connected by citations are placed close together. This type of “embedding” process is commonly used, for example, to represent words on the basis of their relation to other words in text, Kim says.
The placement of each paper is specified by two vectors. The first is a “past vector,” representing the direction—in citation space—where the paper came from. The other is the “future vector” that shows the direction in which the paper leads. The EDM metric is defined by comparing these two vectors. Specifically, the more the future differs from the past, the higher the paper’s EDM value.
The researchers found that—unlike the CD index—the EDM gave consistently high values for breakthrough work recognized by Nobel Prizes and APS Milestone papers. For the Higgs case, the EDM values for both the Higgs paper and the Englert-Brout paper were in the top 4.1% of the EDM distribution. “Our metric is more reliable,” Ahn says. He and Kim hope that the EDM estimates can be used to spotlight important papers that may have been overlooked.
Antonoyiannakis says it makes sense that the EDM approach performs better than the CD index. “A network, by definition, contains more information than what is contained in the direct forward–backward links of any focal paper.”
“Understanding disruption in science has attracted enormous attention in recent years, and this work offers a fascinating new approach to measuring it,” says Dashun Wang of the Kellogg School of Management at Northwestern University in Illinois. He explains that a foundational idea in the science of science is that important discoveries are often made by multiple groups rather than by single “geniuses.” Finding systematic evidence of simultaneous discoveries has been challenging, Wang says, but the new metric could help uncover more of them.
–Michael Schirber
Michael Schirber is a Corresponding Editor for Physics Magazine based in Lyon, France.
References
- M. Kim et al., “Uncovering simultaneous breakthroughs with a robust measure of disruptiveness,” Sci. Adv. 12, eadx3420 (2026).
- R. J. Funk and J. Owen-Smith, “Dynamic network measure of technological change,” Manag. Sci. 63, 791 (2017).
- P. W. Higgs, “Broken symmetries and the masses of gauge bosons,” Phys. Rev. Lett. 13, 508 (1964).
- F. Englert and R. Brout, “Broken symmetry and the mass of gauge vector mesons,” Phys. Rev. Lett. 13, 321 (1964).
- Milestones from APS journals, from Physical Review Letters, from Physical Review A–D, and from Physical Review E.