Weighing the Value of Physical Review Citation Statistics

As the number and range of research projects continue to grow, reference citations have come to play more important roles in establishing the relative merits of those projects. Of course, many readers seized on Sidney Redner's study of citation statistics (PHYSICS TODAY, June 2005, page 49) to see "how well" they are doing personally, but many others must have viewed his results as evaluative on a larger scale. Just how significant are citations?

Speaking for my own interests, I could scarcely be unhappy. Among the top 10 "hot" papers on Redner's list (page 51), 6 are in the field of electronic structure calculations, in which I did my 15 years of research long, long ago. The amazing popularity of that field is explicable in many ways. The methods developed by physicists are extremely accurate and have been widely adopted by chemists, who contribute most of the citations. The new methods are transparent, reliable, transferable, and available in user-friendly packages. Physicists can be proud of having taken such fascinating and complex problems and rendered them readily accessible to researchers who have many other responsibilities.

Still, some aspects of the citation game are disappointing. All the top 10 papers are theoretical. Does this mean that experimental physics is dying? No, it just means that citations don't mean so much, and they should not be used to measure impact or importance of a field or as a facile substitute for understanding how science grows and develops. Citations reflect many incidental factors, including the wish to conform to standard practice, and even reflect the convenience of citing a previously cited paper (without reading it). Experimental discoveries, often published in specialized journals, are still by far the most important part of physics, regardless of how many citations a single paper receives in Physical Review.

Of the famous five papers Albert Einstein wrote in 1905, the one deriving E = mc² is not the most cited. The reason is undoubtedly connected to the fact that E = mc² is all the reference we use when referring to this relationship. A more recent example will prove my point. Kary Mullis's Nobel Prize paper on polymerase chain reaction may well be the most cited paper of all time. But after a while, authors simply use "PCR method" as shorthand with no reference to the author. Moreover, to the typical citing author, actual use of the PCR method has clearly been of much greater value than the typical reference one cites. Citation theorists have paid insufficient attention to this transmutation of citations to acronyms for the paper. Should they not also be counted?

Sidney Redner's analysis of citation statistics appears well thought out, but the assumption that the number of Physical Review citations to other Physical Review papers is even an "approximate proxy of scientific quality" is dubious at best. One nice example is Felix Bloch's 1954 paper showing that sample rotation during a nuclear magnetic resonance measurement can greatly increase the spectral resolution.¹ According to PROLA, the Physical Review Online Archive, Bloch's paper has been cited in PR a total of six times. However, his result is routinely used for all high-resolution NMR and is the parent for many other developments in the field. I do not think this example is isolated.

Reference

1. 1. F. Bloch, Phys. Rev. 94, 496 (1954) [SPIN].

Redner replies: I appreciate the Letters written about my article. To amplify a comment in the letter by J. C. Phillips, only 3 of the top 100 cited articles as of June 2003 are experimental.¹ From my myopic theorist's perspective, it is also surprising that all these articles are old. Given that physics is an experimental science, the caveats of Phillips and of Bryan Suits are well taken.

The letter by Rustum Roy makes the good point that citations can get transmuted to acronyms or to no citations at all, as in the case of the polymerase chain reaction. More dramatically, the most important scientific advances ultimately get incorporated into the canon; thus we never cite Isaac Newton when writing F = ma. It is clear that citations alone are an imperfect measure of the scientific importance of a paper, and one must proceed with caution in developing citation-based productivity measures.

Reference

1. 1. N. Bloembergen et al., Phys. Rev. 73, 679 (1948) [SPIN], #56 in citation rank; E. Wollan, W. Koehler, Phys. Rev. 100, 545 (1955) [SPIN], #66; and E. Hahn, Phys. Rev. 80, 580 (1950) [SPIN], #81.