The issue of journal article retractions has been getting a lot of press lately. Why? Because people love reading about controversy, especially when it takes our most venerated institutions down a few notches. Unfortunately, this issue hasn’t come under much scrutiny yet. We’ve been taking the word of a few researchers that a big problem exists and amplifying the media reports of this problem without questioning the conclusions of either the researchers or the journalists.
First a bit of background: There are currently around 25,000 peer reviewed science journals that publish over 1.5 million scholarly articles every year. Both of these quantities—the journals and published articles—have exploded over the last few decades, although definitive estimates are hard to come by. In the most recent widely-cited recent study casting stones at science research—an article by Ferris Feng, Grant Steen, and Arturo Casadevall published in the October 6, 2012 issue of the Proceedings of the National Academy of Sciences (PNAS)—the authors point out that the retraction rates for journal articles have increased 10-fold since 1975. Fraud and suspected fraud were highlighted as the most common cause of retractions, accounting for about 43 percent (889 cases) of the 2,047 retractions they discovered in the 25 million biomedical and life-science research articles indexed by PubMed since the 1940s (the first retraction appeared in 1977).
This isn’t great news for science. But the elephant in the room is the non-story—the fact that out of all the articles published last year alone, everywhere in the world, in every language and on every subject (and remember, we’re talking about 1.5 million of these) it’s possible that only 75 or so—or 0.005 percent if we apply the authors’ findings to the broader universe of journal articles (including those indexed by PubMed)—were retracted for a variety of reasons, and not just fraud. Journal articles were also retracted because of errors, duplication, plagiarism, ethics violations and other causes, and the retraction rates in biomedical research were higher than in most other fields, and varied by geography as to the specific causes. That’s like walking into your neighborhood Barnes & Noble, and in this wall-to-wall mass of published work covering acres of shelf space on every subject known to man, finding just seven books that contain suspect information, plagiarized passages, or calculation errors. This might be touted as an impressive statistic in most circles, not a dire warning.
Is it news, then, that there have been a few hundred cases of research fraud over the last 36 years? Or let’s be cautious and say that maybe there were a few thousand cases. It’s not comforting but it’s also not the sky-is-falling type of news that should inspire the soul-searching that has occurred in science since reports of this research were published in late 2012 (this research wasn’t the first to highlight the issue but it marked the crescendo of a busy year of researchers making similar accusations). But what makes these particular findings even less newsworthy is that the 889 cases of fraud and suspected fraud noted in this study were largely perpetrated by a few bad actors. A mere 38 labs with five or more retractions accounted for 44% of the total number of retractions due to fraud or suspected fraud (390 cases out of 889), and 17 researchers with 10 more retractions accounted for fully 37% of the total (325 cases). What’s more, the definition of fraud cited in this paper was broadened to include everything from “conclusions set forth…cannot be relied upon” to “critical data can’t be reproduced,” to “errors in calculations,” to contaminated samples, inappropriate data collection methods, ethics violations, and more. And the authors’ conclusions about a 10-fold increase in retractions since 1975 is sensationalist at best since retractions didn’t even exist before 1975; they are comparing the large publishing industry and robust oversight and protection capacity of today’s science with the far less developed systems of 36 years ago. It would be just as accurate to claim that there’s been an infinite-fold increase in retractions since 1974. It’s apples and oranges.
So while the public has been led to believe that something widespread and sinister is happening in science, the findings of this research paper simply don’t justify making this kind of conclusion. Is this the fault of the research team or the journalists who have written about this study? Perhaps both. The data do not support making sweeping conclusions and media reports about this research haven’t bothered to dig deeper—maybe it’s just too good of a story (or maybe the fact that this research paper—like far too many other papers—is locked up behind a pay wall is an impediment to actually reading it). In an ironic twist, this research and reporting on research fraud has evolved into something of a fraud itself.
But there is a story here—it’s just more nuanced. How so? Read this PLOS One article by Michael Grieneisen and Minghua Zhang which was published at roughly the same time as Feng’s PNAS article. Grieneisen and Zhang do a much better job of ferreting out the true scope of the fraud issue and its possible causes, noting that fraud needs to be precisely defined (since most cases of fraud don’t also include scientific misconduct), that retraction rates vary by discipline, that the incidence of fraud is more prevalent in just a handful of journals and bad actors, that publication inflation is playing a role in the analysis, that most retractions occur at least four years after publications (meaning that simply discovering a lot of fraud in one year doesn’t mean fraud is increasing in that year), and more.
Both papers agree that there is an issue, although they disagree on the scope and trend, and both conflict with news accounts that make fraud seem like an endemic issue in science.
Is there a problem? Yes. How big is it, is it out of control, and what can be done about it? These are all good questions. No one knows for sure yet. Why does science fraud occur and how it can be rooted out? It has always occurred, of course, driven by human nature—our desire to be the first to discover something, to be recognized, to make money. We don’t need a research paper to confirm this suspicion. But the exact scope of this problem and whether our institutions and practices are contributing to it is entirely unknown (note: nSCI’s upcoming communications survey will try to get at this question). Author Tony Shan raises this question in his new post: whether the pressure to publish, to publish first, and to make this work seem newsworthy—to spin discoveries beyond their scientific merit—may be the real culprits here.
For more background on this story, you can download the Grieneisen and Zhang study here, and also read this recent New York Times article describing the science fraud issue and possible solutions in more detail.
