Scientific Journal Editors Don’t Trust The Research; Decades-Old Problem Resurfaces In Viral Video

Emily Kaplan, co-founder of the Broken Science Initiative, has drawn attention to a crisis of replication, corruption, and lost trust when it comes to scientific journals and the reliability of the studies that are being published. The statement is being widely shared on social media and echoes statements made by top medical journal editors 15-20 years ago, including those of the New England Journal of Medicine, the Lancet, and BMJ.

“We have peer-reviewed, high-impact editors in most of the journals that are the most high-impact, saying that they don’t believe what is being published in those journals is trustworthy anymore,” Kaplan said. “The BMJ, The Lancet, all of those editors have come out and said, we have a huge problem. We can’t replicate this research and we actually don’t even know who did the research.” 

The Lancet Editor Richard Horton called scientific journals “information laundering operations for the pharmaceutical industry” in 2004. In 2005, BMJ Editor Richard Smith said medical journals are “an extension of the marketing arm of pharmaceutical companies.” In his essay, he explained that industry-funded trials make up to three-quarters of the studies published in medical journals, and they rarely produce unfavorable results for the sponsor.

Former NEJM Editor Marcia Angell described how pharma companies co-opted academic medicine, including the way trials were designed, analyzed and twisted by the sponsors. Negative results would be buried while favorable results are hyped across the mainstream media ecosystem. “It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines,” Angell said.

Amgen scientists tried to reproduce 53 landmark preclinical cancer and hematology studies in 2005, and the scientific findings were confirmed for only 6 (11%) of them. Research C. Glenn Begley met with one of the lead scientists of a study they failed to replicate. “We went through the paper line by line, figure by figure,” said Begley. “I explained that we re-did their experiment 50 times and never got their result. He said they’d done it six times and got this result once, but put it in the paper because it made the best story. It’s very disillusioning.”

John Ioannidis authored a 2005 paper titled “Why Most Published Research Findings Are False.” He wrote, “Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.”

While Kaplan’s comments are going viral, the replication crisis and industry corruption have been known and documented concerns of the editors of large medical journals for more than two decades. During the 2020 coronavirus pandemic, “trust the science” and “trust the experts” were some of the prevailing phrases.

A 2022 study on how surveys measure trust in science found that polls typically focus on broad, macro-level “trust in science” as an abstract concept rather than probing distrust in scientists or institutions. The public largely supports the scientific method and the pursuit of truth through evidence. Skepticism and distrust in science actually come from the biases, conflicts of interest, and financial incentives of the people and institutions that conduct the research. Journal editors have been sounding the alarm about the perverse financial incentive to reach a predetermined conclusion, which leads institutions to create study methodologies that cannot be replicated.

Medical journals depend on the pharmaceutical industry for financial support. Former BMJ Editor Smith wrote in his 2005 essay that journals receive a substantial amount of money from pharmaceutical advertising, but this is the least corrupting form of dependence. He argued that while the advertisements bring in millions of dollars and can be misleading, readers understand it is bought and paid for by the drug manufacturers.

“The much bigger problem lies with the original studies, particularly the clinical trials, published by journals,” Smith wrote. “Far from discounting these, readers see randomized controlled trials as one of the highest forms of evidence. A large trial published in a major journal has the journal’s stamp of approval (unlike the advertising), will be distributed around the world, and may well receive global media coverage, particularly if promoted simultaneously by press releases from both the journal and the expensive public-relations firm hired by the pharmaceutical company that sponsored the trial.”

Smith also explains that companies will not hesitate to pay a million dollars for reprints of a trial for worldwide distribution. Even if the doctors don’t read them, they will be impressed by the name of the journal that is promoting the drug. He added, “The quality of the journal will bless the quality of the drug.”

A 2012 paper evaluated the number of reprint orders for studies in different medical journals. The authors concluded that funding from the pharmaceutical industry is associated with high numbers of reprint orders, another source of income for journals from manufacturers. Six of the seven journals evaluated had reprint orders for an article that earned them over € 100,000, and The Lancet had at least one order exceeding £1.5 million.

While the recent clip of Kaplan’s speech about the unreliability of medical journals is going viral, the problem has been well documented for two decades and prevalent for at least three decades, according to Smith.