There's an often overlooked bias in scientific research that can perpetuate "bad science" for a long time to come, and it centers around "how research is published and used in supporting future work," Aaron Carroll, writes in the New York Times' "The Upshot."
Carroll is a professor of pediatrics at Indiana University School of Medicine and a prominent health care columnist.
According to Carroll, biases that influence how research is published and spun can be "even more pernicious" than commonly discussed biases like a researcher's undisclosed ties or financial conflicts.
For instance, Carroll cites a recent study that identified four common publication biases in research on antidepressants. The study, published in Psychological Medicine, analyzed 105 FDA-registered studies of antidepressants to determine which trials were eventually published in medical literature and which remained hidden from the public.
All told, the researchers identified four common biases that could influence whether a trial is ultimately published and how it's spun.
These biases are not unique to antidepressant research, Carroll writes.
According to Carroll, the systematic reviews of studies with research biases "provides empirical evidence that the biases are widespread and cover many domains," and these biases often paint a more positive picture of study results than what was actually found, which can result in the dissemination of biased research.
According to Carroll, study preregistration could help researchers control for these biases.
Study preregistration requires authors to describe the study, the hypothesis, the data that will be collected, and the analysis process before any data is collected for the study.
When the study is complete, reviewers compare the completed study to the preregistered version. If the versions are similar, the results are published—regardless of the outcome.
However, Carroll noted that preregistration only "works sporadically." A 2011 study on preregistered research found that up to half of the publications omitted primary outcomes after the study was complete. While there could be valid reasons for the adjustments, Carroll says that "too often, there are no explanations."
While a lot of medical studies are influenced by research bias, Carroll writes that we shouldn't "discount all results from medical trials." Instead, "we need, more than ever, to reproduce research to make sure it's robust," he writes.
Carroll believes authors should be held to more "rigorous standards" to report results that are accurate and transparent, regardless of whether they are negative or positive. Doing so requires building an accepting culture: "We can celebrate and elevate negative results, in both our arguments and reporting, as we do positive ones."
Unfortunately, this is easier said than done, as "[t]hese actions might make for more boring news and more tempered enthusiasm," Carroll explains. "But they might also lead to more accurate science" (Carroll, "The Upshot," New York Times, 9/24).
Been awhile since your last statistics class? It can be difficult to judge the quality of studies, the significance of data, or the importance of new findings when you don't know the basics.
Download our cheat sheets to get a quick, one-page refresher on some of the foundational components of evidence-based medicine.
Create your free account to access 2 resources each month, including the latest research and webinars.
You have 2 free members-only resources remaining this month remaining this month.
Never miss out on the latest innovative health care content tailored to you.