August 27, 2019

When a doctor recommends a treatment, we usually assume "it's based on solid evidence," but research shows that the number of popular clinical treatments that have been proven beneficial is "less than you might think," Austin Frakt writes for the New York Times' "The Upshot."

Cheat sheets: Evidence-based medicine 101

The long history of unproven treatments

Frakt notes that there's a long history of health care providers using unproven treatments.

For instance, Frakt writes, "At 67 years old and a few months shy of three years after his presidency, [former President George] Washington reportedly awoke short of breath, with a sore throat, and soon developed a fever. Over the next 12 hours, doctors drained 40% of his blood, among other questionable treatments. Then he died."

According to Frakt, bloodletting at that time was commonly used to treat a variety of conditions, but "its benefits were based on theory, not rigorous evidence." Frakt writes that, in Washington's case, "bloodletting did little but cause additional misery, and most likely hastened his death."

Frakt notes that we might like to believe that medicine has evolved beyond a reliance on hunches and theories—but "hundreds of years later," he writes, "the same thing still happens."

For example, Frakt writes that in the late 1970s, some doctors thought they'd found a way to treat breast cancer by harvesting bone marrow stem cells from patients and then reintroducing them to the patients after high doses of chemotherapy. According to Frakt, patients battled insurers in court to gain access to the transplants, and in 1994, health plans for federal workers were required to cover the procedure.

But "not a single randomized trial had been done" to show the treatment's effectiveness, Frakt writes, and clinical trials eventually showed that the practice was "ineffective at best, lethal at worst."

According to Frakt, "There are countless other examples of common treatments and medical advice provided without good evidence," including "magnesium supplements for leg cramps, oxygen therapy for acute myocardial infarction, IV saline for certain kidney disease patients," and many others.

He adds, "In some of these cases, there is even evidence of harm."

Frakt cites an analysis by the British Medical Journal that revealed "a great deal" of treatments that are covered by private insurers and public programs are not evidence-based. According to the analysis, which analyzed evidence for thousands of medical treatments, there was "evidence of some benefit" for slightly more than 40% of the treatments, while about 3% were ineffective or harmful, and 6% were found to be unlikely to be helpful."

Further, "a whopping 50% are of unknown effectiveness," simply because "[w]e haven't done the studies," Frakt writes.

Why do doctors recommend unproven treatments?

In some cases, unproven treatments are warranted, according to Frakt. "When there is no known cure for a fatal or severely debilitating health condition, trying something uncertain—as evidence is gathered—is a reasonable approach, provided the patient is informed and consents," Frakt writes.  

In fact, many treatments that have proven effective "were originally experimental," according to Jason Wasfy, an assistant professor of medicine at Harvard Medical School and a cardiologist at Massachusetts General Hospital.

However, "not every experimental treatment ends up effective," Wasfy said, adding, "and many aren't better than existing alternatives." Frakt cites a study by Vinay Prasad, an Oregon Health & Science University School of Medicine physician, finding that 40% of articles that examined an existing medical practice found the practice to be harmful.

And in such cases, "[i]t's an uphill battle" to get providers to change their practice, Frakt writes. "Some practitioners cling to weak evidence of effectiveness even when strong evidence of lack of effectiveness exists," according to Frakt.

Ultimately, Frakt concludes, "an honest assessment of the state of science" underlying medical treatments is "humbling" (Frakt, "The Upshot," New York Times, 8/26).

Cheat sheets: Evidence-based medicine 101

Been a while since your last statistics class? It can be difficult to judge the quality of studies, the significance of data, or the importance of new findings when you don't know the basics.

Download our cheat sheets to get a quick, one-page refresher on some of the foundational components of evidence-based medicine.

Get all 6 Cheat Sheets
Download Now

Topics