Hospitals challenge methods behind Leapfrog's safety grades

Health leaders complain about use of outdated metrics, flawed methodology

Echoing frustrations voiced this summer, a handful of hospitals this week challenged The Leapfrog Group's safety grades, calling the methodology flawed and noting that the system favors hospitals that participate in the organization's voluntarily reporting system.

Of the 2,618 hospitals graded in Leapfrog's latest Hospital Safety Scores, 146 for the first time received a "D" or an "F" grade, while 1,004 got a "C" grade.  The not-for-profit organization awards the letter grades based on Hospital Safety Scores determined by hospitals' performance on 26 measures of hospital safety data from the Leapfrog Hospital Survey, CDC, CMS, the Agency for Healthcare Research and Quality, and the American Hospital Association's Annual Survey.

Tom Rosenthal—CMO of Ronald Reagan UCLA Medical Center—takes issue with the weights given to certain measures. For example, he says that the California hospital's score was brought down because of one case involving an air embolism in a transplant patient in 2010. Leapfrog gave the hospital an "F" grade for safety, but Rosenthal says it would have received at least a "C" if not for that single case.

Moreover, Rosenthal says that the organization's methodology favors hospitals that report voluntary data to it. "[Y]ou can't draw any other conclusion when 35% of [Leapfrog's] equation is derived from self-reported measures," he says.  

In addition, Doug Patten—SVP of medical affaris at Phoebe Putney Memorial Hospital—argues that hospitals' safety grades do not reflect current care quality. "Much of what Leapfrog is using is three or four years old… and is based on some proprietary methodology, capriciously assigning adverse grades to someone," he says.

Speaking for smaller, rural hospitals, James Barrett—CEO of 38-bed Richardson Memorial Center in Rayville, La.—noted that "a lot of the things [Leapfrog] surveyed for we don't have," which can negatively impact a small hospital's scores.  

In June, the American Hospital Association (AHA) took issue with much of Leapfrog's grading system and advised patients to ignore them when making decisions about care.

In an email addressing the updated grades, Nancy Foster—the association's vice president of quality and patient safety—said AHA is "still concerned that they continue to use data that is being retired from Hospital Compare because the measures do not provide an accurate picture of the quality efforts in many hospitals today."

However, Foster notes that Leapfrog "has made some important changes in its scoring methodology, in that they minimize scoring disparities between those that report to AHA" (Clark, Health Leaders Media, 11/28; Terhune, Los Angeles Times, 11/28).

Next in the Daily Briefing

Daily roundup: Nov. 30, 2013

Read now