Modern Healthcare: The trouble with hospital ratings

'Rating the ratings' on rising in popularity

More consumers may be paying attention to hospital ratings, according to a February JAMA study. But do they actually matter? Speaking with experts, Modern Healthcare's Sabriya Rice examined the differences in hospital rating systems.

Most Americans consider ratings 'important' in doctor selection

Crowded, confused ratings market

A Modern Healthcare online survey found health care executives are largely ambivalent about ratings. About 53% reported that their hospital had received a poor rating from one group and high rating from another group on similar measures during the same time period. Most executives said the ratings held some value, but more than 81% of respondents said too many groups are publishing ratings.

Experts concurred that the number of ratings services—and the ratings themselves—can be confusing. "It can take a considerable amount of digging on some sites to find the metrics and how much they count toward the final rating," said Dr. Ashish Jha, a Harvard University professor of health policy who helps advise The Leapfrog Group. "If a rating program isn't willing to make its methodology completely transparent, then no one should use it."

"I would say that the market is crowded… There are so many out there that patients and organizations have difficulty distinguishing," The Advisory Board Company's Alicia Daugherty added. (See more from Daugherty on why some hospitals are issuing their own report cards.)

In January 2013, the Advisory Board evaluated 12 rating systems, including those from The Leapfrog Group, U.S. News & World Report, CMS's Hospital Compare website, the Agency for Healthcare Research and Quality's (AHRQ) Patient Safety Indicators, and the Joint Commission. It found that groups vary greatly in the data they choose to base ratings on, the evaluation they use to designate grade letters and numerical scores, and the transparency of the rating methodologies.

Rating the ratings

In an effort to discern the quality of ratings, groups have begun releasing their own ratings of the rating systems.

The Association of American Medical Colleges (AAMC) in March issued guidelines on how to evaluate hospital rating systems—guidelines that were later endorsed by the American Hospital Association, the Federation of American Hospitals, and America's Essential Hospitals. AAMC recommends that hospitals make sure that a ratings system offers a clear purpose statement, provides transparent methodology, and designates an intended audience.

AAMC: What to make of each hospital rating system

Meanwhile, the Informed Patient Institute has also rated the usefulness of 70 different hospital rating systems, 70 online physician websites, and 60 nursing home report cards using a letter grade scale. The Patient Institute gave the Hospital Compare's, Leapfrog's, and U.S. News' rating systems a "B," while giving the systems published by Healthgrades and the Joint Commission a "C."

Last fall, the Healthcare Association of New York State (HANYS) added its own evaluation of ratings, called the "Report on Report Cards," which gave rating systems zero to three stars. HANYS gave the Joint Commission's and Hospital Compare's systems three stars, while the Truven Health Analytics', Healthgrades', and Consumer Reports' systems got one star. HANYS game U.S. News & World Report just half a star.

Turning the tables: Hospital group grades hospital rankings

Raters respond to the criticism

Consumer Reports' medical director John Santa told Rice that it is hypocritical for hospitals to be critical of rating systems when they often use favorable rankings to market their facilities. "I chuckle when I get reports that hospital CEOs are worried or confused about ratings… They're not so confused that they are not using comparisons in their own advertising," Santa says.

Some hospital officials have advocated for a more uniform approach to hospital ratings, but the request is likely impractical, says Jeff Rice, CEO of hospital price comparison website Healthcare Bluebook.  

"I don't think there's any way to keep people from innovating," Jeff Rice says, adding that if hospitals disclosed their own quality and cost information, "then others wouldn't spend so much time trying to reinvent the wheel" (Rice, Modern Healthcare, 5/31 [subscription required]).

Next in the Daily Briefing

Reuters: In one year, 145 Medicare patients got 24,000 drug tests

Read now