Yelp reviews: A missing piece of the patient-experience puzzle?

Narrative reviews provide a more complete view of patient experience, researchers say

Read the Advisory Board's take on this story.

Yelp reviews of hospitals reveal valuable insights about the patient experience missed by CMS's HCAHPS scores, according to a new study published in Health Affairs.

Study background

For the study, researchers examined about 1,350 hospitals that had received both HCAHPS scores and reviews on Yelp, the popular website that publishes user reviews of local businesses. In total, the hospitals had received about 17,000 Yelp reviews.

Researchers then used an algorithm to identify 41 patient experience topics that commonly appeared in Yelp reviews. The study examined the relationship between Yelp star ratings, HCAHPS-derived Hospital Compare star ratings, and the prevalence of topics in Yelp reviews.

Among hospitals with at least two Yelp reviews, Yelp star ratings closely correlated with a hospital's HCAHPS score, according to the study. Moreover, Yelp reviews highlighted many topics that correlated with strong HCAHPS performance but were not tracked by the standard HCAHPS survey. Such topics included insurance, billing, and the compassion of staff.

Overall, 80 percent of topics associated with a positive Yelp rating were unrelated to items covered by the HCAHPS survey. According to the study, the topics most strongly associated with a positive Yelp rating were:

  • Caring doctors, nurses, and staff;
  • Comforting care;
  • Surgery/procedures; and
  • Labor and delivery.

The researchers drew three main conclusions from the study:

  • Yelp reviews are a strong predictor of a hospital's overall HCAHPS performance;
  • Yelp reviews cover a range of topics related to patient experience not captured by HCAHPS surveys; and
  • "Current hospital ratings based on HCAHPS may be missing the major drivers of patients’ overall experiences of care."

Moreover, the researchers noted that Yelp reviews have several advantages over HCAHPS scores, including ease of use, real-time patient experience reporting, and the use of  free-form input to capture information that is not well suited to standardized surveys. Also, Yelp reviews—unlike HCAHPS scores—capture information from patients who aren't admitted to a hospital.

"Narrative reviews can supplement data from the HCAHPS survey by providing actionable insights for hospitals to improve patient satisfaction," the researchers concluded.

What it means for hospitals

Study author Raina Merchant, director of the University of Pennsylvania Social Media and Health Innovation Lab, says the study suggests hospital officials need to look in new places for a comprehensive view of their patients' experiences. Yelp reviews, she notes, capture everything down to their interactions with parking attendants.

Ashish Jha, professor of health policy at the Harvard T.H. Chan School of Public Health, agrees that Yelp has "something useful to offer in terms of assessing hospital quality."

But others caution that Yelp ratings may not be meaningful for hospitals with few reviews. "In thousands of reviews, a 4.5 star actually means something," says Niam Yaraghi, a fellow at the Brookings Institution's Center for Technology Innovation, who also studies the topic. "But you cannot trust a product that has only been rated 20 times, especially if it is as important a product such as medical care."

Moreover, Yaraghi notes that patient experience itself is a limited metric. "You could have a talented surgeon with a bad sense of humor, or be at the most modern hospital with a great ambiance but they have horrible surgeons that kill you," he explains (Rice, "Vital Signs," Modern Healthcare, 3/4; Anyaegbunam, STAT News, 3/4; Ranard et al., Health Affairs, April 2016).

The Advisory Board's take

Jessica Suchy, iRound for Patient Experience

Across the past year, more organizations have begun gathering feedback from Yelp and other sources—including Facebook, their HCAHPS vendor, patient/family councils, and patient rounds—for their Patient Experience Steering Committees or Performance Improvement Committees to review.

Yelp ratings can be useful: They provide timely feedback on aspects of patient experience not covered by HCAHPS survey questions, and they allow a broader range of patients to provide feedback.

However, especially given the small number of reviews currently available for some hospitals, organizations should take a holistic approach to reviewing patient feedback. Organizations that actively seek out honest feedback from patients, regardless of the source, and commit to taking action are best equipped to increase patient satisfaction.

Members of the Advisory Board's iRound for Patient Experience can also gather real-time feedback from patients on topics not covered in HCAHPS surveys. iRound members can tailor rounding forms to garner a wider cross-section of patient feedback, and later this year members will also be able to attribute patient satisfaction and feedback to specific physicians. We anticipate this exciting new feature will allow organizations to bring patient satisfaction data back to their physicians and engage them in open dialogue regarding physician-patient communication.

Learn more about iRound


Next in the Daily Briefing

What a 'backdoor' to iPhones could mean for health care

Read now