How to Find Information on Hospitals Quality of Care?

Member news | November 16, 2021

 This #MemberInsights article is authored by Care2Care International

PART ONE                                                            

Transparency in Coverage (TiC) rules will help patients looking for a hospital to find information on prices, as discussed in one of our previous articles.

Quality of care will be one, if not THE most important factor patients will base their decision on. Today, we’re sharing what we found in our research on the topic.

There are numerous publicly available hospital quality rating systems, the top ones are the following:

  1. Hospital Compare:
  2. US NEWS:
  3. Leapfrog hospital safety grades:
  4. Healthgrades:
  5. The Lown Institute rates hospitals not only on quality metrics but on other criteria such as inclusivity, cost-efficiency, etc.
  6. Finally, take a look at the accreditation level delivered by the JCAHO (Joint Commission for Accreditation) and check if the facility earned a Gold Seal for its level of safety

The problem is while using mostly the same data (that from CMS, and other public systems, like Hospitals CAHPS, PSI, NHSN,…) but with distinct methodologies, these organizations come up with conflicting rankings for patients seeking care.

So, we looked at the 4 major ranking systems, using a NEJM (New England Journal of Medicine) study, published in 2019.


The NEJM study published in August 2019 aimed at finding an answer, in a paper titled “Rating the raters”. They used 6 metrics and rated the raters using a composite A to D scale.

Results were disappointing as shown below, and no system could earn an “A” grade:

  1. US NEWS received an overall B
  2. Hospital Compare (CMS) a C
  3. Leapfrog a C-
  4. Healthgrades a non-passing grade of D+

Important caveat: since this research was published, CMS implemented significant changes to its system, but we cannot assess if these changes would have altered their ranking and grade by the NEJM study.

Limits of ranking methods highlighted by the NEJM study

The 6 authors of the study, specialized in these questions, have identified multiple shortcomings in these ranking systems, such as -but not limited t:

  • limited data and measures,
  • lack of robust data audits,
  • composite measure development,
  • measuring diverse hospital types together,
  • lack of formal peer review of their methods,

Additionally, they often observed a considerable disconnect between the top hospitals identified in the rating systems and those thought by clinicians to be major referral centers, like teaching hospitals: Johns Hopkins for instance, didn’t make it to a 5-star rating with CMS Hospital compare. On the opposite side, small hospitals which had incomplete data sets, are graded by CMS to the mean on these data sets, being essentially unable to be identified as poor performers or good performers.

People who have tools provided by their employer or health plan must know that most of them are using the same original data, provided mainly by CMS.

Our next article will provide practical information

Care2Care International started in the medical travel industry so we initially took a very hard look at the different methods used to accredit and rank hospitals. While not being highly skilled experts, we will look closely at the data provided by the different rankings and try to give the reader advice on how to run their research, using these tools and others.