Blog: Understanding CMS’ Changes to Hospital Overall Star Ratings

In 2020, the Centers for Medicare & Medicaid Services (CMS) overhauled the Hospital Overall Quality Star Ratings methodology with the expressed purpose of making ratings more transparent, equitable, stable and predictable. AHA commissioned KNG Health to assess whether these methodological changes have fully achieved CMS' goals, while also estimating expected performance variation and drivers of performance and variation under the new methodology. Based on this analysis, it’s clear that CMS has more work to do to ensure that these ratings are equitable.

The analysis revealed that while CMS’s new Overall Star Ratings methodology preserves some year-to-year stability, ratings remain volatile for hospitals reporting fewer measures. This was particularly true for smaller, rural hospitals and critical access hospitals (CAHs). The analysis also found that under CMS’ new peer grouping approach, 74% of hospitals are scored on all five measure groups, but the remaining 26% split across the 14 combinations of three to four measure groups — meaning a hospital’s star ratings can reflect very different sets of measure groups. This means these ratings still don’t provide an apples to apples comparison of hospitals.

Ongoing systematic assessment of the methodology will be vital, and CMS should evaluate potential approaches that make the basis of ratings more equitable. AHA also urges CMS to consider:

• Moving away from an overall rating, and instead explore approaches for scoring hospitals on individual topics.
• Assessing the peer grouping approach to see whether factors other than number of reported measure groups could be used, such as number of measures, or CAH/Inpatient Prospective Payment System status.
• Ensuring any public messaging aground Overall Star Ratings reflect that the rating is a function of the measures used in scoring as well as other methodological choices.
• Replacing unstable measures that examine rare events with more robust measures of safety and quality so that more hospitals have sufficient data to be compared on more measures.

View the full report here.