CMS urged to examine 'serious flaws' in its star ratings system

The AHA July 7 and three other national hospital groups called on the Centers for Medicare & Medicaid Services (CMS) to examine the “serious flaws” in the agency’s hospital quality “star ratings” methodology.

“Our continued review of the limited information available to us has raised serious questions about the ability of the proposed ratings approach to provide accurate and meaningful information to patients,” the hospital associations wrote CMS.

CMS on April 20 announced that it was delaying until at least this month the release of overall hospital quality star ratings on its Hospital Compare website. The ratings were originally scheduled to be released April 21 – but were delayed in response to concerns raised by the AHA and others, including a majority of both the House and Senate, about whether the methodology provides a fair, accurate and meaningful representation of hospital performance.

The hospital groups’ letter faulted CMS for not following up on the delay by addressing hospitals’ concerns about the methodology.

“We urge CMS to share additional information with hospitals and the public about how accurately star ratings portray hospital performance,” wrote the groups, which included the Association of American Medical Colleges, America’s Essential Hospitals and Federation of American Hospitals. “We also urge CMS to address several significant underlying methodological problems with its star ratings.”

The hospital groups urged CMS to hold off publishing the star ratings until it takes those steps. 

The hospital groups’ letter to CMS comes on the heels of a new AHA-commissioned analysis that questions the usefulness of the star rating methodology. The analysis, conducted by Georgetown University economist Francis Vella, noted that different hospitals are rated using a number of different measures, which can bias the results, regardless of their actual performance.

Vella also said the methodology is not well constructed. “While it appears to give the impression of being rigorous and objective, the estimation aspect is highly dependent on choice of measures and the weighting scheme is entirely subjective and highly determinant of outcomes,” he wrote. He also said the methodology ignores social determinants of quality outcomes, like race, income and patient composition, which can skew results.