Liberty Place, Suite 700
325 Seventh Street, NW
Washington, DC 20004-2802
(202) 638-1100 Phone
Monday, May 20th 2002
Kenneth W. Kizer, MD, MPH
President and Chief Executive Officer
National Quality Forum
601 13th Street, NW
Suite 500 North
Washington, DC 20005
Dear Dr. Kizer:
The American Hospital Association (AHA), on behalf of its nearly 5,000 member hospitals, health systems, networks, and other providers of care, appreciates the opportunity to review and comment on the National Quality Forum's (NQF) report, "An Initial Measure Set for Hospital Performance Evaluation."
The AHA commends the hard work of the NQF staff and the Hospital Measurement Steering Committee in putting together the initial set of hospital performance measures. We strongly support the project's aim to create a core set of quality measures for hospitals that will be used to assess and improve care. Further, we appreciate the NQF's acknowledgement that there are multiple, competing efforts in this area and that there is a strong need for an agreed-upon consistent set of evidence-based quality measures. We firmly believe that a core set of measures should be used to provide valuable information for benchmarking and that a core measurement set should reduce the redundant and resource-consuming requests for data hospitals are currently subject to from external organizations, including government, accreditors, managed care plans, and employers.
While we are very supportive of the NQF's efforts in this area, we strongly believe that quality data should first and foremost be used to promote learning and improvement. We disagree with the NQF's assertion that the primary purpose should be for "public accountability." Further, while ideally a core set of measures should help ease the burden associated with data collection, the report gives scanty attention to this important issue. The report should be more specific about the need to reduce this burden and the discordant messages sent to hospitals and other providers when different measurement sets are used.
It will also be critical for the NQF to utilize its consensus process to secure an understanding and commitment among private and public entities about what and how much "core data" is important to collect. Part of the NQF's mission was to reduce the hodge-podge of information and resources expended in meeting the demands of disparate measurement efforts. Creating a core set of measures is only half the battle, now we must agree on an over-arching measurement strategy and framework.
We look forward to working with the National Quality Forum on addressing these concerns and providing a framework for how this recommended core hospital measurement set should be used in the future.
Executive Vice President
Click herefor AHA comments.
"An Initial Measure Set for Hospital Performance Evaluation"
Concern: Lack of Framework for Measure Set Use-What's Next?
We have shared and discussed the proposed measurement set widely within our membership. The single most important question raised by state hospital associations and individual hospitals is how will this initial measure set be used and how will it help address the burden and frustration associated with the many competing measurement efforts underway by purchasers, accreditors, state and federal government, and managed care plans [not to mention the many targeted measurement projects hospitals engage in to continually improve their care processes].
The AHA believes it is part of the task of members of NQF members and the NQF's consensus process to engage in the "what's next" discussion. We are very concerned that without such a discussion, it is difficult to comment on the specifics of this report and its recommendations.
The bottom line: it is time for a shared strategy among the many entities involved in requesting, and in many cases, requiring measures from the hospital community. Without a shared strategy and commitment, patient care resources will be wasted, provider frustration will increase, and the public will be confused by conflicting and competing information and information sources.
Recommendation: NQF must finalize the National Framework for Healthcare Quality Measurement and Reporting and the Hospital Steering Committee measurement framework recommendations must be completed and approved by the membership first before any specific measurement set is put before the membership for comment or vote.
Concern: Quality Measurement is Primarily a Tool to Improve Care. We are very concerned that Recommendation 1 states that the hospital performance measurement set is to "facilitate public accountability," with "quality improvement" the potential by-product. AHA rejects this model. Quality improvement should be the primary goal. One means of facilitating quality improvement may be through public sharing of meaningful data. The act of measuring and collecting quality data must have as its core goal to provide information that will be useful in improving care.
Recommendation: Reword Recommendation 1 to read: "The primary purpose of this hospital measure set is to facilitate quality improvement. Public disclosure of hospitals' performance on this measurement set should be in the context of informing health care consumers and the public about the quality of hospital care.
Concern: Measurement Efforts Must be Coordinated and Must Be Sensitive to the Resource Concerns. AHA is concerned that the NQF report does not adequately comment on issues surrounding the burden and inefficient use of resources associated with the current data collection efforts. We are unaware of any pilot or assessment aimed at determining how much resource burden would be incurred if a hospital were required to provide data on all 32 of the measures included in the initial set.
Further, consideration must also be given to the burden of data collection associated with proposed risk adjustment models. The document also fails to emphasize that it is not necessary that the 32 core measures be used in their entirety, i.e. as a "set." Measurement sub-sets may well meet the needs of specific stakeholders.
Recommendation: NQF should recommend that there be standardized timing for the collection of data and time periods for which the data covers, standardized formats, etc. to avoid multiple data requests from purchasers, payers and others that currently occur at differing times and are applied to different patient populations.
Concern: Risk Adjustment. We are pleased that the report recognizes the need to adequately risk-adjust clinical measures. Without such adjustment, the public will be mislead and high quality providers could be unfairly portrayed. However, the report needs to include further analysis and discussion of the data collection burden associated with the proposed risk adjustment models. Further, risk adjustment is a complex topic that requires further work before explicit models are included in the recommendations.
Recommendation: The NQF report needs to acknowledge the burden and consistency concerns providers will have who operate across states where specific risk adjustment models are in effect, and where these model(s) may differ from NQF's recommendations. The report should include an analysis of the proposed risk adjustment models and the steering committee rationale behind the recommended models.
Concern: Process for Updating Measures. We understand that the NQF has engaged the Hospital Measure Steering Committee in deliberations around how to build a framework for updating measures. We support this work. As we learn more about what works to improve care and why, we need to diffuse this information into our data collection efforts. At the same time, there needs to be a reasonable timeline and threshold for any measurement changes. Data abstraction tools, training, and customized software are vital for measurement collection, and costly and complicated to update or change.
Recommendation: We look forward to working on these issues through the NQF process, but strongly suggest that reference to the importance of these longer- term issues should be included in any final report accompanying the initial set of measures.
Specific Measurement Comments
Concern: Surgical Complication Measures fail the following inclusion criteria
developed by the Hospital Measures Steering Committee: 1) useful for quality improvement; 2) usable for decision making; 3) precise; and 4) adequately tested.
For example, all three measures are composite measures (e.g. the timing of antibiotic administration represents, at a minimum, the composite measurements of eight different surgical categories). As composite measures, they fail to provide useful information to facilitate quality improvement. Interested parties would not be able to discern whether antibiotic prophylaxis was adequately provided for in a specific surgical category.
Recommendation: We suggest that a more effective and better-structured approach would be to "tie" specific measures to other measures in the data set. For example, assessing the timing, selection, and duration of antibiotic prophylaxis for cardiac bypass surgery would be an effective method of 1) assessing the adequacy of the system for providing antibiotics; and 2) would provide additional useful information to consumers making choices regarding cardiac surgery.
Concern: Wording under timing of antibiotic administration. The wording implies that two indicators have been merged into one, i.e. percent of patients receiving antibiotics within certain time frames and percent of patients not receiving antibiotics.
Recommendation: A better measure would separate the two indicators, especially since the desirable trend lines would be for an increase in the former and a decrease in the latter. By combining into a single concept, public perception of what the results mean may be confused unless very clear public education is included with the reporting to explain what the desirable trends should be.
Concern: The surgical complication measures are new measures in the Quality Improvement Organizations' 7th Scope of Work and have not been tested. As such, these measures fail the "adequately tested" criteria. To be clear, we agree that surgical complications is an important area to examine, and we stand ready to work with the NQF and others to implement valid, meaningful measures in this area.
Recommendation: Suspend the surgical complication measurement set until it has been adequately tested and we have experience with whether the information yielded will accurately inform providers and the public about surgical quality in a hospital.