Liberty Place, Suite 700
325 Seventh Street, NW
Washington, DC 20004-2802
(202) 638-1100 Phone
Monday, July 15th 2002
Kenneth Kizer, MD, MPH
President and CEO
National Quality Forum
Attn: Nursing Home Performance Measures Project
601 Thirteenth Street, NW
Suite 500 North
Washington, DC 20005
Dear Dr. Kizer:
On behalf of the American Hospital Association’s (AHA) 5,000 hospital, health system, network and other health care provider members, we are pleased to submit the attached ballot for the Nursing Home Performance Measures. Our membership includes about 2,000 hospital-based skilled nursing facilities (SNFs).
America’s hospitals support the overarching goals of the nursing home performance measures project. We believe that meaningful, accurate performance data can be used to improve the delivery of care and assist the public in making more informed decisions about their post-acute and long-term care choices. Unfortunately, as we noted in our pre-voting comments in May, the measures and associated risk adjustment approaches we are asked to vote on today have not yet been sufficiently validated, especially for the purpose of public reporting. As we noted in our pre-voting comments, quality measures to be shared with the public require even higher standards of reliability and validity than those used for other purposes. The lack of availability of validation study results forces us to cast no votes on most of the recommendations.
Absent validation of the proposed measures and associated risk adjustment approaches, the publicly reported data has the potential to create unintended consequences, including:
Reduction in access to care for medically complex patients as a result of facilities being implicitly discouraged to admit patients with certain conditions because the facility is being compared to inadequately adjusted norms;
Demoralization among staff of high quality providers because of erroneous implications from measures that turn out to be less reflective of quality care than originally thought;
Confusion in the general public as a result of measures being changed soon after their initial introduction; and
Modification of care delivery practices as a result of providers reacting to potentially erroneously specified measures, instead of focusing on commonly agreed-upon and validated measures of quality of care.
All of these outcomes have the potential to reduce quality of care, rather than enhance it.
AHA’s view of this ballot would likely be different if two important things had preceded this vote:
results were available from the CMS-funded “Mega QI” field test and validation project which are, in fact, expected shortly and are designed to answer questions about the validity of the proposed measures; and
results were available from an independent evaluation of the six-state pilot project on nursing home measures commenced in April of this year.
It is particularly frustrating that the National Quality Forum (NQF) chose not to postpone this vote until issuance of the “Mega QI” report, especially as many organizations, including the AHA, had suggested this approach several months ago. NQF’s decision to proceed with a member vote on quality measures before study results are available, has needlessly undermined member confidence in this project and in NQF’s decision-making process.
The AHA will continue to work with CMS, the Quality Improvement Organizations and NQF to promote quality improvement in nursing homes and to address our concerns. If you have any questions about this ballot, please call Brian Ellsworth, Senior Associate Director, Policy, at (202) 626-2320.
Executive Vice President
Rationale for American Hospital Association Votes on the
National Quality Forum (NQF) Ballot for the
Nursing Home Performance Measures
July 15, 2002
With respect to the nine proposed chronic care and three proposed post-acute care measures derived from the Minimum Data Set (MDS), there are two fundamental rationales for AHA’s “no” vote.
The proposed measures do not meet the threshold criteria established by the NQF Steering Committee for measure selection – there has been no formal, independent evaluation of the validity of these measures. Such an evaluation is essential to knowing whether the selected measures capture what they purport to measure and that the measures make meaningful distinctions between good and poor quality care.
The proposed measures lack appropriate risk adjustment. Without appropriate risk adjustment, a facility may exceed the published norms on a measure due simply to attracting a disproportionate share of patients with that condition and/or because the facility more accurately completes the patient assessment forms. Ironically, either of those reasons might actually be indicative of a superior facility, rather than a substandard one.
In addition to these cross-cutting concerns, there are specific concerns about most of the measures. For instance, with respect to pain management (both the chronic and post-acute proposed measure), the MDS does not support accurate calculation of the facility’s efforts at controlling pain levels.
Specific concerns about the post-acute measures
Of particular concern to hospital-based skilled nursing facilities (SNFs) are the three post-acute measures - pain management, delirium and improvement in walking. Many hospital-based SNFs provide care to medically complex patients for a short duration (often less than 14 days) after a hospitalization. Due to limitations in the data source - the MDS - information concerning post-acute measures for successful, short-term skilled nursing facility discharges are not recorded in the data.
This occurs because the MDS was not designed for outcome analysis in post-acute care. As a result, in many cases these short-stay, post-acute patients are discharged prior to the timeframe for next assessment required under the MDS. For example, patients who are successfully treated in a SNF and discharged home within 14 days of admission have no data recorded in the post-acute quality indicators. Only the longer staying post-acute patients are recorded in the MDS database, which can create a potentially misleading view of a skilled nursing facility’s actual quality of care. We believe this is a fatal flaw in the three proposed post-acute measures derived from MDS data.
Other post-acute and chronic measures
Proposed measure: pneumococcal polysaccharide vaccination
The proposed measure cannot be calculated from currently available data. The data source listed in the specifications includes an element that indicates only the number of residents in the facility at the time of the most recent survey that received the pneumococcal vaccination. It does not indicate the number of “residents who are screened for eligibility for pneumococcal vaccine status within 30 days of admission and are either not eligible or are eligible and receive the vaccination” (per proposed specification). The proposed denominator for measure cannot be calculated from the proposed data source. The data source listed indicates only the number of residents in the facility at the time of the most recent survey. It cannot be used to identify “all newly admitted residents age 65 and older” (per proposed specification).
Proposed measure: influenza vaccination
The proposed measure cannot be calculated from currently available data. The data source listed in the specifications includes an element that indicates only the number of residents in the facility at the time of the most recent survey that received the influenza vaccination. It does not indicate the number of “residents who receive vaccination against influenza or are not eligible for vaccination” (per proposed specification). The proposed denominator for measure cannot be calculated from the proposed data source. The data source listed in the specifications indicates only the number of residents in the facility at the time of the most recent survey. It cannot be used to identify, “nursing home residents who have resided in the facility for any length of stay from October 1 through March 31 of the year prior to the measurement…” (per proposed specifications).
Proposed measure: nurse staffing
In the absence of sufficient research about the relationship between staffing ratios and variations in facility case-mix/acuity levels, comparative information for facilities with dissimilar populations can be highly misleading to consumers. An approach to adequate adjustment of staffing thresholds for differences in case-mix/acuity has yet to be developed. Per CMS, “Nurse staffing data currently does not exist that is sufficiently accurate for consumer information and for determining compliance with any staffing requirement that might be implemented. (see Appropriateness of Minimum Nurse Staffing Ratios in Nursing Homes, Phase II Final Report, December 2001, p. ES-2)
Recommendation: risk adjustment
AHA supports this recommendation. Accurate and fair risk adjustment is absolutely essential for any quality improvement process that involves external reporting of data. The measures themselves and the associated risk adjustment methods are inextricably intertwined.
Recommendation: public reporting
AHA supports this recommendation only if the sentence that starts on line 8 of page 10 is reworded as follows: “A standardized national reporting format for public presentation of nursing home quality information should be formally assessed for understanding and usefulness with intended audiences, using the best available scientific methods possible.” This wording is intended to ensure that entities beyond just CMS are in a leadership role with respect to developing a standardized public reporting format and that all such entities utilize the best available scientific methods in assessing understanding and usefulness to intended audiences.
Recommendation: updating and improving core sets
AHA supports this recommendation; however, we believe that each of the three sub-recommendations (formal evaluation, examination of unintended consequences, re-evaluation of core sets) contained in this paragraph is so critical to the long-run success of the project that each merit’s separate discussion as NQF discussions proceed.