Major Methodological Flaws Negate Findings from Recent 340B Study
The 340B Drug Savings Program provides eligible providers, many of which are hospitals, with resources to maintain vital services in their communities and to provide additional services to their patients. A recent study published in the New England Journal of Medicine (NEJM) alleges that the 340B program does not expand access to care to low-income populations or improve their mortality rates, while driving hospital/physician consolidation.1
The study as designed and executed by its authors fails to draw meaningful, valid conclusions about the program due to constraints and flaws in the methodology used. Of particular concern are broad statements that apply the inaccurate findings of the study to all of those in the program– even though the study excluded a majority of 340B hospitals. Below, we outline some of our concerns. For more detail, see a methodological critique by an independent economist here. In subsequent posts, we will address the study findings specifically, including how the 340B program helps hospitals serve their communities and the truth about health care consolidation.
The study uses a methodological approach called “regression-discontinuity” that does not support making broad claims about the 340B program. The authors examined a sliver of 340B hospitals – just 20 percent – yet made expansive statements about the implications of the program. Regression-discontinuity analyses may have strong internal validity but weak external validity – that is, the results may not be extrapolated to program participants not included in the analysis. (And this is only if the method is applied correctly, which, in this case, it appears it was not. See the methodological critique linked above for more information.)
Ignoring this limitation, the authors extrapolate the results to, and draw broad conclusions about, the 340B program as a whole. The first example of this broad application of findings is in the title of the study itself: “Consequences of the 340B Drug Pricing Program.”
The study relies on fee-for-service Medicare data only to make claims about the impact of the 340B program on low-income individuals, ignoring that the vast majority of low-income people are not enrolled in Medicare. Low-income individuals are diverse: the largest subgroup are children, who make up approximately 35 percent of low-income individuals, followed by the working poor.2 Only 23 percent of low-income individuals are elderly or disabled and therefore potentially eligible for Medicare. However, the study authors chose to look at fee-for-service Medicare claims only to evaluate the impact of the 340B program on low-income individuals more broadly.
The total reliance on fee-for-service Medicare claims is an odd choice for another reason – that the study authors claimed to be interested in looking at improvements in access to care. Medicare beneficiaries receive a relatively comprehensive benefit package compared to other insured individuals and are the group most likely to have a medical home. The authors could instead have evaluated the impact on vulnerable populations such as the uninsured, the underinsured and Medicaid beneficiaries.
The study authors applied their own limited interpretation of the intent of the program when selecting measures to evaluate 340B hospitals. According to the authors, “…the program may not elicit the intended responses from hospitals – such as providing more care to low-income communities, investing in safety-net providers, or reducing health disparities….” While these are all laudable goals, they reflect the study authors’ own beliefs of how the 340B program should work, not Congress’s. Congress intended for the program to “stretch scarce Federal resources as far as possible, reaching more eligible patients and providing more comprehensive services.”3 While many hospitals may use 340B savings to invest in safety-net providers or fund programs intended to reduce health disparities, there are many ways in which 340B entities may meet the true Congressional intent of the program of expanding access to care and more comprehensive services. The study authors cannot make broad claims about program performance when looking at such limited performance measures.
The study also fails to account for changes in coding of physician practices during the study period. The authors note that “…increases in hospital ownership of physician practices could have been over stated if practices owned by hospitals merely changed place-of-service codes….” However, they choose to ignore a change in federal reporting requirements that did just this. Beginning in 2011, the Health Resources and Services Administration (HRSA) required that all outpatient and other community-based sites of care that intended to use 340B drugs for their patients register separately for the 340B program, along with other requirements. By ignoring this HRSA reporting change, the study authors fail to acknowledge that the increase in the registration of hospital-owned outpatient clinics and services in the 340B program may simply be a matter of changes in reporting.
These mistakes – and many others detailed here – lead to erroneous conclusions. To learn more about the real 340B story, click here.
1 Desai S, McWilliams JM. Consequences of the 340B Drug Pricing Program. N Engl J Med 2018; DOI: 10.1056/NEJMsa1706475.
2 The Brookings Institution, “Who is Poor in the United States,” June 2016. Accessed at: https://www.brookings.edu/research/who-is-poor-in-the-united-states/
3 H.R. REP. No. 102–384(II), at 12 (1992).