Keep an Eye on Clinical Validation Gaps in AI-Enabled Medical Devices

Health care leaders may want to keep a closer eye on clinical validation for artificial intelligence (AI)-enabled medical devices (AIMDs), based on a recent JAMA Health Forum study.

The study examined 950 AI medical devices authorized by the Food and Drug Administration through November 2024. Sixty of the devices were associated with 182 recall events.

The most common causes of recalls were diagnostic or measurement errors, followed by functionality delay or loss. About 43% of all recalls also took place within one year of FDA authorization.

Because 510(k) clearance does not require prospective human testing, many AIMDs enter the market with limited or no clinical evaluation; meanwhile, recalls may undermine clinician and patient confidence in their performance, according to the study.

In this cross-sectional study, recalls of FDA-cleared AIMDs were uncommon but were concentrated early after clearance and predominantly involved products lacking clinical validation and manufactured by publicly traded companies, suggesting that the 510(k) process may overlook early performance failures of AI technologies.

In fact, publicly traded companies accounted for about 53% of the recalls on the market and were associated with more than 90% of recall events in the study and 98.7% of recalled units.

Tinglong Dai, lead author of the study and a professor at the Johns Hopkins Carey Business School, told Healthcare Dive that the “vast majority” of recalled devices had not undergone clinical trials. For the majority of AI-enabled devices, which went through the FDA’s 510(k) pathway, clinical studies are not required.

The study’s authors note that heightened premarket clinical testing requirements and postmarket surveillance measures may improve identification and reduction of device errors, similar to risk-based strategies in pharmacovigilance. “The association between public company status and higher recalls may reflect investor-driven pressure for faster launches, warranting further study,” the report states.

Study limitations include reliance on publicly available validation reports, exclusion of software updates not labeled as recalls, and limited follow-up for devices cleared after 2022. Nevertheless, linking premarket evidence gaps and manufacturer type to postmarket actions offers practical guidance for regulators, clinicians and health systems adopting AI-based tools, the study states.