How to Safely Use AI for Diagnosis: 14 Recommendations from ECRI

Artificial intelligence (AI) can be a useful augmentative tool to support diagnosing patients but should not replace clinical judgment, according to ECRI. The nonprofit health care research organization ECRI and the Institute for Safe Medication Practices (ISMP) recently released their annual report on leading patient safety concerns based on input from senior health care executives, including leaders at integrated health systems, rural community health centers, children’s hospitals and national associations. The top issue according to ECRI is “Navigating the AI Diagnostic Dilemma.”

A blog on the ECRI site states that while “AI has immense potential to improve clinical workflows and expand access to expertise,” it can also create safety and governance issues.

The ECRI report asserts that “users should not treat it as a replacement for clinical expertise” but rather as a “tool designed to supplement and support clinical expertise.” The report also outlines actions providers can take to ensure responsible implementation.

The Rise of AI for Diagnostic Purposes

A growing number of health care providers rely on AI for various tasks. From 2023 to 2024, the portion of hospitals utilizing predictive AI integrated with their electronic health records (EHRs) increased from 66% to 71%, according to a U.S. Department of Health and Human Services Assistant Secretary for Technology Policy brief based on AHA data.

The ECRI report states that AI can enhance diagnostic accuracy “by automating data retrieval, decreasing cognitive load, reducing cognitive biases and providing clinicians with information to help guide their decisions.” The technology has already proven its usefulness in areas such as diagnostic radiology, where AI tools can enhance the accuracy and efficiency of reviews.

14 Recommendations for Safely Utilizing AI

Clinicians can safely leverage AI to support diagnosing if they’re properly trained and employ a balanced approach, in addition to maintaining awareness of the technology’s abilities and limitations, according to the report. ECRI recommends the following actions for providers seeking to responsibly adopt AI:

Workforce Safety and Well-Being

1. Evaluate AI-powered health tech with human factors-based assessments to estimate impact and address how they’ll integrate into clinical workflows.
2. Encourage staff members to voice concerns about AI and investigate and address those concerns accordingly.
3. Gauge staff satisfaction and user experience with AI-powered systems.

Culture, Leadership and Governance

4. Define AI utilization guidelines, policies and procedures — including clear roles and responsibilities for the technology’s adoption, governance, oversight, documentation and monitoring.
5. Require documentation of AI usage and how it impacts diagnostic processes.
6. Evaluate the value of AI tools in terms of outcomes and cost, and assess for risks to preventable harm.
7. Assess AI tool usability based on human factors engineering principles.
8. Ensure staff members receive thorough training on proper AI use.

Learning Systems

9. In training materials and policies, make it clear that AI is merely a tool and that clinicians should utilize their own judgment in addition to seeking second opinions on AI-generated diagnoses.
10. Train staff on proper procedures for identifying, documenting and reporting any errors or adverse events related to AI.
11. Track and identify any potential disparities among patient populations throughout all stages of AI implementation.
12. Address critical thinking skills in staff training programs via diagnostic thought process evaluations, regular skill assessments and information on cognitive biases.

Patient and Caregiver Engagement

13. Disclose AI utilization to patients, obtain informed consent to use it to support diagnosis or to upload information in AI models, and provide the opportunity to opt out.
14. Address any patient or caregiver concerns about AI usage and reassure them that it supports rather than supplants.

Ultimately, to properly include AI in the diagnostic process, the ECRI report advises that “clinicians must view it as a tool designed to supplement and support clinical expertise — not replace it.”