3 Takeaways from Cleveland Clinic’s AI Scribe Pilot Process and System Selection

Developing a well-rounded approach to assessing artificial intelligence (AI) medical scribes can be a daunting process, with an ever-expanding array of systems available and significant capital investment ramifications.

So, when Cleveland Clinic leaders recently went through their evaluation process, they opted for a customized approach. They test drove five companies’ systems before they automated their documentation with AI.

The organization believes it was the first head-to-head test of the major AI medical scribe programs for health care, involving extensive piloting of the systems. The health system conducted a rigorous pilot program of AI documentation solutions throughout 2024, evaluating the technology across more than 80 specialties and subspecialties.

The five AI scribe products were evaluated for documentation quality, product features, provider satisfaction, ease of implementation and return on investment. Cleveland Clinic analyzed data from Epic, provider survey results, patient feedback and technical evaluations, Beth Meese, executive director of digital health at Cleveland Clinic, said in a recent interview. She added that the organization’s experience with all five vendors was positive.

On Feb. 19, the organization announced that it would roll out Ambience Healthcare’s AI platform for documentation, clinical documentation integrity and point-of-care coding. The technology aims to provide more time for personal interaction during visits and reduce the administrative workload for caregivers.

3 Takeaways from the Analysis

1. This is a health care play more than a tech play.

   Cleveland Clinic involved 25 to 35 clinicians for each of the five systems evaluated in the pilots that lasted three to five months. The organization emphasized evaluating companies on numerous factors, including culture and fit across different specialty areas. “You want the company to have passion for health care. This is not a technology play when all is said and done; this is a health care play,” Rohit Chandra, executive vice president and chief digital officer, told Becker’s Hospital Review.

2. Provider engagement is critical.

   Providers must read the note in its entirety to confirm its accuracy and completeness and edit as needed before signing. Providers in ambulatory settings will have the ability to try the software but will not be required to use this tool. Patients will be notified before the AI tool is used and can opt out.

3. Anticipate that AI scribes will continue to improve.

   Today’s products are “amazing” and are only [going] to get better Chandra says. "Over time, we should be able to substantially reduce the documentation burden, enable providers to operate at the top of [their] license, improve their experience and satisfaction, improve the quality of the physician-patient interaction — and that is starting to happen today," he adds.