Auto logout in seconds.
Continue LogoutAI is moving fast in healthcare — and health systems are racing to keep up. With more than 80% of physicians already using AI tools, leaders say the next challenge is avoiding pilot overload and focusing on the technologies that truly improve care.
The use of AI in healthcare has grown rapidly, becoming a key part of operations for many organizations.
According to a recent survey from the American Medical Association (AMA), 81% of physicians now report using AI tools at work — more than double the number of doctors who said the same in 2023. Over 75% of physicians also said they believe AI improves their ability to care for patients, up from 65% in 2023.
Some common ways physicians are using AI include summarizing medical research and standards of care (39%), creating discharge instructions, care plans, or progress notes (30%), and documentation of billing codes, medical charts, or visit notes (28%).
"AI has quickly become part of everyday medical practice" said John Whyte, AMA's CEO and EVP. "Physicians see real promise in its ability to support clinical decisions and cut down on administrative burden. But as this technology advances, it is critical that augmented intelligence be designed to enhance — not replace — physicians."
Aside from physicians, health systems, payers, and pharmaceutical companies are also utilizing AI technology to drive operational efficiency.
For health systems, AI has largely been used for revenue cycle management and prior authorization, clinical diagnostics, and virtual nursing/smart patient rooms. Payers have focused their AI efforts on utilization management and prior authorization, risk adjustment and care management, and employer insurance benefits and enrollment. At pharmaceutical companies, AI is being used for drug discovery and target identification, clinical trial optimization and real-world evidence, and marketing to health systems, physicians, and consumers.
Over the last two to three years, AI technology has advanced significantly — to the point where it is outperforming doctors on some tasks.
For example, a 2024 study published in JAMA Network Open found that OpenAI's AI chatbot ChatGPT performed significantly better than doctors who used traditional resources to make a diagnosis, as well as doctors who used both traditional resources and ChatGPT. Doctors who used ChatGPT had a median score of 76% for making a diagnosis and explaining a reason for it while doctors who only used traditional resources had a median score of 74%. In comparison, ChatGPT alone received a median score of 90%.
However, there are still limitations to AI's capabilities, with research showing that the technology can be prone to errors.
"Having a plan to unwind a pilot forces those involved to contemplate a more successful endpoint, which can drive better implementation practices."
In a study published in Nature Medicine earlier this year, researchers found that ChatGPT Health, OpenAI's healthcare-focused chatbot, under-triaged 51.6% of emergency cases compared to physicians, recommending the patient see a doctor within 24 to 48 hours rather than recommending they go to the ED. The chatbot also over-triaged 64.8% of nonurgent cases, recommending the patient see a doctor when it wasn't necessary.
In a separate study published in NEJM AI, researchers found that when physicians were presented with deliberately erroneous information from an AI chatbot, their diagnostic accuracy dropped significantly. The physicians exposed to erroneous chatbot information had a mean diagnostic accuracy of 73.3% compared to 84.9% for the control group.
"Physicians demonstrate substantial automation bias when exposed to erroneous LLM recommendations, even with voluntary consultation and prior AI literacy training," the researchers wrote. "These findings highlight safety risks that require robust validation frameworks and regulatory safeguards before widespread clinical AI deployment."
Although concerns about AI's safety and effectiveness continue, Carol Chouinard, VP and provider technology lead at Optum Advisory*, said that adoption of the technology will likely continue to accelerate, especially as AI models quickly evolve and improve their performance.
As hospitals and health systems incorporate more AI tools into their operations, healthcare leaders will need to implement new strategies to avoid AI pilot proliferation and move past the hype to find tools that actually work for their organization.
1. Take a streamlined approach
According to Colin Walsh, an associate professor of biomedical informatics, medicine, and psychiatry at a major academic medical center in Tennessee, running multiple AI pilots simultaneously can lead to them drowning each other out.
"Let's say, there's two concurrent pilots happening in the same setting, it actually becomes much more difficult to understand which one is working and why," Walsh said. "One danger is a successful pilot may be seen as not being successful because there's too many other things going on, and [the] converse would also be true."
Another potential issue is clinician engagement. If a pilot doesn't address a clear challenge, clinicians may be less likely to engage in it.
"Like any technology, be clear about the intended benefit of the AI tool, describe what good looks like, and name the workflow or task the new tool replaces," said Ben Isenhour, a senior director of provider technology services at Optum Advisory. "That clarity drives change management before launch and grounds the efficiency case. It is also why the best AI tools, like ambient listening and chart summarization, have moved beyond efficiency and are now recruiting hooks."
2. Focus on user experience
According to Isenhour, organizations should be intentional and transparent about the AI tools they use "to establish psychological safety and trust in new AI-enabled workflows" with both clinicians and patients.
In 2022, WellSpan Health partnered with AI-enabled imaging company Aidoc to pilot several different clinical use cases. Aidoc's model analyzes images from CT scans, MRIs, and X-rays to detect diseases, and WellSpan used it to identify strokes, pulmonary embolisms, and cervical spine fractures.
Since then, WellSpan has expanded Aidoc's model to more than 250 locations. In 2025, WellSpan's radiologists used Aidoc to analyze 200,000 scans, which is up to 81% faster than their previous workflows. As a result, 5,500 patients received earlier diagnoses in this period.
"The big learning there was that when you introduce AI in the clinical space, it's not just that it works, but that it's easy to use," said Mark Kandrysawtz, SVP and chief innovation officer at WellSpan.
As AI technology continues to evolve, it will transform clinical workflows and impact the roles clinicians will play in the future, Chouinard said. In the future, AI tools will include less repetitive interventions, be able to utilize more data to handle more complex situations, and be geared more toward care management and population health.
3. Be prepared to move on
At Tampa General Hospital, leaders have a set timetable for every pilot, which means that when a pilot begins to slow or drag on, they end the pilot and move on.
"Let's say, we have a new technology that should take 90 days to implement," said Scott Arnold, the hospital's chief digital and innovation officer. "If we're at day 90 and nothing's happened, and it's going to take another 90 days, which turns into 180 days. You're burning daylight, you're wasting a lot of people's time, and nothing's even started yet."
"What I've learned over my career is that [pilots] can become a slow no, especially for the ones that just go on and on forever," Arnold said.
"Having a plan to unwind a pilot forces those involved to contemplate a more successful endpoint, which can drive better implementation practices," Isenhour said.
Chouinard also noted that while current AI models have their limitations, they will continue to rapidly advance and improve. Over time, there will be fewer "move on" scenarios and more "let's see how fast it evolves" scenarios, he said.
*Advisory Board is a subsidiary of Optum. All Advisory Board research, expert perspectives, and recommendations remain independent.
(O'Reilly, American Medical Association, 3/12; Mazer, The Atlantic, 6/17; Qazi, et al., NEJM AI, 4/23; Famakinwa, Modern Healthcare, 6/16; Circle Square AI in Healthcare 2026: Top Trends by Stakeholder report, accessed 6/25)
Create your free account to access 1 resource, including the latest research and webinars.
You have 1 free members-only resource remaining this month.
1 free members-only resources remaining
1 free members-only resources remaining
You've reached your limit of free insights
Never miss out on the latest innovative health care content tailored to you.
You've reached your limit of free insights
Never miss out on the latest innovative health care content tailored to you.
This content is available through your Curated Research partnership with Advisory Board. Click on ‘view this resource’ to read the full piece
Email ask@advisory.com to learn more
Never miss out on the latest innovative health care content tailored to you.
This is for members only. Learn more.
Never miss out on the latest innovative health care content tailored to you.