Commercial risk will be a critical catalyst of progress – it’s complicated, but is it possible? We think so.

Blog Post

Augmented intelligence—virtual assistants come to health care

By Andrew Rebhan

July 10, 2018

    Natural language processing (NLP) techniques allow digital systems to streamline user interactions, allowing machines to read text, understand meaning, and generate narratives from existing information. Recent advances in artificial intelligence (AI) technologies have accelerated progress in a broad range of NLP applications for health care, including digital assistants for clinical staff, concierge services for patients, and digital scribes to streamline documentation processes.

    How natural language processing will help achieve a payback on EHRs

    Growth of virtual assistants in health care

    For example, last fall, Nuance Communications released its Dragon Medical Virtual Assistant to help health care providers interact with clinical workflows using NLP and other conversational AI functionality. Nuance announced at HIMSS18 that it will integrate its virtual assistant technology into Epic's EHR.

    According to the press release, the new partnership allows physicians to use the virtual assistant to ask for information from a patient's chart, retrieve labs, medication lists, and visit summaries using Epic Haiku. Nurses using Epic Rover can use the assistant to conversationally interact with flowsheets to enter and confirm patient info and vitals. Finally, scheduling staff using Epic Cadence can converse with the assistant to check physician schedules and create or modify patient appointments. And Vanderbilt University Medical Center recently announced it is leveraging Nuance's technology to build a prototype voice assistant called "V-EVA" (Vanderbilt EHR Voice Assistant) to help caregivers navigate the hospital's Epic EHR using natural dialogue.

    A number of other health care providers have started piloting voice assistants. For instance, Northwell Health is testing Amazon's Alexa across multiple use cases, including one that helps users determine the wait time at nearby EDs and urgent care centers in Northwell's system. People can ask their Alexa-enabled home devices to either search for the shortest wait time based on their zip code, or can ask for the wait time for a specific location. Once the user asks for this information, Alexa queries Northwell's database of wait times (which analyzes check-in data every 15 minutes) for the best option. The Alexa feature can respond back with the location's name, address, and wait time.

    In another example, the University of Pittsburgh Medical Center (UPMC) is collaborating with Microsoft to create an intelligent scribe called EmpowerMD. The project is part of Microsoft's Healthcare NExT initiative, which aims to use AI to accelerate health care innovation. The virtual scribe listens to conversations doctors have with patients, analyzing speech for clinically relevant concepts to make suggestions in the medical record. The goal is to allow doctors or other staff to engage with patients face to face, without the need to divert their attention to a computer screen. The scribe can make suggestions or take notes for follow up, which the doctor accepts or modifies after the encounter. Staff can also view a transcript of the conversation for greater context on the assistant's suggestions. Using machine learning, the virtual assistant improves its performance as suggestions are accepted, rejected, or modified by the user.

    Patients increasingly interested in virtual assistants

    Patients are also interested in AI-powered virtual assistants. Accenture recently released the findings of its 2018 Consumer Survey on Digital Health, which polled 2,301 U.S. consumers on topics such as wearables, virtual care, and AI. Among the findings, the survey showed that roughly one-in-five consumers has experienced health-related AI, and, in particular, showed an openness to using intelligent virtual assistants. Specifically, the survey found that:

    • 61% of respondents said they would use "an intelligent virtual health assistant that helps to estimate out-of-pocket costs, schedule health care appointments and explain benefit coverage, bills and payment options";
    • 57% would use an intelligent virtual coach;
    • 55% would use "an intelligent virtual nurse that monitors your health condition, medications, and vital signs at home"; and
    • 50% would use "an intelligent virtual clinician that helps to diagnose health issues and navigate you to the right treatment options."

    3 tips to incorporate virtual assistants

    Is your team interested? Here are three key considerations to get you started:

    1. Identify your goals: Virtual assistants can perform a variety of tasks described above. In addition, they can set reminders, answer basic patient questions, call for a nurse, or even address loneliness. However, virtual assistants may not always be the best solution for a given problem, particularly complex tasks that may benefit from visual displays (such as on a computer or tablet). Make sure your team is specific about how the technology will improve processes and where it fits into existing workflows.


    2. Explore what's possible: The major technology companies such as Google and Amazon are trying to make their software development kits and APIs as open and user-friendly as possible—which means your organization can build new skills into these virtual assistants to better suit your needs, assuming you have the right staff skills to code these features. As you evaluate options, ensure that potential solutions are properly evaluated for HIPAA compliance, as natural language interfaces in a health care setting may capture sensitive information whether or not that is not part of their intended use.


    3. Expect change: The health care industry is starting to see rapid advancements in NLP, computer vision, and other subsets of AI—but the use of virtual assistants in hospitals is still nascent. The technology will likely continue to evolve as more organizations adopt and test these devices, and the broader industry forms new ways to implement and regulate their use. Early adopters will have an advantage in getting to use and gain experience with these tools, but they may also have to update them more often as vendors release new editions with enhanced capabilities.

    Sept. 6 webconference: Hear the latest in analytics and the rise of AI

    Join the webconference to learn how new artificial intelligence (AI) technologies will play a role in improving clinical and financial outcomes.

    Register Now

    Have a Question?


    Ask our experts a question on any topic in health care by visiting our member portal, AskAdvisory.