Commonly available large language models (LLMs) like ChatGPT, Copilot, Claude, Gemini, and Grok are not designed or regulated for healthcare purposes, yet many users turn to them for quick answers about medical conditions, treatments, or concerns like how to use a medical device or what supplies to purchase.

According to ECRI, these uses can have critical implications for patient safety, as LLMs are prone to "hallucinations" — incorrect or misleading responses — or distortions from biases embedded within their dataset.

These limitations, along with LLMs' predisposition to agree with users, can cause a model to provide incorrect medical advice.

Users should be aware of the limitations of LLMs and scrutinize their responses, ECRI said. LLM output should never be used as a replacement for qualified medical advice.

A "digital darkness" event is a sudden loss of access to electronic systems and patient information, and these events can compromise care delivery, delay treatment, and jeopardize patient safety, according to ECRI.

Cyberattacks, natural disasters, vendor outages, and internal system failures can all cause digital darkness events and potentially paralyze a healthcare facility.

To better prepare, ECRI advises healthcare organizations to strengthen their disaster recovery planning, build robust recovery capabilities, and ensure organizational readiness through training, tabletop exercises, and safety drills.

Product quality has become a major concern in healthcare, as substandard and falsified medical products are reaching the United States with an alarming frequency, according to ECRI.

This can have major implications, as medical devices or supplies could not function as intended, or medications could not meet the proper quality standards and cause harm.

ECRI recommended that healthcare providers strengthen their supply chains, leverage their purchasing power to demand high-quality products, and implement measures to protect patients and staff from the use of flawed products.

Improvements in home diabetes management technologies, like the integration of insulin pumps with continuous glucose monitors (CGM), have significantly improved the quality of life for millions of patients, but the use of these technologies comes with risk, ECRI said.

If product recalls and updates don't reach patients and caregivers in a timely manner, patients can get hurt. For example, an integrated insulin pump could deliver too much insulin, or sensors on a CGM could yield incorrect glucose readings.

As a result, home users need to be proactive in identifying and responding to safety notices related to their devices and apps, and product manufacturers, medical equipment suppliers, and healthcare providers should implement processes to provide users with product safety information in clear, easy-to-understand ways, ECRI said.

The incorrect connection of a syringe, tubing, or other device to a patient line intended for a different use can lead to medications, solutions, nutrition, or gas being incorrectly introduced into a line, which can lead to severe consequences, ECRI said.

This hazard is a result of the Luer-lock design for making connections in a number of healthcare applications, according to ECRI. Safer connector designs exist, including ENFit connectors for enteral/oral applications and NRFit connectors for neuraxial applications; however, adoption has been slow.

ECRI recommends healthcare organizations develop and institute plans to make the transition to ENFit and NRFit products.

Not implementing medication safety technologies in perioperative settings puts patients at risk of serious medication errors. These errors can occur at several points in the perioperative workflow, including the ordering, selection, dosing, preparation, and administration of medications.

Medications administered to surgical patients are typically "high alert" medications, ECRI said, and have a heightened risk of causing significant harm if used incorrectly.

ECRI recommends that healthcare organizations work toward incorporating tools that can reduce the risk of medication errors.

The failure to properly clean and disinfect or sterilize reusable medical devices between uses can lead to infections spreading, device damage, and other forms of harm, according to ECRI. But doing this successfully can be challenging given the wide variation in the content, quality, and feasibility of reprocessing instructions provided by product manufacturers.

Without adequate instructions, healthcare workers can find it difficult to complete tasks effectively, potentially causing harm to patients through the spread of infections, injury to healthcare workers from repeatedly performing reprocessing procedures, and costs to organizations from financial penalties and reputational damage.

Reprocessing considerations should be considered during the prepurchase risk assessment of a product, and manufacturers need to provide reprocessing instructions that are complete and have been validated as practical and effective.

Cyber threats are constantly a concern for healthcare organizations, and legacy medical devices are particularly a concern when it comes to guarding against cyberattacks as they provide an opening that hackers can exploit, according to ECRI.

"Legacy devices" include software-based devices and systems that aren't being updated with sufficient cybersecurity protections anymore, as well as those that aren't practical to secure within the environment or with available controls.

Nearly every healthcare organization has legacy devices and software systems, and as a result, mitigating actions are needed, ECRI said. This could include segmenting devices on — or disconnecting them from — the network, deploying security tools to manage vulnerabilities, or planning for device replacement.

Implementing healthcare technologies without understanding the workflow of frontline users can cause device usability problems that can lead to patient harm, ECRI said.

Device usability issues can arise when manufacturers, IT personnel, or healthcare technology management staff implement devices without sufficient knowledge of current clinical practice, the environment of use, and the needs of the patient population.

To avoid safety risks, organizations must establish collaborative, clinically informed technology selection, configuration, and implementation processes, including establishing multidisciplinary healthcare technology teams, conducting comprehensive workflow analysis before technology deployment, and instituting training programs that are sufficiently comprehensive and extended to all clinical staff, ECRI said.

The failure to maintain sufficient water quality during instrument sterilization risks exposing patients to potentially infectious pathogens or causing delays in treatment that prevent timely access to care, ECRI said.

Healthcare facilities that experience water quality issues could be forced to delay surgeries or other medical procedures until such concerns can be resolved.

To prevent these issues, ECRI recommends healthcare organizations work toward standards developed by the American National Standards Institute and the Association for the Advancement of Medical Instrumentation. ECRI also recommends routinely assessing the cleanliness of processed devices and instruments, periodically sampling water quality, and developing a contingency plan for maintaining appropriate patient care if a water quality event should arise.