IT Forefront

How to evaluate mobile health apps for high-risk patients


Today's health care consumer in search of mobile health applications has numerous choices, ranging from fitness trackers to disease-specific apps that monitor and track patient data.

Unfortunately, there are few mechanisms for evaluating these apps to determine whether they are clinically useful, user-friendly, safe, and secure.

A recent study in Health Affairs used a series of criteria to examine a subset of these apps intended specifically for use by patients with high-risk conditions. Such apps could potentially contribute in a meaningful way to improving the care of individual patients and high-risk patient populations. The study sheds light on important aspects of the value and hazards of such mobile health applications.

The researchers examined nine criteria, including clinical utility as assessed by clinicians; usability; app store ratings; response to entry of data suggesting a dangerous condition; and privacy and security of data sharing. The results confirm—and, if anything, elevate—the concerns of clinicians who are skeptical of the clinical value of mobile applications, and should concern consumers.

Here are some of the findings:

  • Many apps had clinical utility shortcomings. While many apps had the ability to track patient-input data, most were limited in their capacity to respond to patient-specific data input—for example, to offer a tailored recommendation in response to a particular patient-entered symptom or data point. This is typical of many kinds of clinical decision support, which often operate using relatively small subsets of patient-specific data, such as medication lists and allergies. This significantly limits their clinical utility.

  • Few apps flagged danger. A second, related, and more worrisome finding was the absence in over 75% of applications of a mechanism for responding to potentially dangerous user-entered data, such as suicidal ideation or extreme ranges of vital signs. The authors suggest that apps could, for example, automatically call the patient's emergency contacts or return a message that the user should consider calling 911.

  • High ratings didn't mean good apps. The researchers also found that app store ratings were poor indicators not only of clinical utility but also of usability. This finding likely reflects the general lack of sensitivity and specificity of the app store star rating methodology.

  • Lots of apps had security issues. Regarding privacy and security, the study showed that many apps with data sharing functions (such as connections with social networks, messaging, and the like) utilized unsecured methods for data sharing and messaging.

Overall, the study reminds us that in this realm as in others, the internet remains the Wild West. While the Office for Civil Rights (OCR,) FDA, and FTC regulate health apps, the scope of regulation by each body is narrow; for instance, FDA's purview is restricted to apps that interface with medical devices.

The study authors argue that additional regulatory requirements are not the solution. This seems reasonable. The mobile health space contains numerous apps; there is constant turnover of available apps and functionality is evolving rapidly, making effective regulation extremely difficult. This contrasts, for example, with the EHR market, in which vendor products and software implementations tend to be stable over longer timespans, making regulation of specific functionality possible (as we see in MACRA's Advancing Care Information requirements, formerly known as Meaningful Use).

The authors recommend that medical professional societies take a more active role in application evaluation and recommendation. This raises the question of whether these societies would be willing to make the investments needed to become deeply involved in that business. It would be challenging to establish evaluation methods that are sufficiently rigorous yet manageable without considerable resources. Application of such methodologies to mobile health apps could also raise liability concerns. That said, professional societies are probably better suited to performing this function than governmental or other regulatory bodies.

Unfortunately, many app developers today are driven more by volumes of downloads, ad click-throughs, and user ratings than by rigorous attention to clinical utility and user privacy and security. As a result, it seems unlikely that the Wild West will be settled any time soon.

We would advise consumers to be extremely wary of depending too much on these apps unless recommended by professional organizations. If your clinicians are recommending any of these apps, be sure to vet them thoroughly—and if needed, urge them to not recommend the apps at all.

3 key questions for remote patient monitoring

The increase in remote patient monitoring is being driven by the many advances in mobility and sensor technology, by the need to more comprehensively monitor an aging and chronically ill population, and by a growing retail reality that rewards patient engagement.

For providers to succeed with RPM, they need to answer three key questions: Is it technically feasible? Is it clinically relevant? Is it cost effective?

To learn more, download our infographic.

Download Now