Fitness trackers are effective tools for measuring heart rate, but when it comes to calorie counting, these devices tend to be inaccurate, according to a study published in the Journal of Personalized Medicine.
For the study, Stanford University researchers tested how accurately seven commercially available fitness trackers estimated heart rate (HR) and calories burned, compared with the traditional tools doctors typically use to monitor those vitals. The seven devices included:
- Apple Watch;
- Basis Peak;
- Fitbit Surge;
- Microsoft Band;
- Mio Alpha 2;
- PulseOn; and
- Samsung Gear S2.
The researchers tested the products among a diverse group of 29 men and 31 women who wore the trackers while sitting, walking, running, and cycling. Euan Ashley, a professor of cardiovascular medicine at Stanford University and co-author of the study, said, the researchers looked at a "diversity of ages, male and female, and then also … at diversity of skin tone, and then size and weight to try and represent the population generally."
The researchers said the ideal error rate would be under 10 percent for an everyday user in a non-medical setting.
Overall, the researchers found the devices' heart rate monitors were within that error rate, ranging from a median error rate of 2 percent for the Apple Watch to 6.8 percent for the Samsung Gear S2.
However, when it came to measuring burned calories, the degrees of inaccuracy ranged significantly—from about 27 percent to 93 percent, according to the study. Among the devices, the Fitbit Surge had the lowest median error rate for measuring burned calories at 27.4 percent, and the PulseOn had the highest error rate at 92.6 percent.
Across the devices, researchers found the lowest relative error rates for measuring burned calories occurred when study participants engaged in walking and running tasks. The highest relative error rates came when participants were involved in sitting tasks, according to the study.
In addition, the researchers found that the error rate was higher among certain groups of people than in other groups. For example, error rates were greater among men with darker skin and higher body mass indexes (BMIs), than among white women with lower BMIs.
Ashley said the findings indicate fitness trackers can accurately measure heart rate, but are "way off the mark" when it comes to measuring burned calories, especially "for those whom it might matter the most" such as individuals with higher BMIs who might be trying to lose weight.
Ashley also said the errors could have significant consequences because people base "life decisions on the data provided by these devices." He said the major takeaway from the study is that people should focus on their diets and exercise instead of calorie counts from devices.
Tim Church, a preventive medicine professor at Louisiana State University's Pennington Biomedical Research Center who was not involved in the study, said the study was "very well-designed and well-done," and shows that fitness trackers are more "fiction than fact." He said, "People are checking these inaccurate counts and they think they've earned a muffin or earned some ice cream," but in reality "they're sabotaging their weight-loss program."
Mark Gorelick, chief science officer at the device company Mio Global, said he agreed with the findings and "that more accurate calorie estimation is important for the industry as a whole, since most individuals are monitoring calorie deficits for weight loss."
Anna Shcherbina, a study co-author and Stanford graduate student, in statement acknowledged that it is difficult to create "an algorithm that would be accurate across a wide variety of people because energy expenditure is variable based on someone's fitness level, height, and weight, etc."
However, the device makers behind Fitbit and PulseOn said despite the findings, they remain confident that their trackers accurately measure calories burned and heart rate. PulseOn added that the high margin of error could "suggest that the authors may not have properly set all the user parameters on the device."
According to NPR's "Shots," the other device makers did not immediately respond to requests for comment (Andrews, "Morning Mix," Washington Post, 5/25; Neighmond, "Shots," NPR, 5/24; Shcherbina et al., Journal of Personalized Medicine, 5/24; Stanford University release, 5/24).
From FitBit to Apple Watch: More about the Internet of Things in health care
As more and more devices in the world become increasingly electronic, software-enabled, intelligent, wireless, and connected, it is worth exploring how the Internet of Things—the connectivity and interoperability of increasingly smart objects, such as appliances, sensors, controllers, wearables, and medical devices—can be leveraged in health care and health IT.