The Apple Watch provides highly accurate measurements of heart rate and step count, but their estimates of calories burned can be significantly off, according to a new peer-reviewed meta-analysis conducted by researchers at the University of Mississippi (via CNET).

The study reviewed 56 previously published studies evaluating the Apple Watch's performance against gold-standard clinical tools in three core areas: heart rate monitoring, step count tracking, and energy expenditure estimation.

The researchers reported low mean absolute percentage errors (a common metric used to assess measurement accuracy) of 4.43% for heart rate and 8.17% for step counts. These fall within the threshold generally considered acceptable for consumer-grade fitness devices. In contrast, the average error for energy expenditure was 27.96%, more than three times the margin considered acceptable for accurate measurement.

The analysis incorporated data from studies spanning multiple Apple Watch models and user groups. The high error margin in calorie estimation was consistent across all cohorts and forms of physical activity.

These devices are great for keeping track of habits and staying motivated. But do not take every number as 100% truth, especially the calories. Think of it as a helpful guide, not a diagnostic tool. It is useful but not perfect.

The findings align with previous independent evaluations that have raised concerns about the reliability of calorie burn estimates from consumer-grade wearable devices like Apple Watches.

While the Apple Watch has undergone continuous generational improvements since its debut in 2015, the researchers noted that even newer models still exhibit considerable error in calorie estimation. However, the study did observe a trend toward improved accuracy in more recent models:

While we cannot say every update is a big leap forward, there is a noticeable trend of gradual improvements over time. It shows that Apple is refining the technology over time.

The researchers emphasized that their analysis is not intended to discredit the utility of wearables, but rather to clarify their limitations and inform both consumers and smartwatch makers.

By showing where the weaknesses are, we can help developers get real feedback. If they know what needs to be fixed, they can design better sensors or algorithms. Our findings can guide improvements and help make these devices more useful for both everyday users and health care providers.

Apple does not publish the algorithms used in Apple Watch fitness tracking, nor does it claim that the device provides clinical-grade energy expenditure measurements. The company has consistently positioned the Apple Watch as a general wellness tool rather than a medical diagnostic device, though it has introduced several advanced health features in recent years, such as body temperature monitoring and sleep apnea detection.

Source: Macrumors