Smartwatch vs Hospital Device: Correlation Is Not Agreement

Smartwatches, patches and continuous glucose sensors increasingly claim hospital-grade measurements. The usual proof offered is a high correlation with the clinical device. It sounds rigorous, and it is a classic statistical trap: two devices can correlate almost perfectly and still disagree by a clinically meaningful amount.

A near-perfect correlation

Take ten hypothetical people whose pulse is recorded at the same moment by an ECG and by a wristband. ECG readings run from 62 to 126 beats per minute, and the watch reads slightly higher each time. The Pearson correlation calculator returns r = 0.9996, which sounds flawless. Yet Bland and Altman showed in 1986 that correlation measures how strongly two variables relate, not whether they agree (Bland and Altman, 1986).

Look at the differences

The agreement question uses the differences instead. Here the watch exceeds the ECG by 4.0 beats on average, with a standard deviation of 1.6. The paired samples t-test calculator shows that bias is clearly not zero, with p < 0.001, and the standard deviation calculator gives the spread of the differences.

Bland-Altman limits of agreement then take the mean difference plus or minus 1.96 standard deviations. Here that is about 0.8 to 7.2 beats. In plain language: for most people, the watch will read between 1 and 7 beats above the ECG.

Does the error depend on the level?

Plot each difference against the average of the two readings. In our example the gap grows as heart rate rises, by roughly 0.7 beats for every 10 beats, a proportional bias. A simple linear regression of the differences on the averages detects this with p < 0.001. A single set of limits would then understate the error at high rates.

Is that good enough?

Statistics describes the disagreement but cannot say whether it is acceptable. The limits must be compared with a threshold chosen in advance from clinical need (Giavarina, 2015). A 7-beat error is irrelevant for counting steps and may matter for detecting arrhythmias.

Marketing materials often quote accuracy as a single percentage. A mean absolute percentage error squeezes bias and spread into one number and hides whether the device tends to over-read or under-read. Asking for the full Bland-Altman picture makes any claim far easier to judge, and regulators and clinicians set their own tolerance thresholds.

The same reasoning applies to glucose sensors, blood-pressure cuffs and oxygen monitors. Always ask for the mean difference, the limits of agreement, whether the error changes with the measured level, and how many people were tested. Our article on correlation versus causation covers a related caution about reading r.

Small samples and repeated readings

The limits of agreement are estimates too. With only ten people, each limit is uncertain by roughly ±2 beats, so the stated range of 0.8 to 7.2 could easily shift in a larger study. Good validation studies report confidence intervals for the limits, not just the limits themselves.

Another trap is repeated readings. A hundred readings from ten people are not a hundred independent observations, because each person brings the same quirks to every reading. Treating them as independent makes the device look more precise than it is, so analyses must account for the clustering of readings within individuals.

Before trusting any wearable claim, look for the Bland-Altman plot.

A glowing correlation is only the first line of the story.