Statistical evaluation of agreement between two methods for measuring a quantitative variable.

Abstract

Methodologic research is often concerned with determining whether two methods (procedures, laboratory instruments) can be used interchangeably for measuring some quantitative variable of interest. Logically, one method can be used as a surrogate of another provided the methods show high agreement on the measured results. Although the product-moment correlation (r) is often used as an indicator of agreement, this index is in fact inappropriate for this purpose. The intraclass correlation (r1) is the correct statistic for assessing agreement or consistency between two methods. Another criterion sometimes used for supporting interchangeability is the similarity of the mean measured results obtained by the two methods. However, similarity of means (aggregate agreement) does not necessarily indicate individual-subject agreement, and it is the latter that is the pre-requisite for interchangeability. On the other hand, a marked difference between two means (lack of aggregate agreement) does necessarily indicate lack of individual-subject agreement and therefore non-interchangeability. Herein we suggest that two methods for measuring a quantitative variable can be judged interchangeable provided all of the following conditions are met: first the methods must not exhibit marked additive or nonadditive systematic bias; second the difference between the two mean readings is not "statistically significant"; third, the lower limit of the 95% confidence interval of the intraclass correlation is at least 0.75. Statistical procedures to evaluate these conditions of interchangeability are described in detail. A computer program coded in SAS to carry out the procedures is listed in the Appendix. A similar program coded in DBASE III PLUS for the microcomputer is available upon request.