Quality of case management of sexually transmitted diseases: comparison of the methods for assessing the performance of providers.
This article examines the reliability and validity of direct observation of patient-provider encounters, interviews with providers, and use of patients simulating sexually transmitted diseases (STD) as methods for assessing the quality of STD case management in developing countries. Data were collected during an STD health facility survey in Malawi; the performance of 49 providers was observed, and the providers were also interviewed; 20 of them were visited by a simulated patient complaining of urethral discharge. Agreement (based on the kappa statistic) was generally poor between direct observation and provider-interview data, and also between direct observation and simulated-patient data. In contrast, percentage agreements between direct observation and simulated-patient data were often high. Multiple observations on providers indicated that a provider's behaviour is not consistent across several patients. Simulated-patient data are probably the best in reflecting normal performance, but their feasibility for routine quality assessment is limited because the provider's behaviour is not consistent and would require multiple data points. Direct observation data are the best option for assessing quality if the results are assumed to reflect better than normal levels of quality of care. Data from interviews with providers should be viewed with caution, because they may reflect provider knowledge and not necessarily performance.