Can randomised trials rely on existing electronic data? A feasibility study to explore the value of routine data in health technology assessment.
OBJECTIVES: To estimate the feasibility, utility and resource implications of electronically captured routine data for health technology assessment by randomised controlled trials (RCTs), and to recommend how routinely collected data could be made more effective for this purpose. DATA SOURCES: Four health technology assessments that involved patients under care at five district general hospitals in the UK using four conditions from distinct classical specialties: inflammatory bowel disease, obstructive sleep apnoea, female urinary incontinence, and total knee replacement. Patient-identifiable, electronically stored routine data were sought from the administration and clinical database to provide the routine data. REVIEW METHODS: Four RCTs were replicated using routine data in place of the data already collected for the specific purpose of the assessments. This was done by modelling the research process from conception to final writing up and substituting routine for designed data activities at appropriate points. This allowed a direct comparison to be made of the costs and outcomes of the two approaches to health technology assessment. The trial designs were a two-centre randomised trial of outpatient follow-up; a single-centre randomised trial of two investigation techniques; a three-centre randomised trial of two surgical operations; and a single-centre randomised trial of perioperative anaesthetic intervention. RESULTS: Generally two-thirds of the research questions posed by health technology assessment through RCTs could be answered using routinely collected data. Where these questions required analysis of NHS resource use, data could usually be identified. Clinical effectiveness could also be judged, using proxy measures for quality of life, provided clinical symptoms and signs were collected in sufficient detail. Patient and professional preferences could not be identified from routine data but could be collected routinely by adapting existing instruments. Routine data were found potentially to be cheaper to extract and analyse than designed data, and they also facilitate recruitment as well as have the potential to identify patient outcomes captured in remote systems that may be missed in designed data collection. The study confirmed previous evidence that the validity of routinely collected data is suspect, particularly in systems that are not under clinical and professional control. Potential difficulties were also found in identifying, accessing and extracting data, as well as in the lack of uniformity in data structures, coding systems and definitions. CONCLUSIONS: Routine data have the potential to support health technology assessment by RCTs. The cost of data collection and analysis is likely to fall, although further work is required to improve the validity of routine data, particularly in central returns. Better knowledge of the capability of local systems and access to the data held on them is also essential. Routinely captured clinical data have real potential to measure patient outcomes, particularly if the detail and precision of the data could be improved.