Biomedical subjects
Sally Hopewell
Publications and source records attributed to Sally Hopewell.
Systematic reviews of diagnostic tests in cancer: review of methods and reporting.
OBJECTIVES: To assess the methods and reporting of systematic reviews of diagnostic tests. DATA SOURCES: Systematic searches of Medline, Embase, and five other databases identified reviews of tests used in patients with cancer. Of these, 89 satisfied our inclusion criteria of reporting accuracy of the test compared with a reference test, including an electronic search, and published since 1990. REVIEW METHODS: All reviews were assessed for methods and reporting of objectives, search strategy, participants, clinical setting, index and reference tests, study design, study results, graphs, meta-analysis, quality, bias, and procedures in the review. We assessed 25 randomly selected reviews in more detail. RESULTS: 75% (67) of the reviews stated inclusion criteria, 49% (44) tabulated characteristics of included studies, 40% (36) reported details of study design, 17% (15) reported on the clinical setting, 17% (15) reported on the severity of disease in participants, and 49% (44) reported on whether the tumours were primary, metastatic, or recurrent. Of the 25 reviews assessed in detail, 68% (17) stated the reference standard used in the review, 36% (9) reported the definition of a positive result for the index test, and 56% (14) reported sensitivity, specificity, and sample sizes for individual studies. Of the 89 reviews, 61% (54) attempted to formally synthesise results of the studies and 32% (29) reported formal assessments of study quality. CONCLUSIONS: Reliability and relevance of current systematic reviews of diagnostic tests is compromised by poor reporting and review methods.
Reporting of trials presented in conference abstracts needs to be improved.
OBJECTIVES: To assess how trial information reported in conference abstracts differs to their subsequent full publication. METHODS: Randomized trials reported at the American Society of Clinical Oncology conference (1992) were identified. CENTRAL and PubMed (December 2002) were searched to identify corresponding full publications. A checklist (based on CONSORT) was used to compare abstracts for 37 trials with their full publication. RESULTS: Some aspects were well reported. Ninety-five percent of study objectives, 92% of participant eligibility, 100% of trial interventions, and 84% of primary outcomes were the same in both abstract and full publication. Other areas were more discrepant. Forty-six percent reported the same number of participants randomized in the abstract and full publication; only 22% reported the same number analyzed (median number analyzed per trial was 96 for abstracts and 117 for full publications). Eighty-two percent of trials were closed to follow-up in the full publication compared to 19% of abstracts. Lack of information was a major problem in assessing trial quality: no abstracts reported on allocation concealment, 16% reported on blinding and 14% reported intention to treat analysis. These figures were 49, 19, and 46%, respectively, for full publications. CONCLUSION: The information given for trials in conference proceedings can be unstable, especially for trials presenting early or preliminary results, and needs to be improved.
Abstracts presented at the American Society of Clinical Oncology conference: how completely are trials reported?
PURPOSE: To assess how completely trials published in conference proceedings are reported and whether this has changed over time. METHODS: Conference abstracts published at the American Society of Clinical Oncology (ASCO) conference (1992 and 2002) were read to identify reports of randomized trials. A checklist was devised (based on CONSORT) to assess the completeness of reporting. RESULTS: Four-hundred and ninety-four abstracts reporting randomized trials were identified; 209 in 1992 and 285 in 2002. More trials included "randomized" in the title in 2002 compared to 1992 (54% versus 36%). Almost no trials stated the method of allocation concealment, 12% stated the method of blinding, 95% described eligible participants and 98% described the interventions. Ninety-five per cent reported the number of participants in each trial. The median number of participants per trial increased over time; 120 in 1992 and 209 in 2002 (P < 0.01). In 1992, 67% of trials reported the number of participants analysed, compared to only 49% in 2002 (P < 0.01), 28% reported or suggested intention to treat analysis dropping to 15% in 2002. Twenty-nine abstracts in 2002 and five in 1992 reported no results, with a promise of presentation at the meeting. CONCLUSIONS: The reporting of conference abstracts for trials should be improved to further facilitate understanding of their conduct and validity.
How important is the size of a reprint order?
OBJECTIVES: This study aims to assess the impact of articles with very high reprint orders ("high-reprint articles") by measuring their citation in the subsequent literature as compared with a control group of articles. METHODS: The twenty-one articles (published in the Lancet in 1998) with reprint orders of over 10,000 were matched with a control set of twenty-one articles with smaller reprint orders. The Science Citation Index was used to obtain the number of citations for each of the forty-two articles. RESULTS: The twenty-one high-reprint articles were cited 2,548 times; the mean number of citations was 121 (range, 3 to 499 citations per article). Five of the twenty-one high-reprint articles had more than 200 citations, but seven (33%) were cited twenty-five times or fewer. The twenty-one control articles were cited 986 times; the mean number of citations was forty-seven (range, 1 to 165). Fifteen (71%) of the twenty-one control articles were cited twenty-five times or fewer. Thirteen of the high-reprint articles were reports of randomized trials with a mean of 163 citations. In the control articles, six were reports of randomized trials with a mean of eighty-eight citations. CONCLUSIONS: Articles with a high-reprint order were cited more frequently than other articles. However, some high-reprint articles were cited infrequently. If the size of a reprint order is related to the importance of an article, those articles with very high reprint orders may, therefore, be perceived as more important. Further research is needed to explore other aspects of the relative importance and impact of high-reprint articles.
Assessing the impact of abstracts from the Thoracic Society of Australia and New Zealand in Cochrane reviews.
OBJECTIVE: The aim of this study was to assess the potential impact of including trials, reported in conference abstracts from the Thoracic Society of Australia and New Zealand, in Cochrane reviews. METHODOLOGY: Abstracts from the Thoracic Society of Australia and New Zealand, published in the Australian and New Zealand Journal of Medicine (1981-1998), were read to identify all reports of randomized trials. A search was carried out of the Cochrane Database of Systematic Reviews (Issue 1, 2002) for each trial reported in a conference abstract to try to identify Cochrane reviews in which the conference abstract might be eligible for inclusion. If it was unclear, the authors of the review were contacted. RESULTS: A total of 187 reports of randomized trials were identified: 101 (54%) had been published as a full report and 86 (46%) remained unpublished. Thirty-four (72%) were reports of randomized controlled trials and 52 (28%) were quasi-randomized or possibly randomized trials. The total number of patients included in the trials was 9691; range 4-1203 (median 20; IQR 11-47). No possible Cochrane review was found for 145 of the 187 trials reported in the conference abstracts. Possible reviews were identified for 42 trials, 24 of which were already mentioned in Cochrane reviews. For the remaining 18 trials, only three were said to be eligible for inclusion. CONCLUSION: A search of conference abstracts identified a number of reports of randomized trials, potentially eligible for inclusion in reviews of health care. However, the majority of trials were not relevant for inclusion in an existing Cochrane review. This is most likely because there are currently too few reviews to deal with the topics covered in the abstracts.
Publishing protocols of systematic reviews: comparing what was done to what was planned.
CONTEXT: Publication of research protocols minimizes bias by explicitly stating a priori hypotheses and methods without prior knowledge of results. METHODS: We conducted a retrospective comparative study to assess the extent to which the content of published Cochrane reviews had changed compared with their previously published protocols and to assess any potential impact these changes may have had in introducing bias to the study. We identified previously published protocols for new Cochrane reviews appearing in The Cochrane Library; 2000, issue 3. The texts of published protocols and completed reviews were compared. Two raters independently identified changes to the different sections of the protocol and classified the changes as none, minor, or major. RESULTS: Of the 66 new Cochrane reviews, we identified a previously published protocol for 47 reviews. Of these, 43 reviews had at least 1 section that had undergone a major change compared with the most recently published protocol. The greatest variation between protocols and reviews was in the methods section, in which 68% of reviews (n = 32) had undergone a major change. Changes made in other sections that may have resulted in the introduction of bias included narrowing of objectives, addition of comparisons or new outcome measures, broadening of criteria for the types of study design included, and narrowing of types of participants included. CONCLUSIONS: Research protocols, even if published, are likely to remain, at least to some extent, iterative documents. We found that a large number of changes were made to Cochrane reviews, some of which could be prone to influence by prior knowledge of results. Even if many of the changes between protocol and review improve the overall study, the reasons for making these should be clearly identified and documented within the final review.