[Clinical study design].

PURPOSE: Factors associated with the survival of truth of clinical conclusions in the medical literature are unknown. The authors hypothesized that conclusions derived from studies using better methodology should have a longer half-life. DATA SOURCES: MEDLINE and hand searches of journals with studies on cirrhosis and hepatitis. STUDY SELECTION: Original articles and meta-analyses published from 1945 to 1999 about cirrhosis or hepatitis in adults. DATA SYNTHESIS: In 2000, 285 of 474 conclusions (60%) were still considered to be true, 91 (19%) were considered to be obsolete, and 98 (21%) were considered to be false. The half-life of truth was 45 years. The 20-year survival of conclusions derived from meta-analysis was lower (57% +/- 10%) than that from nonrandomized studies (87% +/- 2%) (P < 0.001) or randomized trials (85% +/- 3%) (P < 0.001). The survival of conclusions was not different when studies of high methodologic quality were compared with those of low quality. In randomized trials, the 50-year survival rate was higher for 52 negative conclusions (68% +/- 13%) than for 118 positive conclusions (14% +/- 4%) (P < 0.001). CONCLUSIONS: Contrary to the authors' hypothesis, conclusions based on recognized, good methodology had no clear survival advantage. To better convince clinicians of the long-term utility of evidence-based medicine, better prognostic factors should be developed.

Clinical Trials as Topic↗

PubMed2002

Do multiple outcome measures require p-value adjustment?

BACKGROUND: Readers may question the interpretation of findings in clinical trials when multiple outcome measures are used without adjustment of the p-value. This question arises because of the increased risk of Type I errors (findings of false "significance") when multiple simultaneous hypotheses are tested at set p-values. The primary aim of this study was to estimate the need to make appropriate p-value adjustments in clinical trials to compensate for a possible increased risk in committing Type I errors when multiple outcome measures are used. DISCUSSION: The classicists believe that the chance of finding at least one test statistically significant due to chance and incorrectly declaring a difference increases as the number of comparisons increases. The rationalists have the following objections to that theory: 1) P-value adjustments are calculated based on how many tests are to be considered, and that number has been defined arbitrarily and variably; 2) P-value adjustments reduce the chance of making type I errors, but they increase the chance of making type II errors or needing to increase the sample size. SUMMARY: Readers should balance a study's statistical significance with the magnitude of effect, the quality of the study and with findings from other studies. Researchers facing multiple outcome measures might want to either select a primary outcome measure or use a global assessment measure, rather than adjusting the p-value.

Clinical Trials as Topic↗