Sequential stopping rules and sequentially adjusted P values: does one require the other?

Abstract

During the course of a clinical trial it is normally necessary to conduct periodic reviews of the data in order to determine whether the trial should be terminated. Since these reviews affect the probability of the final outcome, many statisticians recommend that the P values quoted for a clinical trial be sequentially adjusted to account for the possibility of premature termination. In this article it is argued that the sequentially adjusted P value is an inappropriate measure of the strength of evidence justified by a clinical trial. This is because the size of sequentially adjusted P values will vary according to actions that might have been taken if the trial had gone differently than it in fact did. Although such contingencies will effect the frequency of occurrence of certain events in hypothetical sequence of trial replications, it is hard to see why decisions that would have been made in response to outcomes that did not occur should have any bearing on the strength of evidence that can be attributed to the results that were actually observed. The credibility merited by a clinical trial depends not only on the implausibility of the observed results under the null hypothesis, but also on factors such as the medical plausibility of hypothesis well supported by the data, and the extent to which observed results have been predicted in advance. It is argued that publishing these factors along with fixed sample P values is the best way to indicate the degree of certainty that should be attributed to the conclusions of a clinical trial.