Search PubMed⌕ Search
Home/Archive/PubMed

PubMed · 2655641

Perspectives on statistical significance testing.

R F Woolson·J C Kleinman
View original source ↗Export metadata ↓

Abstract

The question of whether statistical significance testing should be used for the analysis of public health and epidemiologic data has received considerable attention in recent years. In this paper we have described some of the arguments for and against the use of hypothesis testing for the analysis of biomedical data. In addition, we have reviewed the literature from related fields, in particular sociology and psychology, in which similar discussions have taken place within the last 30 years. Many of the significance testing criticisms in these scientific fields have been raised in the more recent discussions taking place in the biomedical field. We present an example that emphasizes the use of both confidence interval estimation and significance testing. The example is particularly pertinent because it represents a more complex problem than has generally been discussed by critics of significance testing. Much of the discussion on this topic has focused on simple data analysis, such as the analysis of a 2 x 2 table or problems involving simple linear regression. Most epidemiologic data are far more complicated and warrant the use of both confidence interval estimation and significance testing for statistical analysis. Both of these techniques have no doubt been misused in the analysis of data. These misuses may have arisen from a lack of understanding of the role of statistical methods in data analysis and the choice of such methods for data analysis. If used prudently and judiciously, significance testing can help reduce the number of variables involved in a statistical analysis, thereby resulting in shorter confidence intervals for the models presented. Both significance testing and confidence interval estimation can serve and have served very useful functions for the analysis of public health and biomedical data.

Explore related subjects

Data Interpretation, StatisticalEpidemiologic MethodsHumansModels, StatisticalResearchJournal ArticleReview

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

R F Woolson, J C Kleinman. 1989. Perspectives on statistical significance testing.. https://doi.org/10.1146/annurev.pu.10.050189.002231

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2007

A comparison of regression trees, logistic regression, generalized additive models, and multivariate adaptive regression splines for predicting AMI mortality.

Clinicians and health service researchers are frequently interested in predicting patient-specific probabilities of adverse events (e.g. death, disease recurrence, post-operative complications, hospital readmission). There is an increasing interest in the use of classification and regression trees (CART) for predicting outcomes in clinical studies. We compared the predictive accuracy of logistic regression with that of regression trees for predicting mortality after hospitalization with an acute myocardial infarction (AMI). We also examined the predictive ability of two other types of data-driven models: generalized additive models (GAMs) and multivariate adaptive regression splines (MARS). We used data on 9484 patients admitted to hospital with an AMI in Ontario. We used repeated split-sample validation: the data were randomly divided into derivation and validation samples. Predictive models were estimated using the derivation sample and the predictive accuracy of the resultant model was assessed using the area under the receiver operating characteristic (ROC) curve in the validation sample. This process was repeated 1000 times-the initial data set was randomly divided into derivation and validation samples 1000 times, and the predictive accuracy of each method was assessed each time. The mean ROC curve area for the regression tree models in the 1000 derivation samples was 0.762, while the mean ROC curve area of a simple logistic regression model was 0.845. The mean ROC curve areas for the other methods ranged from a low of 0.831 to a high of 0.851. Our study shows that regression trees do not perform as well as logistic regression for predicting mortality following AMI. However, the logistic regression model had performance comparable to that of more flexible, data-driven models such as GAMs and MARS.

Data Interpretation, Statistical↗
PubMed2007

Multiple linear regression with some correlated errors: classical and robust methods.

In this paper we consider classical and robust methods of estimation and diagnostics for the multiple linear regression model when some of the errors are correlated. This work was motivated by the analysis of a medical data set, from an observational study aimed at identifying factors affecting the outcome of a surgical method for the correction of scoliosis (abnormal lateral spinal curvature). There are 392 observations but some of them are on the same patient (double curves). It seems adequate to consider a multiple linear regression model but, since it is not desirable to discard the double curves, the assumption of non-correlated errors is clearly violated, and this is indeed confirmed by related diagnostics on the residuals (Durbin-Watson test). A more appropriate model retains the linear structure but allows for non-null correlation between the errors on the same patient. We propose two different procedures for the estimation of the parameters of the linear model and the correlation parameters: maximum likelihood assuming normal errors and a robustified version obtained by plugging-in results from robust linear regression. The latter procedure is designed to be resistant to outlying observations or error distributions with heavy tails and has produced the most satisfactory results for the analysed data set.

Data Interpretation, Statistical↗
PubMed2007

Adaptive design method based on sum of p-values.

Bauer and Kohne proposed an adaptive design using Fisher's combination of independent p-values based on subsamples from different stages (Biometrics 1994; 50(4):1029-1041). Their method provides great flexibility in the selection of statistical methods for hypothesis testing of subsamples. However, the choices for the stopping boundaries are not flexible enough to meet practical needs (Biometrics 2001; 57(3): 886-891). In this paper, an adaptive design method is proposed using linear combination of the independent p-values. The method provides great flexibility in the selection of stopping boundaries and no numerical integration is required for the two-stage designs. The stopping boundaries and p-values can be calculated manually. The operating characteristics of the adaptive designs are studied using computer simulations with and without sample size adjustment. Examples are presented for superiority and non-inferiority trials with different endpoints (normal, binary, and survival) under different adaptations. The statistical efficiency of the proposed method is compared with other methods based on conditional power.

Data Interpretation, Statistical↗