Search PubMed⌕ Search
Home/Archive/PubMed

PubMed · 11297884

Adjusting for multiple testing--when and how?

R Bender·S Lange
View original source ↗Export metadata ↓

Abstract

Multiplicity of data, hypotheses, and analyses is a common problem in biomedical and epidemiological research. Multiple testing theory provides a framework for defining and controlling appropriate error rates in order to protect against wrong conclusions. However, the corresponding multiple test procedures are underutilized in biomedical and epidemiological research. In this article, the existing multiple test procedures are summarized for the most important multiplicity situations. It is emphasized that adjustments for multiple testing are required in confirmatory studies whenever results from multiple tests have to be combined in one final conclusion and decision. In case of multiple significance tests a note on the error rate that will be controlled for is desirable.

Explore related subjects

BiasConfounding Factors, EpidemiologicData Interpretation, StatisticalEpidemiologic MeasurementsEpidemiologic MethodsHumansModels, StatisticalReproducibility of ResultsJournal ArticleReview

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

R Bender, S Lange. 2001. Adjusting for multiple testing--when and how?. https://doi.org/10.1016/s0895-4356(00)00314-0

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2026

Assessment of blinding in pharmacotherapy and noninvasive neuromodulation randomized controlled trials for neuropathic pain in adults.

In randomized controlled trials (RCTs), study participants and research personnel are often blinded to minimize biases related to knowing treatment allocation. To determine if blinding was effective, participants may be asked which treatment they believe they received ("treatment guess"). This descriptive review characterized blinding assessment (BA) reporting in pharmacotherapy and neuromodulation neuropathic pain RCTs. Of 288 papers, 36 (12.5%) reported a BA. One paper reported the results of 2 studies, so in total 37 studies with a BA were assessed. Of these, 19 were crossover, 17 parallel, and 1 partial crossover in design. All 37 studies assessed participant blinding, and 10 also assessed investigator blinding. Approximately 27% included an "unsure" answer option for treatment guess, and 38% asked the reason for the guess. There were no clear patterns in BA reporting across time nor based on treatment type. Seventeen trials provided sufficient data to calculate Bang Blinding Index (BI) to determine blinding success. Participants remained blinded (BI = 0 &#xb1; 0.2) in 10/17 placebo and 10/17 treatment arms, 6 placebo and 5 treatment arms had a BI > 0.2 suggesting possible unblinding, whereas 1 placebo and 2 treatment arms had a BI < -0.2 suggesting misinformed guessing. Overall, we found that BAs are done in a minority of published neuropathic pain trials and with variable methodology. Given the importance of minimizing risk of bias because of treatment unblinding, future studies should consider including BAs, and further consensus building is necessary to determine if and how BAs should be conducted and interpreted in analgesic clinical trials.

Bias↗
PubMed2001

Country estimates of maternal mortality: an alternative model.

Ever since the publication of country level estimates of maternal mortality for 1990 by WHO and UNICEF, there has been some degree of controversy about these estimates. The recent publication of a 1995 revision, based on the modification of the multivariate model used for 1990, has not managed to put this controversy to rest. Countries with national estimates of their own have generally protested against the higher figures resulting from the multivariate modelling approach used by WHO and UNICEF, but some experts have also objected to the model itself. As a result of earlier discussions with the WHO/UNICEF team, some adjustments were incorporated into their model, notably the age standardization of maternal mortality ratios (MMRs) and proportions maternal among deaths of females of reproductive age (PMDF) of demographic and health surveys (DHS) direct sisterhood data, as the use of unstandardized values was shown to cause systematic biases. However, a model feature that continued to be controversial was the use of the PMDF as the dependent variable. As will be shown in this paper, the use of this dependent variable has a number of conceptual and practical disadvantages, such as its dependence on non-maternal deaths and the need for separate projections of births and deaths of women of reproductive age, in order to convert the estimated PMDF into a more conventional MMR. The latter greatly increases the uncertainty of the resulting MMR estimates, even though this additional variance is ignored in the WHO/UNICEF estimates of confidence intervals. On balance, the MMR, while also subject to some legitimate objections, is still considered preferable as an independent variable. This paper therefore derives alternative country estimates for 1995 based on a multivariate model of the MMR. The model is shown to lead to smaller root mean square relative errors of the MMR estimates. While the overall number of maternal deaths estimated worldwide is very similar to the number reached by WHO/UNICEF, there are major disagreements with respect to particular countries. Finally, a discussion is included on the appropriate way to incorporate the DHS direct sisterhood data, as this affects the results substantially.

Bias↗
PubMed2001

Evaluating the safety of medicines, with particular reference to contraception.

Toxicological studies and clinical trials cannot be expected to predict all important adverse effects of medicines and contraceptives. Post-marketing surveillance is essentially an epidemiological task that involves detecting associations between drugs and events. The first alerts about drug safety problems have often come from case reports, but epidemiological studies are needed to confirm adverse (or beneficial) effects and to provide quantitative information. This article illustrates methodological principles by considering three examples from the field of contraceptive safety: oral contraceptives and breast cancer, intrauterine contraception and pelvic inflammatory disease, and newer oral contraceptives and venous thromboembolism. Key issues that emerge include bias and confounding, the place of subgroup analyses, random error, and the use of computerized databases. In research on contraceptive and drug safety, conclusions usually need to be based on careful assessment of multiple observational studies.

Bias↗