Study design: statistical and methodologic considerations.

In randomized controlled trials (RCTs), study participants and research personnel are often blinded to minimize biases related to knowing treatment allocation. To determine if blinding was effective, participants may be asked which treatment they believe they received ("treatment guess"). This descriptive review characterized blinding assessment (BA) reporting in pharmacotherapy and neuromodulation neuropathic pain RCTs. Of 288 papers, 36 (12.5%) reported a BA. One paper reported the results of 2 studies, so in total 37 studies with a BA were assessed. Of these, 19 were crossover, 17 parallel, and 1 partial crossover in design. All 37 studies assessed participant blinding, and 10 also assessed investigator blinding. Approximately 27% included an "unsure" answer option for treatment guess, and 38% asked the reason for the guess. There were no clear patterns in BA reporting across time nor based on treatment type. Seventeen trials provided sufficient data to calculate Bang Blinding Index (BI) to determine blinding success. Participants remained blinded (BI = 0 ± 0.2) in 10/17 placebo and 10/17 treatment arms, 6 placebo and 5 treatment arms had a BI > 0.2 suggesting possible unblinding, whereas 1 placebo and 2 treatment arms had a BI < -0.2 suggesting misinformed guessing. Overall, we found that BAs are done in a minority of published neuropathic pain trials and with variable methodology. Given the importance of minimizing risk of bias because of treatment unblinding, future studies should consider including BAs, and further consensus building is necessary to determine if and how BAs should be conducted and interpreted in analgesic clinical trials.

Bias↗

PubMed2000

First steps in analysing NHS waiting times: avoiding the 'stationary and closed population' fallacy.

The aim of this paper is to demonstrate the effect of excluding incomplete observations and competing events when calculating cross-sectional measures of NHS waiting times, and to obtain a more accurate estimate of the 'time-to-admission' of those listed on NHS waiting lists using life-table methods. The official 'times-since-enrollment' of all elective 'admissions' in England, 1 July to 31 December 1994 inclusive, were extracted from Hospital Episode Statistics. The official 'times-to-census' of all those on a waiting list in England at 30 September 1994 were obtained from aggregated KH07 data. The percentage waiting at least three months, at least six months etc., was calculated separately for each data set and compared with a period life-table derived from the combined data. The cumulative likelihood of elective admission is markedly overestimated across the whole range of waiting times. The experience of those still waiting, those removed from the list, those suspended or deferred and those put to the back of the queue is not taken into account in the calculation of official waiting times. The Department of Health currently presents the 'time-since-enrollment' of those admitted as though it indicates how long all patients can expect to wait for admission. The consequent bias in published summary statistics incorrectly quantifies the real experience of patients. It is recommended that calculation of waiting times from KH07 census counts and Hospital Episode Statistics be reconsidered in the light of what patients, clinicians, managers and politicians need to know about treatment delay.

Bias↗

PubMed2000

Heterogeneity and statistical significance in meta-analysis: an empirical study of 125 meta-analyses.

For meta-analysis, substantial uncertainty remains about the most appropriate statistical methods for combining the results of separate trials. An important issue for meta-analysis is how to incorporate heterogeneity, defined as variation among the results of individual trials beyond that expected from chance, into summary estimates of treatment effect. Another consideration is which 'metric' to use to measure treatment effect; for trials with binary outcomes, there are several possible metrics, including the odds ratio (a relative measure) and risk difference (an absolute measure). To examine empirically how assessment of treatment effect and heterogeneity may differ when different methods are utilized, we studied 125 meta-analyses representative of those performed by clinical investigators. There was no meta-analysis in which the summary risk difference and odds ratio were discrepant to the extent that one indicated significant benefit while the other indicated significant harm. Further, for most meta-analyses, summary odds ratios and risk differences agreed in statistical significance, leading to similar conclusions about whether treatments affected outcome. Heterogeneity was common regardless of whether treatment effects were measured by odds ratios or risk differences. However, risk differences usually displayed more heterogeneity than odds ratios. Random effects estimates, which incorporate heterogeneity, tended to be less precisely estimated than fixed effects estimates. We present two exceptions to these observations, which derive from the weights assigned to individual trial estimates. We discuss the implications of these findings for selection of a metric for meta-analysis and incorporation of heterogeneity into summary estimates. Published in 2000 by John Wiley & Sons, Ltd.

Bias↗