Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “External validation”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2Linked to original sources
PubMed2006

Development and external validation of an extended 10-core biopsy nomogram.

OBJECTIVES: To test the accuracy of a previously externally validated sextant biopsy nomogram in referred men exposed to > or =10 or more biopsy cores. Moreover, we explored the hypothesis that a more accurate predictive tool could be developed. METHODS: Previous nomogram predictors (age, digital rectal examination, prostate-specific antigen, and percent free PSA) were used to assess the accuracy of our previous nomogram in a cohort consisting of 2900 men referred for prostatic evaluation. Moreover, these variables were complemented with sampling density (SD) (i.e., ratio of gland volume and the number of planned biopsy cores) within multivariable logistic regression models (LRM) predicting presence of prostate cancer (pCA) on the initial 10 or more core biopsy. The LRMs were used to develop and internally validate (200 bootstrap resamples) a new nomogram in 1162 men from Hamburg, Germany. The LRMs' external validity was tested in three separate cohorts (Hamburg, n=582; Milan, n=961; Seattle, n=195). RESULTS: The contemporary external validation of the previously validated sextant nomogram demonstrated 70% accuracy. Internal validation of the new nomogram demonstrated 77% accuracy, and external cohorts demonstrated 73-76% accuracy. CONCLUSIONS: In the era of extended biopsy schemes, previously developed predictive models are less accurate in predicting the probability of pCA on initial biopsy. We developed a new tool that allows obtaining more accurate predictions. Moreover, before biopsy, it also allows defining the ideal ratio between gland volume and the number of planned biopsy cores that would yield the ideal biopsy rate.

Adult↗
PubMed2005

Can we reliably predict long-term mortality after exercise testing? An external validation.

OBJECTIVE: The aim of the study was to derive and externally validate a mortality prediction rule for patients undergoing exercise testing. BACKGROUND: The prognostic value of exercise testing is increasingly appreciated. However, global prognosis estimates ideally should account for numerous routinely obtained variables, including demographics, risk factors, resting electrocardiogram, and multiple exercise test measures. METHODS: A prediction rule was derived by parametric hazards modeling on a derivation set of 46047 Cleveland Clinic patients (age 55 +/- 11 years, 67% male) who had no history of heart failure, valve disease, or atrial fibrillation. Twenty-two variables covering demographics, risk factors, exercise hemodynamics, and electrocardiogram findings at rest and during exercise were considered. The resulting model included 16 variables and was tested on 4981 patients (age 50 +/- 12 years, 55% male) who underwent exercise testing at West Virginia University. RESULTS: In the derivation cohort there were 3173 deaths during a mean of 7 years of follow-up, whereas in the validation cohort there were 180 deaths during a mean of 5 years of follow-up. Comparisons of predicted and observed death rates showed very good agreement among all patients across all spectrums of risk, as well as among prespecified high-risk subgroups. Model discrimination was also good, with c statistic of c = 0.79 in the derivation group and c = 0.81 in the validation cohort. CONCLUSIONS: We have externally validated a mortality prediction rule for patients undergoing exercise testing and confirmed its accuracy among a wide spectrum of patients.

Adult↗
PubMed2006

External validation of comorbid patterns of anxiety disorders in children and adolescents.

To evaluate the external validity of comorbid patterns of anxiety disorders among youth who presented to an anxiety disorders clinic, comorbid cases were compared to "pure" anxiety disorder cases. Children and adolescents (N=329; mean age=10.04 years) and parents were administered structured interviews and four groups were formed, Pure Anxiety, Anxiety+Anxiety, Anxiety+Externalizing, and Anxiety+Depressive, and compared along with four external validation criteria: sociodemographics, clinical phenomenology, psychosocial, and family factors. All comorbid groups were more severe than the pure anxiety group on clinical phenomenology and psychosocial factors. The Anxiety+Depressive Disorders group was most severe on all criteria except sociodemographics. Results provide evidence for the external validity of comorbid diagnostic presentations among anxiety disorders, as there was differential meaningfulness in the diagnostic presentation of a pure anxiety disorder versus anxiety disorder comorbid with other disorders. Assessment and future research implications are discussed.

Adolescent↗
PubMed2003

Estimating the validity of the guilty knowledge test from simulated experiments: the external validity of mock crime studies.

This experiment was designed to examine the external validity of the standard mock-crime procedure used extensively to evaluate the validity of polygraph tests. The authors manipulated the type of mock-crime procedure (standard vs. a more realistic version) and the time of test (immediate vs. delayed) and examined their effects on the validity of the Guilty Knowledge Test (GKT) and the recall rate of the relevant items. The results indicated that only the type of mock-crime affected the 2 outcome variables. The realistic procedure was associated with a lower recall rate and weaker detection efficiency than the standard procedure. However, these effects were mediated by the type of GKT questions used. Practical implications of these results are discussed.

Adult↗
PubMed2002

Behavior change intervention research in healthcare settings: a review of recent reports with emphasis on external validity.

BACKGROUND: Information to judge both the internal and external validity of health behavior research conducted in healthcare settings is vital to translate research findings to practice. This paper reviews the extent to which this research has reported on elements of internal and external validity, with emphasis on the extent to which research has been conducted in representative settings with representative populations. METHODS: A comprehensive review was conducted of controlled interventions for dietary change, physical activity, or smoking cessation conducted in healthcare settings and published in 12 leading health behavior journals between 1996 and 2000. Using the RE-AIM (reach, effectiveness, adoption, implementation, maintenance) framework, the characteristics and results of these studies were summarized to document the extent to which intervention reach, adoption, implementation, and maintenance were reported and what has been learned about each of these dimensions. RESULTS: A total of 36 studies qualified for review. Participation rates among eligible patients were reported in 69% of studies and were generally quite high; in contrast, only 30% of studies reported on participation rates among either healthcare settings or providers. Implementation data were reported in 77% of the studies and these rates were generally high, with the caveat that intervention was often delivered by paid research staff. Long-term maintenance results were reported very consistently at the individual level, but program continuation was almost never reported at the setting level. CONCLUSIONS: We conclude that a much stronger emphasis needs to be placed on the representativeness of providers and settings that are studied. Examples of how this can be done and recommendations for future research are provided.

Feeding Behavior↗
PubMed1992

External validation, repeat determination, and precision of risk estimation in misclassified exposure data in epidemiology.

STUDY OBJECTIVE: The aim was to quantify the difference in precision of risk estimates in epidemiology between the situations where misclassification of exposure is corrected for by external validation and where it is corrected for by internal repeat measurement. Precision was measured in terms of the expected width of the 95% confidence interval on the odds ratio. DESIGN: In a hypothetical case-control study, first with 100 cases and 100 controls, then with 100 cases and 1000 controls (the latter to approximate the cohort study situation), expected estimated odds ratios and confidence intervals were calculated based on postulated underlying true odds ratios and misclassification error rates. The sizes of the confidence intervals using the two design strategies were compared, based on the same number of subjects receiving internal repeat measurements as were used in the external validation study. MAIN RESULTS: Confidence intervals obtained using internal repeat measurement were considerably narrower than those using external validation. Both methods yielded approximately correct point estimates. CONCLUSIONS: In terms of precision, it is preferable to correct for misclassification using internal repeat measurement rather than external validation.

Analysis of Variance↗
PubMed2007

External validation of the San Francisco Syncope Rule.

STUDY OBJECTIVE: We externally validate the ability of the San Francisco Syncope Rule to accurately identify syncope patients who will experience a 7-day serious clinical event. METHODS: Patients who presented to a single academic emergency department (ED) between 8 am and 10 pm with syncope or near-syncope were prospectively enrolled. Treating physicians recorded the presence or absence of all San Francisco Syncope Rule risk factors. Patients were contacted by telephone at 14 days for a structured interview. A 3-physician panel, blinded to the San Francisco Syncope Rule score, reviewed ED medical records, hospital records, and telephone interview forms to identify predefined serious clinical events. The primary outcome was the ability of the San Francisco Syncope Rule to predict any 7-day serious clinical event. A secondary outcome was the ability of the San Francisco Syncope Rule to predict 7-day serious clinical events that were not identified during the initial ED evaluation. RESULTS: Of 592 eligible patients, 477 (81%) provided informed consent. Direct telephone contact or admission/outpatient records were successfully obtained for 463 (97%) patients. There were 56 (12%) patients who had a serious 7-day clinical event, including 16 (3%) who received a diagnosis after the initial ED evaluation. Sensitivity and specificity of the San Francisco Syncope Rule for the primary outcome were 89% (95% confidence interval [CI] 81% to 97%) and 42% (95% CI 37% to 48%), respectively, and 69% (95% CI 46% to 92%) and 42% (95% CI 37% to 48%), respectively, for the secondary outcome. Estimates of sensitivity were minimally affected by missing data and most optimistic assumptions for missing follow-up information. CONCLUSION: In this external validation cohort, the San Francisco Syncope Rule had a lower sensitivity and specificity than in previous reports.

Adult↗
PubMed1998

Assessment of malingering with simulation designs: threats to external validity.

Comprehensive forensic evaluations are predicated on the accurate appraisal of response styles that may affect evaluatees' clinical presentation and experts' conclusions associated with psycholegal issues. In the assessment of malingering, forensic experts often rely heavily on standardized measures that have been validated exclusively via analogue research. While such research augments internal validity, the threats to external validity are readily apparent. As the first study of these threats, type of incentive (positive versus negative), context (a familiar versus unfamiliar scenario), and relevance to the participants was investigated systematically with a between-subjects factorial design. A sample of 231 undergraduates was asked to either (a) feign major depression and given an easily understood description of this disorder or (b) serve as controls responding honestly. They were administered a brief measure of psychopathology (Hopkins Symptom Checklist; Derogatis, Lipman, Rickels, Uhlenhuth, & Covi, 1974) and a recent screen for malingering (Screening Inventory of Malingered Symptoms or SIMS; Smith, 1992) in 1 of 18 experimental conditions. Results suggested that incentive had a main effect on the SIMS. More specifically, simulators under negative incentives appeared more focused in their feigning; they produced more bogus depressed symptoms, but fewer symptoms unrelated to depression. Interactions were also observed between context and incentive, and context and relevance. Implications of these results are explored for both analogue research on malingering and current forensic practice.

Adult↗
PubMed2003

External validity of predictive models: a comparison of logistic regression, classification trees, and neural networks.

BACKGROUND AND OBJECTIVE: The utility of predictive models depends on their external validity, that is, their ability to maintain accuracy when applied to patients and settings different from those on which the models were developed. We report a simulation study that compared the external validity of standard logistic regression (LR1), logistic regression with piecewise-linear and quadratic terms (LR2), classification trees, and neural networks (NNETs). METHODS: We developed predictive models on data simulated from a specified population and on data from perturbed forms of the population not representative of the original distribution. All models were tested on new data generated from the population. RESULTS: The performance of LR2 was superior to that of the other model types when the models were developed on data sampled from the population (mean receiver operating characteristic [ROC] areas 0.769, 0.741, 0.724, and 0.682, for LR2, LR1, NNETs, and trees, respectively) and when they were developed on nonrepresentative data (mean ROC areas 0.734, 0.713, 0.703, and 0.667). However, when the models developed using nonrepresentative data were compared with models developed from data sampled from the population, LR2 had the greatest loss in performance. CONCLUSION: Our results highlight the necessity of external validation to test the transportability of predictive models.

Classification↗
PubMed2004

Predicting outcome after acute ischemic stroke: an external validation of prognostic models.

OBJECTIVE: To externally validate two prognostic models predicting functional outcome and survival 100 days after acute ischemic stroke. METHODS: Using prospectively collected data from 1,470 patients, the authors evaluated two previously developed models. Model I predicts incomplete functional recovery (Barthel Index <95) vs complete functional recovery with 11 variables, whereas model II predicts mortality vs survival with 3 variables. On admission to a participating hospital, patients were registered prospectively and included according to defined criteria. Within 72 hours, predictive variables under investigation were assessed. Follow-up was performed 100 days after the event. RESULTS: Model I correctly predicted 68.1% of the patients who had incompletely recovered or had died and 85.7% of the completely recovered patients, model II 46.9% of the patients who had died and 95.9% of the surviving patients. Both models performed better than the treating physicians' predictions made within 72 hours after admission. CONCLUSION: The resulting prognostic models are useful to correctly stratify treatment groups in clinical trials and to accurately predict the distribution of endpoint variables.

Aged↗
PubMed1989

An external validity study of the MMPI Personality Disorder Scales.

This study examined the external validity of the MMPI Personality Disorder (PD) scales. Patients diagnosed with personality disorders (n = 23), according to a structured interview, were contrasted with medical control subjects and psychiatric patients with no personality disorder (n = 33). The MMPI PD scales could discriminate patients with any personality disorder and patients with specific clusters of personality disorders. Convergent validity was demonstrated by high correlations between the MMPI PD scales and the comparable Millon Clinical Multiaxial Inventory scales, as well as the number of DSM-III personality disorder symptoms. This study represents a preliminary step in the external validation of the MMPI PD scales.

Adult↗
PubMed2004

External validity of childhood disintegrative disorder in comparison with autistic disorder.

To examine the external validity of DSM-IV childhood disintegrative disorder (CDD), 10 children (M = 8.2 yrs) with CDD and 152 gender- and age-matched children with autistic disorder (AD) were compared on 24 variables. The CDD children had a significantly higher rate of epilepsy, significantly less uneven intellectual functioning, and a tendency of greater abnormality in auditory responsiveness than AD children, to validate CDD externally. Their short-term outcome, as shown in the degree of retardation, was not worse than the AD children, which is in disagreement with previous studies reporting worse outcomes in CDD than autism. These results need to be verified by a long-term prospective study that compares CDD and AD patients from infancy.

Autistic Disorder↗
PubMed1992

The external validity of age- versus IQ-discrepancy definitions of reading disability: lessons from a twin study.

Recent research has raised the question of whether age- and IQ-discrepancy forms of reading disability (RD) are distinguishable in terms of either their underlying linguistic deficit or their response to treatment, thus threatening the external validity of the traditional distinction between specific reading retardation and reading backwardness. The present study pursued the external validity of this distinction in three domains: (a) genetic etiology, (b) sex ratio and clinical correlates, and (c) neuropsychological profiles. Each of these domains was explored in the RD (n = 640) and control (n = 436) twins participating in the Colorado Reading Project (514 males, 562 females, with an overall mean age of 12.42 years). Little evidence for external validity was found in terms of the clinical correlates of attention deficit-hyperactivity disorder (ADHD), immune disorders, or handedness. Most importantly, there was no evidence of differential genetic etiology of the two phenotypes in this sample, in that deficits in both phenotypes were similarly heritable (h2g = .40 and .46 for age and IQ phenotypes, respectively) and the genetic correlation between them was high (rG = .88 to .96). However, the genetic and neuropsychological profile analyses did suggest that age- and IQ-discrepancy definitions of RD may relate differentially to component reading processes, such as phonological awareness and orthographic coding.

Attention Deficit Disorder with Hyperactivity↗
PubMed2003

The internal and external validity of the Major Depression Inventory in measuring severity of depressive states.

BACKGROUND: We have developed the Major Depression Inventory (MDI), consisting of 10 items, covering the DSM-IV as well as the ICD-10 symptoms of depressive illness. We aimed to evaluate this as a scale measuring severity of depressive states with reference to both internal and external validity. METHOD: Patients representing the score range from no depression to marked depression on the Hamilton Depression Scale (HAM-D) completed the MDI. Both classical and modern psychometric methods were applied for the evaluation of validity, including the Rasch analysis. RESULTS: In total, 91 patients were included. The results showed that the MDI had an adequate internal validity in being a unidimensional scale (the total score an appropriate or sufficient statistic). The external validity of the MDI was also confirmed as the total score of the MDI correlated significantly with the HAM-D (Pearson's coefficient 0.86, P < or = 0.01, Spearman 0.80, P < or = 0.01). CONCLUSION: When used in a sample of patients with different states of depression the MDI has an adequate internal and external validity.

Adult↗
PubMed2004

ODD and ADHD symptoms in Ukrainian children: external validators and comorbidity.

OBJECTIVE: To examine potential external validators for oppositional defiant disorder (ODD) and attention-deficient/hyperactive disorder (ADHD) symptoms in a Ukrainian community-based sample of 600 children age 10 to 12 years old and evaluate the nature of co-occurring ODD and ADHD symptoms using mother- and teacher-defined groups. METHOD: In 1997, parents, children, and teachers participated in extensive clinical assessments using standard Western measures. Four areas of functioning were assessed: child mental health, parent-child interactions, parental well-being, and school/cognitive performance. RESULTS: Mother-defined ODD versus ADHD symptom groups were differentiated by a history of overactivity and tantrums, behavior in school, and maternal anxiety and hostility. Teacher-defined groups were differentiated by conduct problems, internalizing symptoms, mother-child interactions, and paternal alcohol use. The effects of co-occurring ODD and ADHD symptoms were greater than would be expected based on their separate effects for conduct problems, internalizing symptoms, social problems, academic performance, parent-child relations, and marital discord. CONCLUSIONS: Children with ODD versus ADHD symptoms were not significantly different from each other for the majority of variables examined, and group differences were dependent on the rater used to define symptom groups.

Analysis of Variance↗
PubMed1995

A production task evaluation of individual differences in mental addition skill development: internal and external validation of chronometric models.

A production task paradigm for obtaining reaction times to mental addition stimuli was used for internal and external validation of chronometric models of mental addition processing. The first analysis explored the internal validity of extant chronometric models and found that three models, (a) a tabular memory network retrieval strategy (PRODUCT), (b) a nontabular memory network retrieval strategy (ERROR RATE), and (c) a computational strategy (MIN), were able to encompass individual differences in strategy choice for 155 individuals from Grades 2 to 8 and 111 college students. Patterns of convergent and discriminant validity for these models were also demonstrated. The second analysis explored the external validity of relations among (a) two traditionally measured factor analytic dimensions of ability, Numerical Facility and Perceptual Speed; (b) two information processing dimensions presumed to underlie mental addition. Addition Efficiency and Speediness; and (c) a digit-span measure of Short-Term Memory. We specified a series of two-group (grade school and college) structural equation models to represent the relations among all measures and showed that individual differences in the apparently calculative processes that underlie the traditionally defined ability dimension of Numerical Facility are highly related to individual differences in Addition Efficiency and Speediness of information processing.

Adolescent↗
PubMed2003

External validation of the SAPS II, APACHE II and APACHE III prognostic models in South England: a multicentre study.

OBJECTIVE: External validation of three prognostic models in adult intensive care patients in South England. DESIGN. Prospective cohort study. SETTING: Seventeen intensive care units (ICU) in the South West Thames Region in South England. PATIENTS AND PARTICIPANTS: Data of 16646 patients were analysed. INTERVENTIONS: None. MEASUREMENTS AND RESULTS: We compared directly the predictive accuracy of three prognostic models (SAPS II, APACHE II and III), using formal tests of calibration and discrimination. The external validation showed a similar pattern for all three models tested: good discrimination, but imperfect calibration. The areas under the receiver operating characteristics (ROC) curves, used to test discrimination, were 0.835 and 0.867 for APACHE II and III, and 0.852 for the SAPS II model. Model calibration was assessed by Lemeshow-Hosmer C-statistics and was Chi(2 )=232.1 for APACHE II, Chi(2 )=443.3 for APACHE III and Chi(2 )=287.5 for SAPS II. CONCLUSIONS: Disparity in case mix, a higher prevalence of outcome events and important unmeasured patient mix factors are possible sources for the decay of the models' predictive accuracy in our population. The lack of generalisability of standard prognostic models requires their validation and re-calibration before they can be applied with confidence to new populations. Customisation of existing models may become an important strategy to obtain authentic information on disease severity, which is a prerequisite for reliably measuring and comparing the quality and cost of intensive care.

APACHE↗
PubMed2000

Economic evaluation of parasitic diseases: a critique of the internal and external validity of published studies.

It was estimated that in 1990, major parasitic diseases accounted for 11.7% of the disease burden from communicable disease. As advances in the prevention, diagnosis and treatment of parasitic diseases are made and implemented, there is a growing economic literature to help decision-makers choose the most efficient control method. The aim of this paper is to identify, describe and analyse the available published data on the efficiency of control strategies against parasitic diseases. Internal validity is assessed through the quality of economic evaluations over time using a series of standard questions, and external validity is assessed in terms of the potential to extrapolate results to other settings. This leads to a discussion of the legitimacy and feasibility of pooling data or results from studies for priority setting in the health sector, resulting in three recommendations: to increase the coverage of economic evaluations for parasitic diseases and types of interventions; to improve the internal validity of studies through guidelines and review procedures; and to explore the external validity of research results by examining their predictive validity across settings.

Child↗