Search PubMed⌕ Search

SEARCH · Search PubMed

Results for “External validation”

Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4Linked to original sources

Evaluating the relevance, generalization, and applicability of research: issues in external validation and translation methodology.

Starting with the proposition that "if we want more evidence-based practice, we need more practice-based evidence," this article (a) offers questions and guides that practitioners, program planners, and policy makers can use to determine the applicability of evidence to situations and populations other than those in which the evidence was produced (generalizability), (b) suggests criteria that reviewers can use to evaluate external validity and potential for generalization, and (c) recommends procedures that practitioners and program planners can use to adapt evidence-based interventions and integrate them with evidence on the population and setting characteristics, theory, and experience into locally appropriate programs. The development and application in tandem of such questions, guides, criteria, and procedures can be a step toward increasing the relevance of research for decision making and should support the creation and reporting of more practice-based research having high external validity.

Diffusion of Innovation↗

Scales, diagnoses, and child psychopathology: II. Comparing the CBCL and the DISC against external validators.

Using a multimethod multistage screening procedure, the authors interviewed 201 parents and their children with the Diagnostic Interview Schedule for Children (DISC 2.1). In addition, parents completed the Child Behavior Checklist (CBCL) and other survey measures, while their children completed self-report scales. Receiver operating characteristic (ROC) analyses were done to determine optimal cutpoints on the CBCL, referenced to DISC diagnostic "caseness." DISC diagnoses, DISC "stem" symptoms, CBCL scores, and CBCL ROC-outpoints were compared against "external validators," in order to determine the comparative advantages of each approach for assessing child psychopathology. Overall findings suggest that the controversies about "best" assessment strategies may be artificial: When both assessment approaches are compared using similar methods, they are reasonably comparable. However, highly specific diagnostic categories may show fewer relationships with external validators and may therefore need more systematic validational studies.

Adolescent↗

[Predicted functional outcome 100 days after ischemic strokes: design of a prospective study about the external validation of a prognostic model].

The purpose of this study is to externally validate previously developed prognostic models predicting functional outcome 100 days after acute ischaemic stroke. Data prospectively collected from 1,754 patients were used to develop two models predicting functional dependence (Barthel Index < 95) and mortality. Both models were internally validated and calibrated. A prospective multicentre study is being performed to validate the developed models with an independent data set. On admission to one of 15 participating hospitals, all patients with acute ischaemic stroke will be registered prospectively in the co-ordinating centre. Within 72 hours, potential predictive variables will be assessed. 1,975 patients will have to be recruited to achieve a power of 95% at the 5% significance level. The resulting prognostic models will be useful for adequately stratifying treatment groups in clinical trials and to accurately predicting the outcome variable.

Brain Ischemia↗

External validation of the templeton model for predicting success after IVF.

This study aimed to externally validate the prognostic model presented by Templeton in 1996 for live births resulting from IVF treatment. Data were used from the University Hospital, Nijmegen, The Netherlands, from March 1991 to January 1999. The predictive capacity of the model in our population discriminated between those women with a low probability of success and those with a relatively high probability. Despite these encouraging findings, our data show that implementation of the model in clinical decision-making remains difficult. The Templeton model is not applicable or usable in daily clinical practice, because the model did not give more information about the prognosis for the vast majority of the patients. Therefore, the search for better prognostic factors resulting in better predictive models should continue.

Adult↗

Treatment outcomes for students with learning disabilities: how important are internal and external validity?

This study analyzed the magnitude of experimental intervention outcomes as a function of violations in internal and external validity for studies that included students with learning disabilities. The results indicated that treatment outcomes were significantly affected by the following violations: teacher effects, establishing criterion levels of instructional performance, reliance on experimental measures, using different measures between pretest and posttest, using a sample heterogenous in age, and using incorrect units of analysis. Furthermore, the underreporting of information related to ethnicity, locale of the study, psychometric data, and teacher applications positively inflated the magnitude of treatment outcomes. A weighted hierarchical regression analysis revealed that composite scores of the aforementioned high-risk variables accounted for 16% of the total variance in effect size. The implications for interpreting intervention research to practice are discussed.

Analysis of Variance↗

External validity of a prediction rule for residual mass histology in testicular cancer: an evaluation for good prognosis patients.

We assessed the external validity of a prediction rule for nonseminomatous testicular cancer patients. The rule was developed to predict the probability of retroperitoneal metastases being benign (only necrosis/fibrosis) after chemotherapy treatment. Patients with a high probability of benign residual masses might be offered surveillance as opposed to patients with a low probability, who should undergo retroperitoneal lymph node dissection (RPLND). We compared the observed histology with the predicted probability in 105 patients with good prognosis germ cell cancer who underwent RPLND between 1995 and 1998. We found that predicted probabilities higher than 5% were in good agreement with the observed frequencies of benign masses. The area under the receiver operating characteristic curve was 0.76, suggesting that the rule could reasonably discriminate between benign masses and tumour. However, nearly all predicted probabilities (n=101) were lower than 70%, which might be considered as the lowest value at which surveillance offers a reasonable alternative to RPLND. Further, 35% of patients currently under surveillance (84 out of 241) had predicted probabilities lower than 70%. In conclusion, the clinical relevance of the prediction rule was limited for the patients who underwent RPLND; use of the rule would change the policy from RPLND to surveillance in only a few. On the other hand, the rule might support selection of patients for RPLND, who currently are under surveillance.

Fibrosis↗

External validation of the biodegradability prediction model CATABOL using data sets of existing and new chemicals under the Japanese Chemical Substances Control Law.

External validation of the biodegradability prediction model CATABOL was conducted using test data of 338 existing chemicals and 1123 new chemicals under the Japanese Chemical Substances Control Law. CATABOL predicts that 1089 chemicals will have a BOD < 60% while 925 (85%) actually have an observed BOD<60%. The percentage of chemicals with an observed BOD value <60% tends to increase as the predicted BOD values decrease. In contrast, 340 chemicals were predicted to have a BOD > or = 60% and 234 (69%) actually had an observed BOD > or = 60%. The prediction of poor biodegradability was more accurate than the predictions of high biodegradability. The features of chemical structures affecting CATABOL predictability were also investigated.

Biodegradation, Environmental↗

Panel accretion and external validity in adolescent substance use research.

This study investigates the relationship of panel accretion to panel attrition and examines its impact on threats to external validity in adolescent substance use research. Panel accretion involves the addition of new participants to a panel design after initial baseline data has been obtained. Accretion occurs in longitudinal studies in which data is routinely gathered on all participants, as is often the case in prevention or epidemiological research on adolescent substance use. Accretion, attrition, and panel findings for three, 2-year panel samples in grades 7 through 9 are examined. The results indicate that accretion and attrition samples have an isomorphic relationship to one another on a range of dependent measures when drawn from the same population. The results are discussed in terms of their implications for adolescent substance use research, the examination of validity threats due to panel bias, and understanding the ecological validity of research findings.

Adolescent↗

Potential consequences for recruitment, power, and external validity of requirements for additional risk factors for eligibility in randomized controlled trials in secondary prevention of stroke.

BACKGROUND AND PURPOSE: Eligibility criteria determine the external validity (generalizability) of the results of randomized controlled trials. To increase the number of outcome events, and hence statistical power, some recent stroke prevention trials have required additional vascular risk factors for eligibility. METHODS: To assess the merits of additional eligibility criteria in stroke prevention trials, we analyzed data from 3 trials and 1 hospital-referred series of patients with a transient ischemic attack or minor ischemic stroke. Patients were stratified according to 2 sets of additional risk factors similar to those used in recent trials (MATCH, SPORTIF and PRoFESS); risk of stroke, myocardial infarction, or vascular death was calculated in relation to the number of risk factors. RESULTS: Although the observed risk during follow-up did increase with the number of risk factors present (P<0.01 for both sets), the risks in patients with > or =1 risk factors were not substantially greater than those in all patients. Consequently, although the proportions of patients with no risk factors in the 4 cohorts differed substantially between the 2 sets of eligibility criteria (21% to 28% versus 56% to 73%), in neither case could their exclusion be justified on statistical grounds. CONCLUSIONS: The degree of patient selection introduced by use of additional vascular risk factors as eligibility criteria for trials can differ substantially between apparently similar sets of risk factors. Given that the potential for additional eligibility criteria to undermine generalizability and prolong recruitment outweighs any benefits in terms of statistical power, the exclusion of patients with no risk factors is difficult to justify.

Aged↗

Asperger syndrome: diagnosis and external validity.

This article provides an overview of the history and clinical features of Asperger syndrome and considers guidelines for clinical assessment and treatment. A review of issues related to external validity is provided that points out the limitations of current research and lists several potentially beneficial areas of investigation into the nosologic status of the condition. It concludes with a discussion of the unequivocal need of individuals with severe social disabilities for comprehensive and adequate educational services and other treatments, regardless of the fact that the validity and usefulness of this specific diagnostic concept is far from resolved.

Adolescent↗

External validation of medicare claims for breast cancer chemotherapy compared with medical chart reviews.

BACKGROUND: Although Medicare claims data have been increasingly used to examine the patterns and outcomes of cancer chemotherapy, their external validity has not been well studied. OBJECTIVES: We sought to validate Medicare claims for chemotherapy compared with medical chart reviews. PATIENTS AND METHODS: We completed medical chart reviews for 1228 women who were diagnosed with breast cancer at age 65 and older between 1993 and 1999 in New Mexico that were linked with Medicare claims data, achieving an estimated sensitivity of more than 90% and a 0.05 level of precision. RESULTS: Of the 150 subjects identified by Medicare claims as receiving chemotherapy within 6 months of diagnosis, 75% were confirmed by medical records as having received chemotherapy. Of the remaining 25% of cases without chart verification, (1) 33 cases had 7 or more claims for chemotherapy and also had specific chemotherapy drugs indicated in Medicare data, representing 22% (33/150) of all cases that received chemotherapy according to Medicare claims and (2) 4 cases had 1 to 6 claims for chemotherapy, representing 3% (4/150) of all cases with claims for chemotherapy. Of those 1078 subjects who did not receive chemotherapy according to Medicare claims, more than 99% were confirmed by chart reviews. Observed agreement on chemotherapy between Medicare claims and chart reviews was 94% and overall reliability (kappa) was 0.69 (95% confidence interval = 0.63-0.76). CONCLUSIONS: Of cases identified as receiving chemotherapy by Medicare claims, 97% had strong evidence and only 3% had weak evidence for receiving this therapy.

Aged↗

External validity of the profile variability index for the K-ABC, Stanford-Binet, and WISC-R: another cul de-sac.

Clinicians often attach interpretive significance to high variability among subtest scores in IQ profiles of adults and children. In this study we evaluated the external validity of a relatively new measure of IQ subtest variability, the profile variability index (PVI). Within a sample of referred children, we administered the Kaufman Assessment Battery for Children, the Fourth Edition Stanford-Binet, and the Wechsler Intelligence Scale for Children-Revised. We calculated PVIs for each child from each test and correlated these values with achievement scores and indexes of discrepancy between actual and predicted scholastic achievement. Against these external criteria, PVI information from all three cognitive-ability batteries had essentially nil validity. We discuss implications of these findings for future research in this area.

Child↗

External validation of psychological job demands in a bus driver sample.

Psychosocial risk factors have been considered as characteristics of the work environment rather than an individual issue, but their presence in the workplace is usually measured by self-reported questionnaires, based on worker attitudes. The objective of the study was to compare a self-reported measure of psychological job demands in a bus driver sample with selected indicators of bus company activity, as external 'objective' indicators, in order to assess its external validity. The final sample included 713 drivers in 41 routes. Self-reported measures of psychosocial work risk factors were obtained by the Job Content Questionnaire (JCQ). Four external indicators were available for each route: passengers, break times, bus incidents, and regulating actions. Spearman's correlation coefficient was estimated to assess the validity. Correlation coefficients showed that self-reported psychological demands were statistically significant correlated (p<0.05) with two external indicators: bus incidents (rho=0.397), and regulating actions (rho=0.475). Self-reported psychosocial risk factors have been compared with independent external indicators of the psychosocial work environment, trying to avoid any subject's perception. According to our results, psychological demands measured by the JCQ seem to reflect the actual psychosocial work environment. Other studies are necessary to confirm these results and to assess job control and job social support. This would be another step in improving our knowledge of the quality of measurement of psychosocial risk factors.

Adult↗

External validation of the ICH score.

INTRODUCTION: The ICH score is a clinical grading scale that is composed of five components related to outcome after nontraumatic intracerebral hemorrhage (ICH): Glasgow Coma Scale score, ICH volume, presence of intraventricular hemorrhage, infratentorial origin, and age. The ICH score accurately risk-stratifies patients in the cohort from which it was developed, but it has not yet been fully externally validated. The purpose of this study was to determine whether the ICH score accurately risk-stratifies patients in an independent cohort. METHODS: Records of all patients with acute ICH presenting to Stanford Medical Center and Santa Clara Valley Medical Center from 1998 through 2000 were reviewed. Outcome was assessed as mortality at 30 days. RESULTS: A total of 175 patients formed the cohort for analysis. Thirty-day mortality for the entire cohort was 40%. ICH scores ranged from 0 to 5, and each increase in the ICH score was associated with an increase in 30-day mortality (p<0.01 for trend); 4 of 41 patients with an ICH score of 0 died, whereas all 5 patients with a score of 5 died. Receiver operating characteristic curves for the ICH score were similar for this cohort and for the initial ICH score cohort. CONCLUSION: The ICH score accurately stratifies outcome in an external patient cohort. Thus, the ICH score is a validated clinical grading scale that can be easily and rapidly applied at ICH presentation. Ascale such as the ICH score could be used to standardize clinical treatment protocols or clinical studies.

Adult↗

A methodological approach to enhance external validity in simulation based research.

Simulation methodology, as exemplified by use of vignettes, has many advantages, such as standardization of data collection procedure, control of extraneous variables, and manipulation of variables of interest. The main shortcoming is its artificiality and therefore its limited external validity. To determine the degree to which findings from simulation research are transferable to the real world, a comparison can be made using the same measure on artificial and real situations, or the researcher can determine the degree to which a score on the simulation compares with a score in the field. The predictive validity approach was used in a study that used an artificial method to study patient assault. In this study, the intent was to determine the degree of accuracy that the subjects' causal attribution scores on an assault vignette were predictive of causal attribution scores in the actual assault situation.

Guilt↗

Maximizing internal and external validity in MMPI malingering research: a study of a military population.

The authors investigated the effectiveness of various commonly used Minnesota Multiphasic Personality Inventory (MMPI; Hathaway & McKinley, 1943) indices of exaggeration and malingering in detecting suspected malingering in a military sample of 121 enlisted men. To maximize external validity, only men undergoing psychological evaluation were used as participants. Forty-one participants were identified as suspected malingerers through multiple criteria and were contrasted with schizophrenic-spectrum and clinic outpatient groups. To improve internal validity, the 41 suspected malingering participants were asked to retake the test without exaggerating. Results revealed that there were many false positives and fewer, but nonetheless many, false negatives with standard malingering indices. It appeared that the Gough Dissimulation scale (Gough, 1947) might hold the most promise as a measure of malingering, but other scales are also useful. Individual comparisons between different samples and implications for MMPI-2 (Butcher et al., 1989) are presented.

Journal Article↗

Measuring health status in psychiatric community surveys: internal and external validity of the Spanish version of the SF-36.

The aims of the present study were to determine the acceptability of the Spanish version of the Short Form (SF-36) health survey questionnaire for its use in mental health research on the general population, and to evaluate its internal and external validity, using data obtained from a representative community sample of 1250 adults of working age. The internal consistency of the different dimensions of the questionnaire was high. Concurrent validity was tested by examining its correlation with the 12-item General Health Questionnaire (GHQ-12). Construct validity was determined by studying the extent to which scores on different variables reflected the expected distribution of health status for and between certain groups. Our findings provide strong evidence for the acceptability and validity of the SF-36 when used as part of a community mental health survey.

Adolescent↗

Psychosocial dimensions of learning disabilities: external validation and relationship with age and academic functioning.

This study was designed to assess the following in a group of 152 children with learning disabilities between the ages of 7 and 13 years: (a) the relationships between age and psychosocial functioning: (b) the relationships among psychosocial functioning, cognitive abilities, and academic achievement; and (c) the external validity of statistically derived psychosocial subtypes. Participants were assigned to one of seven psychosocial subtypes on the basis of a profile-matching algorithm. Overall, the findings suggested no increase in psychopathology with advancing age. In addition, clear relationships were found between academic achievement patterns and personality subtypes. Finally, the subtypes could be distinguished on the basis of a behavior problem checklist not used for the construction of the subtypes.

Achievement↗