Work based assessment.
Explore the source record for details and available documents.
Clinical Competence↗
Biomedical subjects
John J Norcini
Publications and source records attributed to John J Norcini.
The mini-CEX: a method for assessing clinical skills.
OBJECTIVE: To evaluate the mini-clinical evaluation exercise (mini-CEX), which assesses the clinical skills of residents. DESIGN: Observational study and psychometric assessment of the mini-CEX. SETTING: 21 internal medicine training programs. PARTICIPANTS: Data from 1228 mini-CEX encounters involving 421 residents and 316 evaluators. INTERVENTION: The encounters were assessed for the type of visit, sex and complexity of the patient, when the encounter occurred, length of the encounter, ratings provided, and the satisfaction of the examiners. Using this information, we determined the overall average ratings for residents in all categories, the reliability of the mini-CEX scores, and the effects of the characteristics of the patients and encounters. MEASUREMENTS: Interviewing skills, physical examination, professionalism, clinical judgment, counseling, organization and efficiency, and overall competence were evaluated. RESULTS: Residents were assessed in various clinical settings with a diverse set of patient problems. Residents received the lowest ratings in the physical examination and the highest ratings in professionalism. Comparisons over the first year of training showed statistically significant improvement in all aspects of competence, and the method generated reliable ratings. CONCLUSIONS: The measurement characteristics of the mini-CEX are similar to those of other performance assessments, such as standardized patients. Unlike these assessments, the difficulty of the examination will vary with the patients that a resident encounters. This effect is mitigated to a degree by the examiners, who slightly overcompensate for patient difficulty, and by the fact that each resident interacts with several patients. Furthermore, the mini-CEX has higher fidelity than these formats, permits evaluation based on a much broader set of clinical settings and patient problems, and is administered on site.
Clinical Competence↗
Setting standards on educational tests.
OBJECTIVE: This instalment in the series on professional assessment provides an introduction to methods of setting standards. METHOD: A standard is a special score that serves as a boundary between those who perform well enough and those who do not. The practical steps in selecting it include: deciding on the type of standard; deciding the method for setting it; selecting the judges; holding the meeting; calculating the cutpoint, and deciding what to do afterwards. Four of the more popular methods are illustrated for both written and clinical examinations. RESULTS: The most important criteria for selecting a method for setting standards are whether it is consistent with the purpose of the test, based on expert judgement, informed by data, supported by research, transparent, and requires due diligence. The credibility of the standard will rely largely on the nature of the standard setters and the selection of a broadly representative and knowledgeable group is essential. After the standard has been set, it is important to ensure that stakeholders view the results as credible and that the pass rates have sensible relationships with other markers of competence. CONCLUSIONS: A standard is an expression of professional values in the context of a test's purpose and content, the ability of the examinees, and the wider social or educational setting. Because standards are an expression of values, methods for setting them are systematic ways of gathering value judgements, reaching consensus and expressing that consensus as a single score on a test.
Clinical Competence↗
Peer assessment of competence.
OBJECTIVE: This instalment in the series on professional assessment summarises how peers are used in the evaluation process and whether their judgements are reliable and valid. METHOD: The nature of the judgements peers can make, the aspects of competence they can assess and the factors limiting the quality of the results are described with reference to the literature. The steps in implementation are also provided. RESULTS: Peers are asked to make judgements about structured tasks or to provide their global impressions of colleagues. Judgements are gathered on whether certain actions were performed, the quality of those actions and/or their suitability for a particular purpose. Peers are used to assess virtually all aspects of professional competence, including technical and non-technical aspects of proficiency. Factors influencing the quality of those assessments are reliability, relationships, stakes and equivalence. CONCLUSION: Given the broad range of ways peer evaluators can be used and the sizeable number of competencies they can be asked to judge, generalisations are difficult to derive and this form of assessment can be good or bad depending on how it is carried out.
Clinical Competence↗
The death of the long case?
Explore the source record for details and available documents.
Clinical Competence↗
Assessing the comparability of standardized patient and physician evaluations of clinical skills.
Accumulating evidence to defend decisions based on scores from evaluations is an ongoing process. The purpose of this investigation was to gather additional data to support the validity of inferences made from scores on the Educational Commission for Foreign Medical Graduates' Clinical Skills Assessment (CSA). This was accomplished by contrasting CSA candidate scores, and pass/fail decisions, with those obtained from the American Board of Internal Medicine's Mini-CEX (Clinical Evaluation Exercise). Data gathering performance based on the number of unweighted history taking and physical examination checklist items adequately predicted the global ratings provided by physician observers. CSA ratings of doctor-patient communication skills correlated with mini-CEX ratings of like constructs, indicating that physician observers, using mini-CEX rating scales, are able to make realistic assessments of interpersonal skills. These results provide evidence of the convergent validity of CSA scores.
Certification↗
Certifying examination performance and patient outcomes following acute myocardial infarction.
OBJECTIVE: To establish whether successful certifying examination performances of doctors are associated with their patients' mortality and length of stay following acute myocardial infarction. DESIGN: Risk adjusted mortality and survivors' length of stay were compared for doctors who had satisfactorily completed training in internal medicine or cardiology and attempted the relevant examination. Specifically, the study investigated the joint effects of hospital location, availability of advanced cardiac care, doctors' specializations, certifying examination performances, year certification was first attempted and patient volume. SETTING AND PARTICIPANTS: Data on all acute myocardial infarctions in Pennsylvania for the calendar year 1993 were collected by the Pennsylvania Health Care Cost Containment Council. These data were combined with physician information from the database of the American Board of Internal Medicine. RESULTS: Holding all variables constant, successful examination performance (i.e. certification in internal medicine or cardiology) was associated with a 19% reduction in mortality. Decreased mortality was also correlated with treatment in hospitals located outwith either rural or urban settings and with management by a cardiologist. Shorter stays were not related to examination performance but were associated with treatment by high volume cardiologists who had recently finished training and who cared for their patients in hospitals located outwith rural or urban settings. CONCLUSIONS: The results of the study add to the evidence supporting the validity of the certifying examination and lend support to the concept that fund of knowledge is related to quality of practice.
Cardiology↗
The international medical graduate pipeline.
This paper presents information from the Educational Commission for Foreign Medical Graduates (ECFMG) on the certification and practice of international medical graduates (IMGs). IMGs constitute 25 percent of the U.S. physician workforce, and there is a vigorous pipeline of highly qualified candidates seeking graduate training in the United States. When considering how to address the potential physician shortfall described by Richard Cooper, policymakers will need to consider U.S. health care needs in the context of the intense international debate regarding global physician migration and its implications for the developing world.
Certification↗