Search PubMed⌕ Search

Biomedical subjects

John J Norcini

Publications and source records attributed to John J Norcini.

17 recordsLinked to original sources
PubMed2006

A comparison of the characteristics and examination performances of U.S. and non-U.S. citizen international medical graduates who sought Educational Commission for Foreign Medical Graduates certification: 1995-2004.

BACKGROUND: International medical graduates (IMGs) make up a substantial part of the U.S. physician workforce. Unfortunately, little is known about the demographic and performance characteristics of these doctors, especially over time. METHOD: Educational Commission for Foreign Medical Graduates (ECFMG) applicants from 1995-2004 were studied. The characteristics and examination performances of U.S. citizen and non-U.S. citizen IMGs were contrasted. RESULTS: ECFMG applications from U.S. citizen IMGs have been increasing. Compared to non-U.S. citizen IMGs, these individuals perform less well on some of the certification examinations, especially in the last 5 years, but are more likely to eventually achieve certification. Based on those medical students/graduates who took the examinations, the performance of U.S. citizen IMGs varied considerably from one medical school to another. CONCLUSION: The composition of the pool of ECFMG-certified IMGs available for graduate medical education has changed over the past 10 years. Shifts in the characteristics and abilities of IMGs are likely to have some future impact on health care delivery in the United States.

Certification↗
PubMed2006

Educational commission for foreign medical graduates certification and specialty board certification among graduates of the Caribbean medical schools.

BACKGROUND: To determine the variability among Caribbean medical schools in the performance of their students against markers of quality. METHOD: The Educational Commission for Foreign Medical Graduates (ECFMG) database and the American Medical Association (AMA) Masterfile were linked. Analyses were restricted to ECFMG applicants, from 1980 to 2000, who attended the 12 Caribbean medical schools that produced 200 or more U.S. International Medical School Graduates (USIMGs). RESULTS: The schools produced 17,049 USIMG (48% of all USIMGs) and 5,840 non-USIMG (5% of all non-USIMGs) applicants for ECFMG certification. By school, the ECFMG certification rate ranged from 28% to 86% for USIMGs and from 27% to 82% for non-USIMGs. Specialty Board certification for graduates of these schools also varied widely. CONCLUSIONS: Medical schools in the Caribbean play an increasingly important role in supplying the U.S. workforce. There is considerable variability among them, both in the numbers of physicians they produce and the performance of their students against indicators of quality such as ECFMG and specialty Board certification.

Certification↗
PubMed2005

Physician migration, education, and health care.

Physician migration is a complex and multifaceted phenomenon that is intimately intertwined with medical education. Imbalances in the production of physicians lead to workforce shortages and surpluses that compromise the ability to deliver adequate and equitable health care to large parts of the world's population. In this overview, we address a special section of the journal and briefly describe the current state of affairs in physician migration and its effects on donor and recipient countries. Global forms of practice and education, based on initiatives such as telemedicine and the International Virtual Medical School, hold promise of redressing some of these imbalances without requiring physicians to leave their countries. Other initiatives, such as those to raise the standards of medical education across the continuum, will improve the quality of care locally and permit the movement of physicians, resulting in better distribution of the workforce. All of these actions require a uniquely global perspective that places a premium on the long-term benefits to humankind while respecting the rights of patients and physicians.

Education, Medical↗
PubMed2005

Scoring standardized patient examinations: lessons learned from the development and administration of the ECFMG Clinical Skills Assessment (CSA).

Throughout the 40 year history of standardized patient assessments and OSCEs, there have been numerous advancements, including many that involve scoring the simulated clinical encounters. While there is no clear agreement on how examinees' performance should be documented or scored in an encounter, there is a consensus that several well-chosen SP encounters are required to produce reliable examinee scores. There also continues to be some debate as to who should do the scoring on an SP-based assessment. While logistics and cost will certainly play a role, it is probably best to use the person who is most familiar with the domain being assessed. In some instances this will be the SP; in others, an outside observer or content expert. Finally, with the growing use of OSCEs for summative purposes (e.g. certification, licensure), special attention must be paid to fairness issues. Since the same test form cannot be used day after day, examinee scores must be 'equated', taking into account the psychometric properties of scores from individual cases and individual SPs. To date, the CSA has been one of the highest-volume, high-stakes, standardized patient assessments to be developed and successfully administered. In 2003 alone, over 11 500 IMGs were tested. The early conceptual framework for this assessment was synthesized from the research endeavours of several notable individuals, including, amongst many others, Harden et al. 1975, Swanson & Stillman, 1990, Newble & Swanson, 1988, Vu et al. 1992 and Colliver, 1995. The early prototype administrations of the CSA, including many operational research studies, were supported and guided by Dr Friedman Ben-David, Friedman et al. 1991, 1993, Stillman et al. 1992, and Sutnick et al. 1993, 1995.

Clinical Competence↗
PubMed2005

Specialty board certification among U.S. citizen and non-U.S. citizen graduates of international medical schools.

BACKGROUND: Graduates of international medical schools (IMGs) make up approximately one-quarter of the physician population and play a key role in the provision of health care in the United States. This study investigated whether they differ from U.S. medical graduates (USMGs) in specialty board certification. METHOD: The study compared USMGs, U.S. citizen IMGs (USIMGs), and non-U.S. citizen IMGs (non-USIMGs) who graduated from medical school between 1958 and 1994 and were involved in direct patient care in 2003. RESULTS: There is variability among the specialties, but overall USMGs have the highest specialty certification rates followed by non-USIMGs, and USIMGs. Among recent medical school graduates, non-USIMGs have certification rates that are comparable to USMGs. CONCLUSIONS: IMGs have lower board-certification rates than USMGs, although a sizeable majority has achieved board certification in the specialty they practice. There are differences between non-USIMGs and USIMGs, with the former more likely to become board certified.

Certification↗
PubMed2005

Using a standardised patient assessment to measure professional attributes.

INTRODUCTION: Professionalism is an important topic in medical education today. While much work has focused on defining professionalism and teaching medical students the appropriate interpersonal behaviours, relatively little research has looked at meaningful ways of assessing the relevant attributes. METHOD: The Educational Commission for Foreign Medical Graduates (ECFMG) clinical skills assessment (CSA) uses standardised patients (SPs) to evaluate the readiness of graduates of international medical schools to enter accredited graduate training programmes in the USA. Doctor interpersonal skills, including professional qualities such as rapport, are evaluated as part of the CSA. Attentiveness, attitude and empathy, all facets of professional behaviour, are specifically targeted as part of the assessment. RESULTS: To date, over 35 000 candidates have been assessed, encompassing more than 370 000 individual SP encounters. Based on a 1-year cohort of examinees, the reliability of the individual professionalism-related component scores ranged from 0.61 to 0.70. Doctors who had graduated from medical school more recently, or were younger, generally obtained higher ratings. Professional qualities were only marginally related to measures of basic science and clinical science proficiency. Female candidates were rated significantly higher than male candidates on all dimensions. CONCLUSIONS: While some professional behaviours are probably best measured using formats such as surveys, self-assessment and critical incident techniques, certain aspects of the domain can be reliably and validly measured in SP examinations.

Attitude of Health Personnel↗
PubMed2005

Current perspectives in assessment: the assessment of performance at work.

BACKGROUND: Traditional assessment has improved significantly over the past 50 years. A number of new testing methods are now in place, the computer is improving both the fidelity and efficiency of examinations, and the psychometric principles on which assessment rests are more sophisticated than ever. AIM: There is growing interest in quality improvement and there are increasing demands for public accountability. This has shifted the focus of testing from education to work. The purpose of this paper is to describe the assessment of work. DISCUSSION: In contrast to traditional assessment, there are no 'methods' for the evaluation of work because the content and difficulty of the examination are not controlled in any fashion. Instead it is a matter of identifying the basis for the judgements (outcomes, process, or volume), deciding how the data will be gathered (practice records, administrative databases, diaries/logs, or observation), and avoiding threats to validity and reliability (patient mix, patient complexity, attribution, and numbers of patients). FUTURE DIRECTIONS: Overall, the assessment of doctors' performance at work is in its infancy and much research and development is needed. Nonetheless, it is being used increasingly in programmes of continuous quality improvement and accountability. It is critical that refinements occur quickly to ensure that patients receive the highest quality of care and that doctors are treated fairly and provided with the information they need to guide their professional development.

Clinical Competence↗
PubMed2003

The mini-CEX: a method for assessing clinical skills.

OBJECTIVE: To evaluate the mini-clinical evaluation exercise (mini-CEX), which assesses the clinical skills of residents. DESIGN: Observational study and psychometric assessment of the mini-CEX. SETTING: 21 internal medicine training programs. PARTICIPANTS: Data from 1228 mini-CEX encounters involving 421 residents and 316 evaluators. INTERVENTION: The encounters were assessed for the type of visit, sex and complexity of the patient, when the encounter occurred, length of the encounter, ratings provided, and the satisfaction of the examiners. Using this information, we determined the overall average ratings for residents in all categories, the reliability of the mini-CEX scores, and the effects of the characteristics of the patients and encounters. MEASUREMENTS: Interviewing skills, physical examination, professionalism, clinical judgment, counseling, organization and efficiency, and overall competence were evaluated. RESULTS: Residents were assessed in various clinical settings with a diverse set of patient problems. Residents received the lowest ratings in the physical examination and the highest ratings in professionalism. Comparisons over the first year of training showed statistically significant improvement in all aspects of competence, and the method generated reliable ratings. CONCLUSIONS: The measurement characteristics of the mini-CEX are similar to those of other performance assessments, such as standardized patients. Unlike these assessments, the difficulty of the examination will vary with the patients that a resident encounters. This effect is mitigated to a degree by the examiners, who slightly overcompensate for patient difficulty, and by the fact that each resident interacts with several patients. Furthermore, the mini-CEX has higher fidelity than these formats, permits evaluation based on a much broader set of clinical settings and patient problems, and is administered on site.

Clinical Competence↗
PubMed2003

Setting standards on educational tests.

OBJECTIVE: This instalment in the series on professional assessment provides an introduction to methods of setting standards. METHOD: A standard is a special score that serves as a boundary between those who perform well enough and those who do not. The practical steps in selecting it include: deciding on the type of standard; deciding the method for setting it; selecting the judges; holding the meeting; calculating the cutpoint, and deciding what to do afterwards. Four of the more popular methods are illustrated for both written and clinical examinations. RESULTS: The most important criteria for selecting a method for setting standards are whether it is consistent with the purpose of the test, based on expert judgement, informed by data, supported by research, transparent, and requires due diligence. The credibility of the standard will rely largely on the nature of the standard setters and the selection of a broadly representative and knowledgeable group is essential. After the standard has been set, it is important to ensure that stakeholders view the results as credible and that the pass rates have sensible relationships with other markers of competence. CONCLUSIONS: A standard is an expression of professional values in the context of a test's purpose and content, the ability of the examinees, and the wider social or educational setting. Because standards are an expression of values, methods for setting them are systematic ways of gathering value judgements, reaching consensus and expressing that consensus as a single score on a test.

Clinical Competence↗
PubMed2003

Peer assessment of competence.

OBJECTIVE: This instalment in the series on professional assessment summarises how peers are used in the evaluation process and whether their judgements are reliable and valid. METHOD: The nature of the judgements peers can make, the aspects of competence they can assess and the factors limiting the quality of the results are described with reference to the literature. The steps in implementation are also provided. RESULTS: Peers are asked to make judgements about structured tasks or to provide their global impressions of colleagues. Judgements are gathered on whether certain actions were performed, the quality of those actions and/or their suitability for a particular purpose. Peers are used to assess virtually all aspects of professional competence, including technical and non-technical aspects of proficiency. Factors influencing the quality of those assessments are reliability, relationships, stakes and equivalence. CONCLUSION: Given the broad range of ways peer evaluators can be used and the sizeable number of competencies they can be asked to judge, generalisations are difficult to derive and this form of assessment can be good or bad depending on how it is carried out.

Clinical Competence↗
PubMed2002

Assessing the comparability of standardized patient and physician evaluations of clinical skills.

Accumulating evidence to defend decisions based on scores from evaluations is an ongoing process. The purpose of this investigation was to gather additional data to support the validity of inferences made from scores on the Educational Commission for Foreign Medical Graduates' Clinical Skills Assessment (CSA). This was accomplished by contrasting CSA candidate scores, and pass/fail decisions, with those obtained from the American Board of Internal Medicine's Mini-CEX (Clinical Evaluation Exercise). Data gathering performance based on the number of unweighted history taking and physical examination checklist items adequately predicted the global ratings provided by physician observers. CSA ratings of doctor-patient communication skills correlated with mini-CEX ratings of like constructs, indicating that physician observers, using mini-CEX rating scales, are able to make realistic assessments of interpersonal skills. These results provide evidence of the convergent validity of CSA scores.

Certification↗
PubMed2002

Certifying examination performance and patient outcomes following acute myocardial infarction.

OBJECTIVE: To establish whether successful certifying examination performances of doctors are associated with their patients' mortality and length of stay following acute myocardial infarction. DESIGN: Risk adjusted mortality and survivors' length of stay were compared for doctors who had satisfactorily completed training in internal medicine or cardiology and attempted the relevant examination. Specifically, the study investigated the joint effects of hospital location, availability of advanced cardiac care, doctors' specializations, certifying examination performances, year certification was first attempted and patient volume. SETTING AND PARTICIPANTS: Data on all acute myocardial infarctions in Pennsylvania for the calendar year 1993 were collected by the Pennsylvania Health Care Cost Containment Council. These data were combined with physician information from the database of the American Board of Internal Medicine. RESULTS: Holding all variables constant, successful examination performance (i.e. certification in internal medicine or cardiology) was associated with a 19% reduction in mortality. Decreased mortality was also correlated with treatment in hospitals located outwith either rural or urban settings and with management by a cardiologist. Shorter stays were not related to examination performance but were associated with treatment by high volume cardiologists who had recently finished training and who cared for their patients in hospitals located outwith rural or urban settings. CONCLUSIONS: The results of the study add to the evidence supporting the validity of the certifying examination and lend support to the concept that fund of knowledge is related to quality of practice.

Cardiology↗
PubMedDate not supplied

The international medical graduate pipeline.

This paper presents information from the Educational Commission for Foreign Medical Graduates (ECFMG) on the certification and practice of international medical graduates (IMGs). IMGs constitute 25 percent of the U.S. physician workforce, and there is a vigorous pipeline of highly qualified candidates seeking graduate training in the United States. When considering how to address the potential physician shortfall described by Richard Cooper, policymakers will need to consider U.S. health care needs in the context of the intense international debate regarding global physician migration and its implications for the developing world.

Certification↗
PubMedDate not supplied

The international medical graduate pipeline: recent trends in certification and residency training.

International medical graduates (IMGs) represent a large proportion of the population entering graduate medical education (GME) programs. Many of these internationally trained physicians go on to practice medicine in the United States. To be eligible for GME, IMGs must be certified by the Educational Commission for Foreign Medical Graduates (ECFMG). The number of certificates issued by the ECFMG has varied over time and historically has exceeded the number of available training positions. More detailed longitudinal analyses are required to better understand the interwoven issues of physician supply, consumers' needs, and the role of IMGs in the U.S. health care system.

Certification↗