Biomedical subjects

Miriam Friedman Ben-David

Publications and source records attributed to Miriam Friedman Ben-David.

6 recordsLinked to original sources

PubMed2006

Setting school-level outcome standards.

BACKGROUND: To establish international standards for medical schools, an appropriate panel of experts must decide on performance standards. A pilot test of such standards was set in the context of a multidimensional (multiple-choice question examination, objective structured clinical examination, faculty observation) examination at 8 leading schools in China. METHODS: A group of 16 medical education leaders from a broad array of countries met over a 3-day period. These individuals considered competency domains, examination items, and the percentage of students who could fall below a cut-off score if the school was still to be considered as meeting competencies. This 2-step process started with a discussion of the borderline school and the relative difficulty of a borderline school in achieving acceptable standards in a given competency domain. Committee members then estimated the percentage of students falling below the standard that is tolerable at a borderline school and were allowed to revise their ratings after viewing pilot data. RESULTS: Tolerable failure rates ranged from 10% to 26% across competency domains and examination types. As with other standard-setting exercises, standard deviations from initial to final estimates of the tolerable failure rates fell, but the cut-off scores did not change significantly. Final, but not initial cut-off scores were correlated with student failure rates (r = 0.59, P = 0.03). DISCUSSION: This paper describes a method to set school-level outcome standards at an international level based on prior established standard-setting methods. Further refinement of this process and validation using other examinations in other countries will be needed to achieve accurate international standards.

China↗

PubMed2005

Ensuring global standards for medical graduates: a pilot study of international standard-setting.

Increasing physician and patient mobility has led to a move toward internationalization of standards for physician competence. The Institute for International Medical Education proposed a set of outcome-based standards for student performance, which were then measured using three assessment tools in eight leading schools in China: a 150-item multiple-choice examination, a 15-station OSCE and a 16-item faculty observation form. The purpose of this study was to empanel a group of experts to determine whether international student-level performance standards could be set. The IIME convened an international panel of experts in student education with specialty and geographic diversity. The group was split into two, with each sub-group establishing standards independently. After a discussion of the borderline student, the sub-groups established minimally acceptable cut-off scores for performance on the multiple-choice examination (Angoff and Hofstee methods), the OSCE station and global rating performance (modified Angoff method and holistic criterion reference), and faculty observation domains (holistic criterion reference). Panelists within each group set very similar standards for performance. In addition, the two independent parallel panels generated nearly identical performance standards. Cut-off scores changed little before and after being shown pilot data but standard deviations diminished. International experts agreed on a minimum set of competences for medical student performance. In addition, they were able to set consistent performance standards with multiple examination types. This provides an initial basis against which to compare physician performance internationally.

Clinical Competence↗

PubMed2005

Effects of basic clinical skills training on objective structured clinical examination performance.

OBJECTIVES: The aim of curriculum reform in medical education is to improve students' clinical and communication skills. However, there are contradicting results regarding the effectiveness of such reforms. METHODS: A study of internal medicine students was carried out using a static group design. The experimental group consisted of 77 students participating in 7 sessions of communication training, 7 sessions of skills-laboratory training and 7 sessions of bedside-teaching, each lasting 1.5 hours. The control group of 66 students from the traditional curriculum participated in equally as many sessions but was offered only bedside teaching. Students' cognitive and practical skills performance was assessed using Multiple Choice Question (MCQ) testing and an objective structured clinical examination (OSCE), delivered by examiners blind to group membership. RESULTS: The experimental group performed significantly better on the OSCE than did the control group (P < 0.01), whereas the groups did not differ on the MCQ test (P < 0.15). This indicates that specific training in communication and basic clinical skills enabled students to perform better in an OSCE, whereas its effects on knowledge did not differ from those of the traditional curriculum. CONCLUSION: Curriculum reform promoting communication and basic clinical skills are effective and lead to an improved performance in history taking and physical examination skills.

Adult↗

PubMed2004

Linking appraisal of PRHO professional competence of junior doctors to their education.

In the UK, new medical graduates are known as Pre-registration House Officers (PRHOs). Postgraduate Deans are responsible for the PRHO year and for the final certification of PRHOs to allow them to be fully registered by the GMC as medical practitioners. However, as much as appraisal of professional growth is central to PRHO training, they are in need of a robust assessment mechanism to detect, at an early stage, individuals with significant clinical and professional deficiencies. Documented, reliable and valid ongoing information on PRHO performance will provide information for early intervention and will establish 'hard' observable evidence, for certification decisions. Thus, an approach that links appraisal/assessment of professionalism and clinical skills to education is the way forward. This paper describes a new approach to appraisal/assessment of PRHOs, which is currently being piloted in a number of regions in Scotland. The conceptual paradigm was developed during the last three years as a proposal for a Scottish national PRHO reform. 'Grounded' qualitative studies were employed to explore trainees' and trainers' perceptions of the expected competences (outcomes) of PRHO performance for appraisal/assessment purpose. The GMC recommendations are reviewed in light of the study results. An assessment model emerged that links appraisal to education. PRHOs' cumulative performance is documented over one year of training resulting in diagnostic profiles that provide guidance for evaluation and training of PRHOs. Poor performers are flagged in the early stages of training, thus allowing early intervention. The feasibility and acceptance of the model by educators, the health system and PRHOs has yet to be established.

Education, Medical↗

PubMed2003

Life beyond OSCE.

Explore the source record for details and available documents.

Clinical Competence↗

PubMed1998

An Investigation of the Sources of Measurement Error in the Post-Encounter Written Scores from Standardized Patient Examinations.

Purpose. Post-encounter written exercises (e.g., patient notes) have been included in clinical skills assessments that use standardized patients. The purpose of this study was to estimate the generalizability of the scores from these written exercises when they are rated by various trained health professionals, including physicians.Method. The patient notes from a 10 station clinical skills examination involving 10 first year emergency medicine residents were analytically scored by four rater groups: three physicians, three nurses, three fourth year medical students, three billing clerks. Generalizability analyses were used to partition the various sources of error variance and derive reliability-like coefficients for each group of raters.Results. The generalizability analyses indicated that case-to-case variability was a major source of error variance in the patient note scores. The variance attributable to the rater or to the rater by examinee interaction was negligible. This finding was consistent across the four rater groups. Generalizability coefficients in excess of 0.80 were achieved for each of the four sets of raters. Physicians did, however, produce the most dependable scores.Conclusion. There is little advantage, from a reliability perspective, in using more than one trained physician, or other health professional who is adequately trained to score the patient note. Measurement error is introduced primarily by case sampling variability. This suggests that, if required, increases in the generalizability of the patient note scores can be made through the addition of cases, and not the addition of raters.

Journal Article↗