Search PubMed⌕ Search

Biomedical subjects

B Jolly

Publications and source records attributed to B Jolly.

At least 19 recordsLinked to original sources
PubMed2005

Psychometric characteristics of simulation-based assessment in anaesthesia and accuracy of self-assessed scores.

The purpose of this study was to define the psychometric properties of a simulation-based assessment of anaesthetists. Twenty-one anaesthetic trainees took part in three highly standardised simulations of anaesthetic emergencies. Scenarios were videotaped and rated independently by four judges. Trainees also assessed their own performance in the simulations. Results were analysed using generalisability theory to determine the influence of subject, case and judge on the variance in judges' scores and to determine the number of cases and judges required to produce a reliable result. Self-assessed scores were compared to the mean score of the judges. The results suggest that 12-15 cases are required to rank trainees reliably on their ability to manage simulated crises. Greater reliability is gained by increasing the number of cases than by increasing the number of judges. There was modest but significant correlation between self-assessed scores and external assessors' scores (rho = 0.321; p = 0.01). At the lower levels of performance, trainees consistently overrated their performance compared to those performing at higher levels (p = 0.0001).

Anesthesiology↗
PubMed2001

Does observation add to the validity of the long case?

BACKGROUND: A London medical school final MBBS examination for 155 candidates. OBJECTIVE: To investigate whether observing the student-patient interaction in a history taking (HT) long case adds incremental information to the traditional presentation component. DESIGN: A prospective study of a HT long case which included both examiner observation of the student-patient interview (Part 1) and traditional presentation to different examiners (Part 2). Checklist and global ratings of both parts were compared. Examiners were paired to estimate inter-rater reliability. The students also took a 20 station Objective Structured Clinical Examination (OSCE). OUTCOME MEASURES: Correlation of (I) examiner ratings for observation and presentation of the HT long case (II) examiner pair ratings and (III) stepwise regression analysis of scores for the HT long case with OSCE scores. RESULTS: Seventy-five (48.4%) candidates had two examiner pairs marking their case history. Observation and presentation scores correlated poorly (checklist 0.38 and global 0.33). Checklist and global scores for each part correlated at higher levels (observation 0.64 and presentation 0.61). Inter-rater reliability correlations were higher for observation (checklist 0.72 and global 0.71) than for presentation (checklist 0.38 and global 0.60). When HT long case scores were correlated with OSCE scores, using stepwise regression, global presentation scores showed the highest correlation with the OSCE score (0.36) and the global observation score contributed a further 12% to the correlation (0.50). CONCLUSION: Observation of history taking in a long case appears to measure a useful and distinct component of clinical competence over and above the contribution made by the presentation.

Clinical Competence↗
PubMed2001

Sheffield Assessment Instrument for Letters (SAIL): performance assessment using outpatient letters.

CONTEXT: Well-designed assessments of performance are urgently required for training doctors, and to provide indicators of the quality of practice. OBJECTIVES: To design an assessment process that uses routine outpatient letters, and to evaluate its validity, feasibility, reliability (reproducibility and discrimination) and potential educational impact. PARTICIPANTS: All 26 paediatric registrars in North Trent attending annual assessment panel in 1999 participated. STUDY DESIGN: An assessment instrument (SAIL) was developed from a consensus framework for good practice in written communication. It comprises an 18-point checklist and a global rating scale. Three judges applied the instrument to 260 letters from the routine clinical practice of the 26 participants. Results We achieved consensus on good practice in written communication. This was in keeping with the published literature. All participants completed the assessment. Scoring took 3-6 min per judge per letter. The reliability coefficient in this test situation is 0.72. Modelling predicts that a coefficient of 0.8 (the threshold for high-stakes judgements about performance) can be achieved with more judges or letters. The assessment results are well suited to formative feedback. CONCLUSIONS: SAIL uses letters as a face valid indicator of written communication performance. The instrument is feasible to use, and produces reliable results when applied to paediatric registrars to inform the annual Record of In-Training Assessment (RITA). Feedback from the assessment should help doctors to improve their written communication. Its use may extend to other specialities and settings including revalidation.

Communication↗
PubMed2001

The General Medical Council's Performance Procedures: peer review of performance in the workplace.

The General Medical Council procedures to assess the performance of doctors who may be seriously deficient include peer review of the doctor's practice at the workplace and tests of competence and skills. Peer reviews are conducted by three trained assessors, two from the same speciality as the doctor being assessed, with one lay assessor. The doctor completes a portfolio to describe his/her training, experience, the circumstances of practice and self rate his/her competence and familiarity in dealing with the common problems of his/her own discipline. The assessment includes a review of the doctor's medical records; discussion of cases selected from these records; observation of consultations for clinicians, or of relevant activities in non-clinicians; a tour of the doctor's workplace; interviews with at least 12 third parties (five nominated by the doctor); and structured interviews with the doctor. The content and structure of the peer review are designed to assess the doctor against the standards defined in Good Medical Practice, as applied to the doctor's speciality. The assessment methods are based on validated instruments and gather 700-1000 judgements on each doctor. Early experience of the peer review visits has confirmed their feasibility and effectiveness.

Clinical Competence↗
PubMed2001

The assessment of poorly performing doctors: the development of the assessment programmes for the General Medical Council's Performance Procedures.

BACKGROUND: Modernization of medical regulation has included the introduction of the Professional Performance Procedures by the UK General Medical Council in 1995. The Council now has the power to assess any registered practitioner whose performance may be seriously deficient, thus calling registration (licensure) into question. Problems arising from ill health or conduct are dealt with under separate programmes. METHODS: This paper describes the development of the assessment programmes within the overall policy framework determined by the Council. Peer review of performance in the workplace (Phase 1) is followed by tests of competence (Phase 2) to reflect the relationship between clinical competence and performance. The theoretical and research basis for the approach are presented, and the relationship between the qualitative methods in Phase 1 and the quantitative methods in Phase 2 explored. CONCLUSIONS: The approach is feasible, has been implemented and has stood legal challenge. The assessors judge and report all the evidence they collect and may not select from it. All their judgements are included and the voice of the lay assessor is preserved. Taken together, the output from both phases forms an important basis for remediation and training should it be required.

Clinical Competence↗
PubMed2001

The General Medical Council's Performance Procedures: the development and implementation of tests of competence with examples from general practice.

OBJECTIVE: This paper describes the development of the tests of competence used as part of the General Medical Council's assessment of potentially seriously deficient doctors. It is illustrated by reference to tests of knowledge and clinical and practical skills created for general practice. SUBJECTS AND TESTS: A notional sample of 30 volunteers in 'good standing' in the specialty (reference group), 27 practitioners referred to the procedures and four practitioners not referred but who were the focus of concern over their performance. Tests were constructed using available guidelines and a specially convened working group in the specialty. METHODS: Standards were set using Angoff, modified contrasting group and global judgement methods, as appropriate. RESULTS: Tests performed highly reliably, showed evidence of construct validity, intercorrelated at appropriate levels and, at the standards employed, demonstrated good separation of reference and referred groups. Likelihood ratios for above and below standard performance based on competence were large for each test. Seven of 27 doctors referred were shown not to be deficient in both phases of the performance assessment.

Adult↗
PubMed2001

Training the assessors for the General Medical Council's Performance Procedures.

From July 1997, the General Medical Council (GMC) has had the power to investigate doctors whose performance is considered to be seriously deficient. Assessment procedures have been developed for all medical specialties to include peer review of performance in practice and tests of competence. Peer review is conducted by teams of at least two medical assessors and one lay assessor. A comprehensive training programme for assessors has been developed that simulates the context of a typical practice-based assessment and has been tailored for 12 medical specialties. The training includes the principles of assessment, familiarization with the assessment instruments and supervised practice in assessment methods used during the peer review visit. High fidelity is achieved through the use of actors who simulate third party interviewees and trained doctors who role play the assessee. A subgroup of assessors, selected to lead the assessment teams, undergo training in handling group dynamics, report writing and in defending the assessment report against legal challenge. Debriefing of assessors following real assessments has been strongly positive with regard to their preparedness and confidence in undertaking the assessment.

Competency-Based Education↗
PubMed2000

Undergraduate projects - do they have to be within the conventional medical environment?

OBJECTIVES: Undergraduate medical curricula now include increasing amounts of project work aimed at developing skills related to lifelong learning. One course allows students to choose from a wide range of projects, including 'conventional' hospital specialties and also from topics outside the mainstream of medicine. DESIGN: 'Conventional' and 'external' projects were compared in terms of the prior academic abilities of the students undertaking them, the assessment results and student and supervisor feedback, in order to consider whether the unconventional projects were equally valid within the undergraduate medical curriculum. SETTING: School of Medicine, University of Leeds, UK. SUBJECTS: Medical students. RESULTS: No difference between the assessment results of the student groups was present, with over 85% of all students reaching a standard of 'excellent' or 'good' in their overall final grade. There was no difference in prior academic abilities between the student groups. Enjoyment of modules was comparable between student groups ('conventional' 89%, 'external' 93%) with good levels of satisfaction with the quality of supervision. There were no differences in students' self-appraisal of generic skill acquisition. Students who had undertaken 'external' projects felt they had gained less experience in data-handling and problem-solving skills. However, 'external' projects were rated higher by students in terms of having realistic and achievable objectives, and the supervisors of these projects were also more realistic about time commitments involved in project supervision. CONCLUSIONS: 'External' modules were very popular, with over 45% of students requesting places which were available for fewer than 20% of students per year. Concerns regarding the appropriateness of self-directed undergraduate medical student projects outside the mainstream of medical practice were unfounded.

Curriculum↗
PubMed2000

Benefiting from clinical experience: the influence of learning style and clinical experience on performance in an undergraduate objective structured clinical examination.

OBJECTIVE: To assess the relationship between clinical experience, learning style and performance in an objective structured clinical examination (OSCE) in medical students at the end of their first clinical year. DESIGN: Prospective study of undergraduate students taking an OSCE examination at the end of their first clinical year. SUBJECTS: 194 undergraduate medical students (95 male). MAIN OUTCOME MEASURES: Performance in the OSCE examination, the Entwhistle Learning Style Inventory1 and a composite self-reported score of clinical activity during the students first clinical year. RESULTS: Performance in the OSCE examination was related to well-organized study methods but not to clinical experience. A significant relationship between clinical experience and organized deep-learning styles suggests that knowledge gained from clinical experience is related to learning style. CONCLUSIONS: The relationship between clinical experience and student performance is complex. Well-organized and strategic learning styles appear to influence the benefits of increased clinical exposure. Further work is required to elucidate the most beneficial aspects of clinical teaching.

Adult↗
PubMed2000

Clinical teaching: maintaining an educational role for doctors in the new health care environment.

CONTEXT AND OBJECTIVES: Good clinical teaching is central to medical education but there is concern about maintaining this in contemporary, pressured health care environments. This paper aims to demonstrate that good clinical practice is at the heart of good clinical teaching. METHODS: Seven roles are used as a framework for analysing good clinical teaching. The roles are medical expert, communicator, collaborator, manager, advocate, scholar and professional. RESULTS: The analysis of clinical teaching and clinical practice demonstrates that they are closely linked. As experts, clinical teachers are involved in research, information retrieval and sharing of knowledge or teaching. Good communication with trainees, patients and colleagues defines teaching excellence. Clinicians can 'teach' collaboration by acting as role models and by encouraging learners to understand the responsibilities of other health professionals. As managers, clinicians can apply their skills to the effective management of learning resources. Similarly skills as advocates at the individual, community and population level can be passed on in educational encounters. The clinicians' responsibilities as scholars are most readily applied to teaching activities. Clinicians have clear roles in taking scholarly approaches to their practice and demonstrating them to others. CONCLUSION: Good clinical teaching is concerned with providing role models for good practice, making good practice visible and explaining it to trainees. This is the very basis of clinicians as professionals, the seventh role, and should be the foundation for the further development of clinicians as excellent clinical teachers.

Australia↗