Search PubMed⌕ Search

Biomedical subjects

H Mulholland

Publications and source records attributed to H Mulholland.

At least 19 recordsLinked to original sources
PubMed2001

Setting performance standards for medical practice: a theoretical framework.

BACKGROUND: The assessment of performance in the real world of medical practice is now widely accepted as the goal of assessment at the postgraduate level. This is largely a validity issue, as it is recognised that tests of knowledge and in clinical simulations cannot on their own really measure how medical practitioners function in the broader health care system. However, the development of standards for performance-based assessment is not as well understood as in competency assessment, where simulations can more readily reflect narrower issues of knowledge and skills. This paper proposes a theoretical framework for the development of standards that reflect the more complex world in which experienced medical practitioners work. METHODS: The paper reflects the combined experiences of a group of education researchers and the results of literature searches that included identifying current health system data sources that might contribute information to the measurement of standards. CONCLUSION: Standards that reflect the complexity of medical practice may best be developed through an "expert systems" analysis of clinical conditions for which desired health care outcomes reflect the contribution of several health professionals within a complex, three-dimensional, contextual model. Examples of the model are provided, but further work is needed to test validity and measurability.

Clinical Competence↗
PubMed1994

Consultation competence in general practice: testing the reliability of the Leicester assessment package.

BACKGROUND: An acceptable assessment must be both valid and reliable; the face validity of the Leicester assessment package has already been established. AIM: This study set out to test the reliability of the Leicester assessment package, and the factors influencing it, when used by multiple assessors to assess performance in general practice consultations. METHOD: Six randomly selected course organizer assessors simultaneously used the package to conduct independent assessments of the performance of five doctors of widely varying abilities in consultation with six simulated patients. The scores allocated were subjected to generalizability analysis. RESULTS: The mean scores allocated for consultation performance of individual doctors ranged from 51% to 70%, with the lower scores being allocated to the less experienced doctors. Scores of each assessor across the cases were examined for internal consistency and five of the six assessors consistently scored the doctors with an alpha coefficient of the minimum accepted level of 0.80 or greater. The other assessor had a consistency of only 0.22. Measurements of consistency within cases between markers indicated that the first case produced unreliable results (alpha coefficient 0.25) but all other cases were scored consistently. Two independent assessors scoring eight consultations are the requisite numbers to achieve acceptable levels of reliability in a formal assessment process; seven consultations produce the minimum acceptable generalizability coefficient of 0.80 plus the first 'non-counting' consultation. CONCLUSION: Required levels of reliability can be achieved when the package is used by multiple markers assessing the same consultations over a wide range of consultation performance. To achieve reliability only two hours of assessment time are required using the Leicester package compared with the previously regarded minimum of 32 hours. Although assessors can produce reliable scores with minimal training, intra-assessor reliability cannot be taken for granted and all assessors should be trained and calibrated before being sanctioned to conduct assessments, particularly for regulatory purposes. The Leicester assessment package has now been shown to be valid, reliable, feasible and easy to use in practice. It can, therefore, be recommended for use in both formative and summative assessment of consultation competence in general practice.

Communication↗
PubMed1994

Consultation competence in general practice: establishing the face validity of prioritized criteria in the Leicester assessment package.

AIM: This study set out to test the face validity of prioritized criteria of consultation competence in general practice as contained in the Leicester assessment package. METHOD: A questionnaire was sent to a geographically stratified random sample of 100 members of the United Kingdom Association of Course Organisers to seek their views on the categories, components and weightings contained in the Leicester assessment package and to determine the proportion of respondents who rejected or suggested a new category, component or weighting or reallocated components to other categories or amended weightings. Their views were sought on a six-point scale (strongly approve, approve, tend to approve, tend to disapprove, disapprove and strongly disapprove). RESULTS: There was a 73% response rate. Of the respondents 99% either strongly approved or approved of the overall set of categories of consultation competence. Only two respondents (3%) expressed any disapproval of individual categories. Thirty five of the 39 suggested components of consultation competence were supported by more than 80% of respondents. There was minimal support for excluding any categories or components of consultation competence, for moving any components to different categories or for the inclusion of new categories or components. Eighty eight per cent of respondents were in favour of the need to identify priorities between any agreed categories of consultation competence and 79% expressed approval of the suggested weightings. Although 42% of respondents indicated a wish for some alteration in weightings, the mean values for all consultation categories suggested by all respondents were almost identical to the original weightings in the Leicester package. CONCLUSION: The face validity of the categories and components of consultation competence contained in the Leicester assessment package has been established, and the suggested weightings of consultation categories have been validated. Consequently, the criteria contained in the Leicester package can be adopted with confidence as measures against which performance can be judged in formative or summative assessment of consultation performance in general practice.

Clinical Competence↗
PubMed1993

An instrument for assessment of videotapes of general practitioners' performance.

OBJECTIVES: To identify those important characteristics of doctors' and patients' behaviour that distinguish between "good" and "bad" consultations when viewed on videotape; to use these characteristics to develop a reliable instrument for assessing general practitioners' performance in their own consultations. DESIGN: Questionnaires completed by patients, general practitioner trainers, and general practitioner trainees. Reliability of draft instrument tested by general practitioner trainers. SETTING: All vocational training schemes for general practice in the Northern region of England. SUBJECTS: First stage: 76 patients in seven groups, 108 general practice trainers in 12 groups, and 122 general practice trainees in 10 groups. Second stage: 85 general practice trainers in 12 groups. MAIN OUTCOME MEASURES: Trainers' ratings of importance; alpha coefficients of draft instrument by trainee, group, and consultation. RESULTS: 6890 characteristics of good and bad consultations were consolidated into a draft assessment instrument consisting of 46 pairs of definitions separated by six point bipolar scales. Nine statement pairs given low importance ratings by trainers were eliminated, reducing the instrument to 37 statement pairs. To test reliability, general practitioner trainers used the instrument to assess three consultations. With the exception of one group of trainers, all alpha coefficients exceeded the acceptable level of 0.80. CONCLUSION: The instrument produced is reliable for assessing general practitioners' performance in their own consultations.

Adolescent↗
PubMed1991

Approaches to learning of students in an Indonesian medical school.

This paper reports on a study of approaches to learning of undergraduate medical students in the University of Gadjah Mada, Indonesia. The Lancaster Approaches to Studying Inventory was translated into Indonesian and the translated form pilot tested. The instrument was then completed by 90 students, 30 each in first, second and fifth year in the Faculty of Medicine. It was found that Indonesian students generally gave higher rates than previous studies have reported. Factor analysis of their responses showed strong resemblances to other groups in the factors of meaning and reproducing orientation. Differences found in Indonesian students' responses were in strategic and non-academic orientations. There were differences between the response patterns of first-, third- and fifth-year students. It is concluded that the instrument is valid for use in Indonesia.

Curriculum↗
PubMed1990

Assessment of the general practitioner.

There is considerable interest in general practice assessment both as a means of monitoring vocational training and as a form of self audit for the general practitioner. Basic principles involved in the selection and design of assessment procedures are reviewed and their application discussed.

Clinical Competence↗