Internal medicine residents' skills at identification of HIV-risk behavior and HIV-related disease.
Explore the source record for details and available documents.
Alcohol Drinking↗
Biomedical subjects
J D Carline
Publications and source records attributed to J D Carline.
Use of peer ratings to evaluate physician performance.
OBJECTIVE: To assess the feasibility and measurement characteristics of ratings completed by professional associates to evaluate the performance of practicing physicians. DESIGN: The clinical performance of physicians was evaluated using written questionnaires mailed to professional associates (physicians and nurses). Physician-associates were randomly selected from lists provided by both the subjects and medical supervisors, and detailed information was collected concerning the professional and social relationships between the associate and the subject. Responses were analyzed to determine factors that affect ratings and measurement characteristics of peer ratings. SETTING AND PARTICIPANTS: Physician-subjects were selected from among practicing internists in New York, New Jersey, and Pennsylvania who received American Board of Internal Medicine certification 5 to 15 years previously. MAIN OUTCOME MEASURE: Physician performance as assessed by peers. RESULTS: Peer ratings are not biased substantially by the method of selection of the peers or the relationship between the rater and the subject. Factor analyses suggest a two-dimensional conceptualization of clinical skills: one factor represents cognitive and clinical management skills and the other factor represents humanistic qualities and management of psychosocial aspects of illness. Ratings from 11 peer physicians are needed to provide a reliable assessment in these two areas. CONCLUSIONS: These findings suggest that it is feasible to obtain assessments from professional associates of practicing physicians in areas such as clinical skills, humanistic qualities, and communication skills. Using a shorter version of the questionnaire used in this study, peer ratings provide a practical method to assess clinical performance in areas such as humanistic qualities and communication skills that are difficult to assess with other measures.
Clinical Competence↗
Ratings of the performances of practicing internists by hospital-based registered nurses.
PURPOSE: To examine the feasibility and reliability of ratings completed by hospital-based registered nurses of the humanistic qualities, communication skills, and selected aspects of the clinical skills of practicing internists. METHOD: In 1988-1989, registered nurses who worked in the same 175 hospitals as 232 internists with admitting privileges at these hospitals rated the internists' performances. The nurses were selected from medicine floors, specialty floors, and intensive care units and/or critical care units, using lists provided by head nurses. A total of 1,877 rating questionnaires with 13 performance categories were collected (with a mean of 8.01 nurses per internist). The ratings were analyzed to determine measurement characteristics and the relationships of the nurses' demographic characteristics to the ratings. In addition, for each of ten performance categories for 152 of the internists, the average rating each internist received from nurses was compared with the average rating each internist received from peer physicians. Statistical analysis used Pearson correlations, canonical correlations, factor analyses, Student's t-tests, analysis of variance, and stepwise multiple regression. Finally, the internists themselves, including physicians who were not actually rated by the nurses, were asked to complete a brief questionnaire that included questions about their opinions of the use of nurses' ratings. RESULTS: The nurses' ratings correlated moderately strongly with the peer physicians' ratings and had a common structure. However, the nurses' ratings were lower for several humanistic qualities, including respect, integrity, and responsibility, and their ratings were higher for medical knowledge and verbal communications. Across the 13 performance categories, approximately 10-15 ratings from nurses were needed to obtain a reliable assessment of an internist's humanistic qualities and communication skills. Many internists felt that nurses' ratings should be used equally with, or at least as a lesser contribution to, ratings by peer physicians of humanistic qualities and communication skills. CONCLUSION: Nurses' ratings appear to provide a feasible and reliable method of evaluating the internists' communication skills and humanistic qualities, when used in conjunction with ratings by peer physicians.
Adult↗
Changes over time in the knowledge base of practicing internists.
OBJECTIVE: To determine factors affecting the knowledge base of practicing internists. DESIGN: An 82-item multiple-choice examination with questions from the 1988 American Board of Internal Medicine (ABIM) certifying examination was used to assess the knowledge base of 289 internists. SETTING AND PARTICIPANTS: Participants were selected from among practicing internists in New York, New Jersey, and Pennsylvania who had received ABIM certification 5 to 15 years previously. RESULTS: significant inverse correlation (r = -.30) was found between examination scores and the number of years elapsed since certification. Knowledge declined sharply within 15 years of certification. In addition, procedure-oriented subspecialists (cardiologists and gastroenterologists) had lower scores than other internists in this examination of general medical knowledge. Multivariate analyses showed that independent variables that predicted test performance were initial ABIM certifying examination score, time elapsed since certification, subspecialty classification, medical school type, and residency type. CONCLUSIONS: These results support the recent decision for time-limited certification of internists and raise questions related to content and standard setting for recertification examinations.
Certification↗
Comparing physicians' specialty interests upon entering medical school with their eventual practice specialties.
The stability of specialty choices of 519 graduates of the University of Washington School of Medicine were studied by means of a mail survey in 1986. The graduates' actual practice specialties were compared with the one or more specialties each had indicated as possible career specialty choices when they entered medical school. Nearly 70% of the graduates' early choices remained stable, double the percentage found by other studies that based stability on an entering medical student's single choice of a career specialty. The findings suggest that students entering medical school have in mind a variety of acceptable specialties and later are likely to choose one of these as a career.
Career Choice↗
Improvement of oncology education at the University of Washington School of Medicine, 1984-1988.
In 1984, performance on the National Board of Medical Examiners (NBME) examination by medical students from the University of Washington School of Medicine was significantly lower in clinical oncology, pathology, surgery, and oncology-related subjects than was their performance in other subjects (p = .002) and inversely proportional to the degree of oncology-relatedness of the material examined. Moreover, their performance on oncology-related questions was significantly lower than the national average (p less than .001). The curriculum committee thus initiated a review of the oncology content in the curriculum of the school of medicine and implemented a two-year plan in 1984-1986 to improve the perceived deficiencies. During the year the two-year intervention was completed, reevaluation demonstrated evidence that substantial improvement had occurred in oncology performance by the students, relative to their performance in non-oncology subjects (p less than .05) and to the national average. Performance on the non-oncology items was unchanged. The authors conclude that oncology education and performance of the students was significantly improved within two years, after instituting the short-term plan to enhance cancer education in the medical school, the improvement in oncology did not occur at the expense of a reduction in performance in non-oncology subjects, and the improved performance in oncology by the students enhanced their overall performance on the NBME examination.
Academic Medical Centers↗
Academic, social and cultural factors influencing medical school grade performance.
Studies of medical student performance have focused on various factors, including premedical academics, maturity, familial background and support, and personal experiences with illness. Most studies have been conducted in countries with highly developed educational systems and similar cultural and social systems. It is not clear that these findings can be applied to developing countries, where the educational and cultural experiences may be very different, and where medical instruction is carried out in a non-native language. Information was obtained from a survey of 153 fifth- and sixth-year medical students at King Abdulaziz University in Saudi Arabia. The survey measured premedical educational, social and cultural experiences that might affect medical school performance. Men performed as well as women in the medical school despite heavy familial and social commitments. Women's performance seems to be more influenced by changes in living environment. Achievement in premedical years was correlated positively with grade performance in medical school. Competence in the high-school English courses was related to medical school performance. Interest in the study of medicine prior to medical school was not related to performance. Other motivations, such as social gains, financial benefits or family wish, were related to lower performance. Current interest in clinical medicine correlated negatively with performance. Students motivated by the presence of chronic ill health in their families performed significantly lower. Factors influencing medical school performance in developed countries had similar impact on medical students in a developing country. Social factors, unique to the country, also play a role in medical student performance.
Achievement↗
Predictive validity of certification by the American Board of Internal Medicine.
STUDY OBJECTIVE: To determine the predictive validity of the American Board of Internal Medicine (ABIM) certification process. DESIGN: Prospective measurement of the knowledge, skills, and attitudes of 185 ABIM-certified and 74 noncertified internists by a written examination; evaluation by professional associates; a patient questionnaire assessing satisfaction with care, physician's counseling role, and preventive care; and review of records of patients with common illnesses. SUBJECTS: Practicing internists who completed training or received ABIM certification 5 to 10 years previously. SETTING: Office-based practices in six western states. RESULTS OF DATA ANALYSIS: Physicians certified by the ABIM had significantly higher scores on the written examination than the noncertified physicians, and scores on our examination correlated highly with the ABIM certification examination (r = 0.73). Ratings of clinical skills by professional associates were significantly higher for certified internists and also correlated highly with ABIM examination scores (r = 0.53 to 0.59). Regression analysis showed that ABIM certification status was the major variable affecting performance on these measures of clinical competence. Results from other measures did not show many differences between certified and noncertified physicians in the care of patients with common illnesses, but modest differences in preventive care and a few differences in outcome favored the certified physicians. CONCLUSIONS: Comparison of findings from the written examination and the professional associate ratings with certification status and original ABIM certification examination scores shows predictive validity of ABIM certification. Further studies are needed to determine if certification status predicts important differences in the care of patients with complex illnesses.
Certification↗
Characteristics of ratings of physician competence by professional associates.
Although the evaluation of medical students and residents frequently includes the use of global ratings scales as measures of clinical competence, few studies have investigated the use of global ratings in evaluating the performance of practicing physicians, or the psychometric properties of these ratings. In this article, the characteristics of ratings of physician competence by professional associates are described, using practicing internists as the subjects for the study. Ratings of nine aspects of clinical competence for 210 internists were obtained from four physician associates with whom patient care had been shared. The results suggest that ratings of physicians' clinical competence by professional associates are both reliable and potentially useful in identifying physicians with low performance. Studies are needed to clarify further the psychometric properties of peer ratings, and to determine factors that affect these ratings.
Attitude of Health Personnel↗
The knowledge base of certified internists. Relationships to training, practice type, and other physician characteristics.
A written examination was used to assess the knowledge base of 183 practicing certified internists. Analyses of the examination scores showed that performance on the initial American Board of Internal Medicine certification examination taken 7.6 years previously was the major factor predicting current knowledge base. By developing regression models, the unique contribution of different variables to prediction of current examination scores was determined. Prior American Board of Internal Medicine certification examination performance accounted for 70.9% of the explained variance, and demographic and practice variables were responsible for 17.8%. Among the demographic and practice variables studied, community size and subspecialty practice were the only variables that contributed significantly to the regression equations. Examination scores were highest for certified internists practicing in smaller communities. General internists received higher scores than subspecialists. Although statistically significant, the apparent adverse influence of subspecialty practice and larger community size on examination performance was modest. Further study is needed to determine if longer periods in practice might produce different relationships between variables such as these and examination performance.
Certification↗
Basic science course grades as predictors of performance on the NBME Part I examination.
Explore the source record for details and available documents.
Anatomy↗
A survey of fourth-year medical students' decisions regarding family practice as a career.
With a decline in the proportion of medical students pursuing family practice careers, it is necessary to better understand the factors influencing their career decisions. The Fishbein model of attitude formation was used as a framework for developing a questionnaire to measure students' attitudes toward a career in family practice and the social support students perceived they had for that career. The questionnaire was administered to fourth-year-medical students a short time before residency match day. Both the attitude and social support scales had excellent psychometric characteristics. The usefulness of these scales in discriminating between career choices of family practice and other clinical careers was tested. The social support scale did not discriminate well, whereas the attitude scale discriminated quite well. The students who chose careers in family practice differed from students who chose other careers in 14 of 19 values about characteristics of family practice but differed from them in only two of 19 beliefs about family practice. The implications for advising medical students about careers in family practice are discussed.
Attitude↗
Early identification of future medical students using initial undergraduate grades.
This study assessed whether overall academic performance in undergraduate medical coursework can be predicted with reasonable accuracy by using grades from initial college-level courses rather than total premedical grade point averages (GPAs). Initial college grades from four areas, MCAT scores, and NBME I and II scores were recorded for students admitted to the University of Washington Medical School, for students admitted to other medical schools, and for students not admitted to any medical school. The results documented a high relationship between cumulative GPAs and initial grades, with differences found between those students admitted to medical school and those not admitted. The importance of this study is the documentation that little predictive utility is gained by waiting for overall college GPAs from medical school applicants. Initial GPAs are available 2 years earlier than overall GPAs and provide virtually the same information. Exploratory suggestions for medical school admission policies are made.
Achievement↗
Resident and faculty differences in student evaluations: implications for changes in a clerkship grading system.
Data were collected for 163 students completing a surgical clerkship including scores on patient write-ups, ward performance evaluations, oral examinations, and National Board of Medical Examiners (NBME) Surgery subscores. Oral examination scores and patient write-ups were rated from 74 (failing) to 100 (honors) by faculty members. The ward performance evaluation included ratings on nine components of ward performance from 1 (unsatisfactory) to 4 (superior) and an overall ward score of 74 (failing) to 100 (honors). Similarities and discrepancies in the way that evaluators viewed clerks were found. Weights given to ward component ratings in relationship to the overall ward score were equivalent for faculty members and residents. Residents rated clerks higher than faculty members on five ward components. Faculty ratings were poorly correlated with resident ratings. Faculty members and residents gave different ratings to the same students in contrast to the similarities in which faculty members and residents gave weight to the ward component ratings. Resident ratings were better than faculty ratings in predicting the NBME Surgery subscore. Although these results suggest that residents are better evaluators of a clerk's performance than are faculty members, other studies indicated the opposite. The elimination of participation of either faculty or residents in the grading of students is unwise. Periodic monitoring of evaluation practices is necessary to ensure fairness in grading procedures.
Clinical Clerkship↗
Predicting performance during clinical years from the new Medical College Admission Test.
The results of a predictive validity study of the new Medical College Admission Test (MCAT) using criteria from the clinical years of undergraduate medical education are presented and discussed. The criteria included course grades and faculty ratings of clerks in internal medicine, surgery, obstetrics and gynecology, pediatrics, and psychiatry; scores on a comprehensive test of clinical knowledge (including patient management type examinations); and scores on Part II of the examinations of the National Board of Medical Examiners (NBME). While the validity coefficients of the MCAT with the Part II examinations ranged from .03 to .47, they were higher than those of undergraduate grade-point averages with the same criteria. The implications of the small-to-medium size validity coefficients for admissions are discussed.
Achievement↗
Comparison of course evaluations by random and volunteer student samples.
Ratings of the effectiveness of courses are usually based on students' voluntary responses to evaluation questionnaires. It is a rare course for which all students complete evaluation forms. This self-selection may result in an overrepresentation of students who are highly pleased or greatly disgruntled, thus biasing the evaluation. Responses of an unbiased random sample of students were compared with the responses of a self-selected sample for the evaluations of two medical school courses. There was no evidence of bias introduced by self-selection for either mean ratings or the variance of responses on a question-by-question basis for eight questions included in the evaluations of both classes. When responses were investigated across items, the patterns of differences between the random and voluntary samples appeared to be unique to each class.
Attitude of Health Personnel↗
Career preferences of first- and second-year medical students: the WAMI experience.
Students admitted directly to the Seattle campus of the University of Washington School of Medicine are predominately from metropolitan areas. Students completing their first year at affiliated programs at Washington State University, University of Alaska, Montana State University, and the University of Idaho are predominately from nonmetropolitan areas. The authors compared these two groups of students with respect to preferred specialties and practice location; in addition, these preferences were examined on the basis of the size of students' home communities. Factor analysis of the preferences indicated by three entering classes at the end of their second year identified interpretable clusters of specialties and location types. Few differences were found in preferences for specialties, either between the groups or based on size of home community. Differences in attitudes toward location of practice indicated that students favor commmunities similar to those in which they were raised. The preferences indicated by second-year students were found to be significantly related to those expressed by the same students as they entered medical school, suggesting stability of student attitudes toward career choices. These results indicate that increased admissions from nonmetropolitan areas may affect the distribution of physicians by region but not by specialty.
Attitude of Health Personnel↗
Factors affecting the reliability of ratings of students' clinical skills in a medicine clerkship.
OBJECTIVE: To determine the overall reliability and factors that might affect the reliability of ratings of students' clinical skills in a medicine clerkship. DESIGN: A nine-item instrument was used to evaluate students' clinical skills. Raters were also asked to provide a grade of each student's overall clinical performance. Generalizability studies were performed to estimate the reliability of the ratings. The effects of rater experience and clerkship setting were investigated by regression analysis. SETTING: Teaching hospitals and community-based sites in three Northwestern states. PARTICIPANTS: All students (328) who had completed the 12-week clerkship in internal medicine at one medical school during the academic years 1987-1989. Raters included attending physicians, chief residents, and other residents. RESULTS: Seven observations were needed to provide a reliable rating of the overall clinical grade. More observations were needed to obtain reliable ratings for individual items, ranging from seven observations needed for the rating of data gathering skills to 27 observations needed for the rating of interpersonal relationships with patients. Rater experience and clerkship setting (i.e., teaching hospitals vs. community-based clinics) were found, in general, not to affect significantly the ratings received by students. CONCLUSIONS: Reliable ratings of students' overall clinical skills, including overall clinical grades, can be achieved by collecting a minimum of seven observations. More observations are needed to measure reliably the interpersonal aspects of clinical performance. These findings support the use of performance ratings to evaluate clinical skills and knowledge of students in clerkship settings.
Clinical Clerkship↗