Search PubMed⌕ Search

Biomedical subjects

Geoffrey R Norman

Publications and source records attributed to Geoffrey R Norman.

13 recordsLinked to original sources
PubMed2003

Practice makes perfect: the critical role of mixed practice in the acquisition of ECG interpretation skills.

PURPOSE: To examine the effect of instructional format on medical students' learning of ECG diagnosis. METHOD: Two experiments employed different learning and practice methods. In the first, students were randomly allocated to one of two instructional approaches, one organized around features (e.g., QRS voltage) and the other around diagnostic categories (e.g., bundle branch blocks), followed by a practice phase. In the second experiment, the instruction was standardized, and students were randomly allocated to one of two practice phases, either "contrastive" where examples from various categories are mixed together, or "non-contrastive" where all the examples in a single category are practiced in a single block. RESULTS: In the first experiment, there was no significant differences in students' diagnostic accuracy on novel ECG examples. In the second experiment, students exposed to the contrastive approach in the practice phase had superior diagnostic accuracy (46%) compared to 30% accuracy for the non-contrastive session, p < 0.05). CONCLUSION: These experiments highlight two important features in the design of instructional materials. First, learning around the features of the problem (analogous to problem-based learning) may have no advantages over learning the category. Second, the design and organization of deliberate practice can result in significant learning gain.

Canada↗
PubMed2003

Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation.

BACKGROUND: A number of studies have computed the minimally important difference (MID) for health-related quality of life instruments. OBJECTIVE: To determine whether there is consistency in the magnitude of MID estimates from different instruments. METHODS: We conducted a systematic review of the literature to identify studies that computed an MID and contained sufficient information to compute an effect size (ES). Thirty-eight studies fulfilled the criteria, resulting in 62 ESs. RESULTS: For all but 6 studies, the MID estimates were close to one half a SD (mean = 0.495, SD = 0.155). There was no consistent relationship with factors such as disease-specific or generic instrument or the number of response options. Negative changes were not associated with larger ESs. Population-based estimation procedures and brief follow-up were associated with smaller ESs, and acute conditions with larger ESs. An explanation for this consistency is that research in psychology has shown that the limit of people's ability to discriminate over a wide range of tasks is approximately 1 part in 7, which is very close to half a SD. CONCLUSION: In most circumstances, the threshold of discrimination for changes in health-related quality of life for chronic diseases appears to be approximately half a SD.

Chronic Disease↗
PubMed2002

A critical look at transition ratings.

Patient ratings of the extent to which they have improved or deteriorated-transition ratings-are extremely common in clinical practice and clinical research. However, some have raised concerns about transition rating validity. We examined data from three studies, each of which explored the relation between a disease-specific health-related quality of life (HRQL) instrument and transition ratings corresponding to instrument domains. For instance, we looked at the relation between differences in score on the dyspnea domain of the Chronic Respiratory Questionnaire at Times 1 (pre score) and 2 (post score), and the patients' global rating of change in dyspnea at time 2. We restricted ourselves to comparisons in which the correlation between the HRQL instrument domain and the corresponding global rating of change was at least 0.5. A perfectly valid transition rating would show correlations with the pre and post scores of equal magnitude and opposite sign, and regression coefficients of similar magnitude. Of 14 comparisons, correlations between pre and post scores and transition ratings were similar in three instances, and regression coefficients similar in eight. After considering the post score in a regression in which the transition score was the dependent variable, the pre score explained a statistically significant portion of the variance at the 0.01 level in all but four instances. Although transition scores seldom show the ideal pattern of association with pre and post scores, pre scores usually show appreciable correlation and highly significant regression coefficients with transition scores. Investigators using transition scores should ensure their validity by exploring relationships with pre and post scores of corresponding domain scores.

Asthma↗
PubMed2002

Adapting the Key Features Examination for a clinical clerkship.

PURPOSE: A written test of clinical decision-making, the Key Features Examination, was developed for use in clerkship. METHODS: Following the guidelines provided by the Medical Council of Canada, a Key Features Examination was developed and implemented in an internal medicine clinical clerkship, during the 1998/99 clerkship year. The reliability and concurrent validity of the exam were assessed. RESULTS: A 2 hour examination, containing 15 key feature problems, was administered to 101 students during 6 consecutive internal medicine clerkship rotations. The reliability of the exam, calculated from Cronbach's alpha, was 0.49. The exam had modest correlation with other measures of knowledge and clinical performance. CONCLUSION: The Key Feature Examination is a feasible and reliable evaluation tool that may be implemented as a component of student assessment during a clinical clerkship.

Canada↗
PubMed2002

Self and peer assessment in tutorials: application of a relative-ranking model.

PURPOSE: While self assessment continues to be touted as being of paramount importance for continuing professional competence, problem-based learning curricula, and adult learning theory, techniques for ensuring valid judgments have proven elusive. This study tested the applicability of an innovative relative-ranking procedure to problem-based learning tutorials. METHOD: A total of 36 students in the McMaster University Faculty of Health Sciences' MD program were provided relative-ranking forms listing seven domains of competence along with their definitions. The student, two of the student's peers, and the student's tutor were asked to complete the ranking exercise after their second, fourth, and sixth tutorials. RESULTS: Combining each level of the time and rater variables generated 66 correlation coefficients, none of which was significantly different from zero. Re-performing the analysis on only the extreme domains did not improve this result. CONCLUSION: The relative-ranking instrument developed did not prove to be a reliable measure of tutorial performance. Ratings were inconsistent from one week to the next as well as across raters within a week.

Attitude of Health Personnel↗
PubMed2002

Validity of admissions measures in predicting performance outcomes: a comparison of those who were and were not accepted at McMaster.

BACKGROUND: In typical validity studies, regression analyses are used to examine the relation between admissions measures and subsequent performance. This approach is problematic as it generally yields low correlation coefficients, which are difficult to interpret. Further, it leaves unanswered the question of how those applicants rejected by the process would fare had they been admitted. PURPOSE: This study examines the validity of the admissions measures used to assess non-cognitive qualities at McMaster's Medical School in a unique manner. METHODS: Three cohorts: (a) those offered an admission on the first round, (b) those offered an admission on the second round and (c) those rejected by McMaster, but accepted to another Canadian medical school were compared on admissions evaluations and licencing examination performance. RESULTS: The results indicate that although the scores of those who were offered an admission were significantly greater than those rejected by McMaster on each of the admission tools, licencing examination performance was comparable. CONCLUSIONS: These results are consistent with a previous regression-based validity study and indicate the need for closer examination of admissions tools.

Cohort Studies↗
PubMed2002

Validity of admissions measures in predicting performance outcomes: the contribution of cognitive and non-cognitive dimensions.

BACKGROUND: Admissions committees face the daunting task of selecting a small number of candidates who are most likely to succeed in medical school from a large pool of seemingly suitable applicants. While numerous studies have shown moderate correlations among measures of academic performance, predictors of the non-cognitive domain (e.g. interpersonal, communication, ethical) remain elusive, in part because of the absence of a sound criterion measure. PURPOSE: We examined the utility of several cognitive and non-cognitive criteria used in the admissions processes in predicting both cognitive and non-cognitive dimensions of the licencing examinations of the Medical Council of Canada (LMCC). METHODS: Predictors included: undergraduate GPA, undergraduate science GPA, an autobiographical letter, scores from a simulated tutorial, a personal interview and the MCAT. Of specific interest was the relation between measures of communication and problem-exploration skills as assessed during the admissions process and Part II of the LMCC Examination, a multi-station OSCE. RESULTS: Undergraduate GPAs were found to have the most utility in predicting both academic and clinical performance. Scores derived from the simulated tutorial did not predict future performance. The MCAT Verbal Reasoning score and the personal interview were found to be useful in predicting communication skills on the LMCC Part II. CONCLUSIONS: The results have implications for any school that uses the interview as an admissions tool.

Cognition↗
PubMed2002

Expert-novice differences in memory: a reformulation.

BACKGROUND: One of the most discriminating measures of expertise in multiple domains has been performance on memory tasks. In medicine, however, the relation between expertise and memory is more equivocal. PURPOSE: To compare and contrast the sufficiency of multiple explanations of this finding by using three probes of memory rather than the traditional free recall task alone. METHODS: Students, residents, and internists were asked to read case histories and assign diagnoses before undertaking free recall, cued recall, and recognition tests. RESULTS: Students consistently outperformed internists. Resident performance was more variable. CONCLUSIONS: Our data appear to rule out (a) the notion that expert memory for cases takes on an encapsulated form, (b) the idea that experts simply say less than students in response to a free recall task, and (c) the possibility that experts attend differentially to highly diagnostic features. The results can best be explained by the idea that students process the featural details of a case history more elaborately than do expert diagnosticians who, instead, read medical cases more holistically.

Clinical Competence↗
PubMed2002

Methods to explain the clinical significance of health status measures.

One can classify ways to establish the interpretability of quality-of-life measures as anchor based or distribution based. Anchor-based measures require an independent standard or anchor that is itself interpretable and at least moderately correlated with the instrument being explored. One can further classify anchor-based approaches into population-focused and individual-focused measures. Population-focused approaches are analogous to construct validation and rely on multiple anchors that frame an individual's response in terms of the entire population (eg, a group of patients with a score of 40 has a mortality of 20%). Anchors for population-based approaches include status on a single item, diagnosis, symptoms, disease severity, and response to treatment. Individual-focused approaches are analogous to criterion validation. These methods, which rely on a single anchor and establish a minimum important difference in change in score, require 2 steps. The first step establishes the smallest change in score that patients consider, on average, to be important (the minimum important difference). The second step estimates the proportion of patients who have achieved that minimum important difference. Anchors for the individual-focused approach include global ratings of change within patients and global ratings of differences between patients. Distribution-based methods rely on expressing an effect in terms of the underlying distribution of results. Investigators may express effects in terms of between-person standard deviation units, within-person standard deviation units, and the standard error of measurement. No single approach to interpretability is perfect. Use of multiple strategies is likely to enhance the interpretability of any particular instrument.

Adult↗