Internet-based continuing medical education.
Explore the source record for details and available documents.
Congresses as Topic↗
Biomedical subjects
David A Cook
Publications and source records attributed to David A Cook.
Current concepts in validity and reliability for psychometric instruments: theory and application.
Validity and reliability relate to the interpretation of scores from psychometric instruments (eg, symptom scales, questionnaires, education tests, and observer ratings) used in clinical practice, research, education, and administration. Emerging paradigms replace prior distinctions of face, content, and criterion validity with the unitary concept "construct validity," the degree to which a score can be interpreted as representing the intended underlying construct. Evidence to support the validity argument is collected from 5 sources: CONTENT: Do instrument items completely represent the construct? RESPONSE PROCESS: The relationship between the intended construct and the thought processes of subjects or observers. INTERNAL STRUCTURE: Acceptable reliability and factor structure. RELATIONS TO OTHER VARIABLES: Correlation with scores from another instrument assessing the same construct. CONSEQUENCES: Do scores really make a difference? Evidence should be sought from a variety of sources to support a given interpretation. Reliable scores are necessary, but not sufficient, for valid interpretation. Increased attention to the systematic collection of validity evidence for scores from psychometric instruments will improve assessments in research, patient care, and education.
Humans↗
Impact of self-assessment questions and learning styles in Web-based learning: a randomized, controlled, crossover trial.
PURPOSE: To determine the effect of self-assessment questions on learners' knowledge and format preference in a Web-based course, and investigate associations between learning styles and outcomes. METHOD: The authors conducted a randomized, controlled, crossover trial in the continuity clinics of the Mayo-Rochester internal medicine residency program during the 2003-04 academic year. Case-based self-assessment questions were added to Web-based modules covering topics in ambulatory internal medicine. Participants completed two modules with questions and two modules without questions, with sequence randomly assigned. Outcomes included knowledge assessed after each module, format preference, and learning style assessed using the Index of Learning Styles. RESULTS: A total of 121 of 146 residents (83%) consented. Residents had higher test scores when using the question format (mean +/- standard error, 78.9% +/- 1.0) than when using the standard format (76.2% +/- 1.0, p = .006). Residents preferring the question format scored higher (79.7% +/- 1.1) than those preferring standard (69.5% +/- 2.3, p < .001). Learning styles did not affect scores except that visual-verbal "intermediate" learners (80.6% +/- 1.4) and visual learners (77.5% +/- 1.3) did better than verbal learners (70.9% +/- 3.0, p = .003 and p = .033, respectively). Sixty-five of 78 residents (83.3%, 95% CI 73.2-90.8%) preferred the question format. Learning styles were not associated with preference (p > .384). Although the question format took longer than the standard format (60.4 +/- 3.6 versus 44.3 +/- 3.3 minutes, p < .001), 55 of 77 residents (71.4%, 60.0-81.2%) reported that it was more efficient. CONCLUSIONS: Instructional methods that actively engage learners improve learning outcomes. These findings hold implications for both Web-based learning and "traditional" educational activities. Future research, in both Web-based learning and other teaching modalities, should focus on further defining the effectiveness of selected instructional methods in specific learning contexts.
Cross-Over Studies↗
Quality improvement: best evidence in clinical practice and clinical evidence of best practice.
Explore the source record for details and available documents.
Advisory Committees↗
Process monitoring in intensive care with the use of cumulative expected minus observed mortality and risk-adjusted P charts.
INTRODUCTION: A health care system is a complex adaptive system. The effect of a single intervention, incorporated into a complex clinical environment, may be different from that expected. A national database such as the Intensive Care National Audit & Research Centre (ICNARC) Case Mix Programme in the UK represents a centralised monitoring, surveillance and reporting system for retrospective quality and comparative audit. This can be supplemented with real-time process monitoring at a local level for continuous process improvement, allowing early detection of the impact of both unplanned and deliberately imposed changes in the clinical environment. METHODS: Demographic and UK Acute Physiology and Chronic Health Evaluation II (APACHE II) data were prospectively collected on all patients admitted to a UK regional hospital between 1 January 2003 and 30 June 2004 in accordance with the ICNARC Case Mix Programme. We present a cumulative expected minus observed (E-O) plot and the risk-adjusted p chart as methods of continuous process monitoring. We describe the construction and interpretation of these charts and show how they can be used to detect planned or unplanned organisational process changes affecting mortality outcomes. RESULTS: Five hundred and eighty-nine adult patients were included. The overall death rate was 0.78 of predicted. Calibration showed excess survival in ranges above 30% risk of death. The E-O plot confirmed a survival above that predicted. Small transient variations were seen in the slope that could represent random effects, or real but transient changes in the quality of care. The risk-adjusted p chart showed several observations below the 2 SD control limits of the expected mortality rate. These plots provide rapid analysis of risk-adjusted performance suitable for local application and interpretation. The E-O chart provided rapid easily visible feedback of changes in risk-adjusted mortality, while the risk-adjusted p chart allowed statistical evaluation. CONCLUSION: Local analysis of risk-adjusted mortality data with an E-O plot and a risk-adjusted p chart is feasible and allows the rapid detection of changes in risk-adjusted outcome of intensive care patients. This complements the centralised national database, which is more archival and comparative in nature.
Critical Care↗
Models of interprofessional learning in Canada.
This article provides an overview of interprofessional education in Canada, with a view to defining programs at all levels in terms of what models have been employed. The available information implies that the lack of convincing evidence of the effectiveness of existing programs is probably the most serious problem for the expansion of interprofessional education. The objectives of the programs are both to increase the knowledge about the other professions and their scope of practice, and to improve team function, and there are a number of well-established interprofessional programs in Canada that are designed to achieve these objectives, and many other examples of programs that are partial or planned. Despite this, the present interprofessional education initiatives tend to involve only a small proportion of the total health work trainees. There is a need for programs that are more widespread. The most frequent model involves a mandatory experience, which is case-based, involves all the students registered in Health Faculties, and where the students form interprofessional student teams. In addition to examining believable cases, the students also learn some specific information about interacting with the other professions and gain knowledge about the roles, knowledge and contributions that can be made by professions other than their own.
Canada↗
Web-based learning in residents' continuity clinics: a randomized, controlled trial.
PURPOSE: To determine whether internal medicine residents prefer learning from Web-based (WB) modules or printed material, and to compare the effect of these teaching formats on knowledge. METHOD: The authors conducted a randomized, controlled, crossover study in the internal medicine resident continuity clinics of the Mayo School of Graduate Medical Education during the 2002-03 academic year. Participants studied two topics of ambulatory medicine using WB modules and two topics using paper practice guidelines in randomly assigned sequences. Primary outcomes were format preference (assessed by an end-of-course questionnaire) and score changes from pre- to postintervention tests of knowledge. RESULTS: A total of 109 consented and 75 (69%) completed the postintervention test. Fifty-seven of 73 (78% [95% CI, 67-86%]) preferred the WB format (p < .001). Test scores improved for both formats (67.7% to 75.0% for WB, 66.0% to 73.3% for paper), but score change was not different between formats both before (p = .718) and after (p = .080) adjusting for topic, clinic site, study group, postgraduate year, and gender. Residents spent less time on WB modules (mean = 47 +/- 26 minutes) than paper (mean = 59 +/- 35, p = .024). Difficulties with passwords limited their use of WB modules for 71% (59-80%) of residents. CONCLUSION: No difference was found between WB and paper-based formats in knowledge-test score change, but residents preferred learning with WB modules and spent less time doing so. Passwords appeared to impede use of WB modules. WB learning is effective, well accepted, and efficient. Research should focus on aspects of WB instruction that will enhance its power as a learning tool and better define its role in specific settings.
Attitude of Health Personnel↗
Learning and cognitive styles in web-based learning: theory, evidence, and application.
Cognitive and learning styles (CLS) have long been investigated as a basis to adapt instruction and enhance learning. Web-based learning (WBL) can reach large, heterogenous audiences, and adaptation to CLS may increase its effectiveness. Adaptation is only useful if some learners (with a defined trait) do better with one method and other learners (with a complementary trait) do better with another method (aptitude-treatment interaction). A comprehensive search of health professions education literature found 12 articles on CLS in computer-assisted learning and WBL. Because so few reports were found, research from non-medical education was also included. Among all the reports, four CLS predominated. Each CLS construct was used to predict relationships between CLS and WBL. Evidence was then reviewed to support or refute these predictions. The wholist-analytic construct shows consistent aptitude-treatment interactions consonant with predictions (wholists need structure, a broad-before-deep approach, and social interaction, while analytics need less structure and a deep-before-broad approach). Limited evidence for the active-reflective construct suggests aptitude-treatment interaction, with active learners doing better with interactive learning and reflective learners doing better with methods to promote reflection. As predicted, no consistent interaction between the concrete-abstract construct and computer format was found, but one study suggests that there is interaction with instructional method. Contrary to predictions, no interaction was found for the verbal-imager construct. Teachers developing WBL activities should consider assessing and adapting to accommodate learners defined by the wholist-analytic and active-reflective constructs. Other adaptations should be considered experimental. Further WBL research could clarify the feasibility and effectiveness of assessing and adapting to CLS.
Aptitude↗
The research we still are not doing: an agenda for the study of computer-based learning.
Media-comparative research-that is, the comparison of computer-based learning (CBL) to noncomputer instruction-is logically impossible because there are no valid comparison groups. Results from media-comparative studies are thus confounded and difficult to meaningfully interpret. In 1994, Friedman proposed that such research be supplanted by investigations into CBL designs, usage patterns, assessment methods, and integration. His proposal appears to have largely been ignored. In this article, the author updates the agenda for research in CBL (including Web-based learning). While media-comparative studies are confounded, CBL-CBL comparisons are often not. CBL instructional designs vary in configuration (e.g., discussion board or tutorial), instructional method (e.g., case-based learning, personalized feedback, or simulation), and presentation (e.g., screen layout, hyperlinks, or multimedia). Comparisons within one level (for example, comparing two instructional methods) facilitate evidence-based improvements, but comparisons between levels are confounded. Additional research questions within the CBL-CBL framework might include: Does adaptation of CBL in response to individual differences such as prior knowledge, computer experience, or learning style improve learning outcomes? Will integrating CBL with everyday clinical practice facilitate learning? How can simulations augment clinical training? And, how can CBL be integrated within and between institutions? In addressing these questions it is important to remember the most important outcome-effect on patients and practice-and outcomes specific to CBL including costs, cognitive structuring, and learning unique to the computer-based environment. CBL is not a panacea, but holds great promise. Realization of this potential requires that media-comparative studies be replaced by rigorous, theory-guided comparisons of CBL interventions.
Computer-Assisted Instruction↗
Reliability and validity of scores from the index of learning styles.
BACKGROUND: No medical education studies have investigated the validity of scores from Felder's Index of Learning Styles (ILS). This study evaluated the reliability of ILS scores, and compared them with scores from Pelley's Learning Styles Type Indicator (LSTI). METHOD: Internal medicine residents completed the ILS twice and the LSTI once. RESULTS: A total of 138 residents consented. Cronbach's alpha and test-retest correlation for ILS scores were 0.61 and 0.75 (active-reflective dimension), 0.78 and 0.81 (sensing-intuitive), 0.70 and 0.60 (visual-verbal), and 0.67 and 0.81 (sequential-global). The multitrait-multimethod matrix provided evidence of convergence and discrimination for scores from the active-reflective and sensing-intuitive dimensions of the ILS and LSTI. Cronbach's alpha for LSTI scores ranged from 0.19 to 0.69. CONCLUSIONS: These data support the reliability and validity of scores from the ILS active-reflective and sensing-intuitive dimensions.
Female↗
What is the validity evidence for assessments of clinical teaching?
BACKGROUND: Although a variety of validity evidence should be utilized when evaluating assessment tools, a review of teaching assessments suggested that authors pursue a limited range of validity evidence. OBJECTIVES: To develop a method for rating validity evidence and to quantify the evidence supporting scores from existing clinical teaching assessment instruments. DESIGN: A comprehensive search yielded 22 articles on clinical teaching assessments. Using standards outlined by the American Psychological and Education Research Associations, we developed a method for rating the 5 categories of validity evidence reported in each article. We then quantified the validity evidence by summing the ratings for each category. We also calculated weighted kappa coefficients to determine interrater reliabilities for each category of validity evidence. MAIN RESULTS: Content and Internal Structure evidence received the highest ratings (27 and 32, respectively, of 44 possible). Relation to Other Variables, Consequences, and Response Process received the lowest ratings (9, 2, and 2, respectively). Interrater reliability was good for Content, Internal Structure, and Relation to Other Variables (kappa range 0.52 to 0.96, all P values < .01), but poor for Consequences and Response Process. CONCLUSIONS: Content and Internal Structure evidence is well represented among published assessments of clinical teaching. Evidence for Relation to Other Variables, Consequences, and Response Process receive little attention, and future research should emphasize these categories. The low interrater reliability for Response Process and Consequences likely reflects the scarcity of reported evidence. With further development, our method for rating the validity evidence should prove useful in various settings.
Education, Medical↗
Screening for cervical cancer and initial treatment of patients with abnormal results from papanicolaou testing.
New techniques for cervical cancer screening and a better understanding of the natural history of human papillomavirus (HPV) and cervical neoplasia have inspired a quest for more rational screening strategies for cervical cancer. Often, screening intervals for women older than 30 years can be expanded safely to every 3 years, and experts now agree that screening may cease after hysterectomy and in elderly women (provided certain criteria have been met). Liquid-based cytology produces more satisfactory specimens than conventional testing and offers the valuable option of treating atypical squamous cells of undetermined significance by "reflex" testing for high-risk types of HPV on the original specimen. Testing for HPV as an adjunct to cervical cytology for primary screening is now considered reasonable for many women older than 30 years.
Adult↗
Pay attention to demographics.
Explore the source record for details and available documents.
Georgia↗
Educational epidemiology.
Explore the source record for details and available documents.
Education, Medical↗
Teaching on the web: automated online instruction and assessment of residents in an acute care clinic.
Time pressures negatively impact teaching and learning in ambulatory settings. Web-based (online) learning may help in meeting this challenge. Although online teaching is increasingly used in medical education, few reports have evaluated online learning in postgraduate training. A pre-test/post-test cohort study was undertaken in an academic medical center acute care clinic to determine whether internal medicine residents accept, learn with and continue to use a 'stand-alone' online curriculum. The participants were 56 first-year internal medicine residents. Eight evidence-based, symptom-oriented online modules in acute ambulatory medicine were completed by residents during a one-month rotation. Fifty residents completed all modules. Pre- and post-tests were administered and scored automatically by computer. Average test score improved from 70% to 95% (p < 0.001). Feedback was very positive, with 96% of residents recommending the curriculum continue and 88% listing the modules as one of the most helpful learning resources for the rotation. Some 74% of residents referred back to the curriculum while caring for patients. This 'stand-alone' online curriculum, with automated instruction and assessment, facilitated effective and satisfying learning without requiring faculty involvement. This method could easily be reproduced in other settings.
Ambulatory Care↗
A practical guide to developing effective web-based learning.
OBJECTIVE: Online learning has changed medical education, but many "educational" websites do not employ principles of effective learning. This article will assist readers in developing effective educational websites by integrating principles of active learning with the unique features of the Web. DESIGN: Narrative review. RESULTS: The key steps in developing an effective educational website are: Perform a needs analysis and specify goals and objectives; determine technical resources and needs; evaluate preexisting software and use it if it fully meets your needs; secure commitment from all participants and identify and address potential barriers to implementation; develop content in close coordination with website design (appropriately use multimedia, hyperlinks, and online communication) and follow a timeline; encourage active learning (self-assessment, reflection, self-directed learning, problem-based learning, learner interaction, and feedback); facilitate and plan to encourage use by the learner (make website accessible and user-friendly, provide time for learning, and motivate learners); evaluate learners and course; pilot the website before full implementation; and plan to monitor online communication and maintain the site by resolving technical problems, periodically verifying hyperlinks, and regularly updating content. CONCLUSION: Teaching on the Web involves more than putting together a colorful webpage. By consistently employing principles of effective learning, educators will unlock the full potential of Web-based medical education.
Computer-Assisted Instruction↗
How reliable are assessments of clinical teaching? A review of the published instruments.
BACKGROUND: Learner feedback is the primary method for evaluating clinical faculty, despite few existing standards for measuring learner assessments. OBJECTIVE: To review the published literature on instruments for evaluating clinical teachers and to summarize themes that will aid in developing universally appealing tools. DESIGN: Searching 5 electronic databases revealed over 330 articles. Excluded were reviews, editorials, and qualitative studies. Twenty-one articles describing instruments designed for evaluating clinical faculty by learners were found. Three investigators studied these papers and tabulated characteristics of the learning environments and validation methods. Salient themes among the evaluation studies were determined. MAIN RESULTS: Many studies combined evaluations from both outpatient and inpatient settings and some authors combined evaluations from different learner levels. Wide ranges in numbers of teachers, evaluators, evaluations, and scale items were observed. The most frequently encountered statistical methods were factor analysis and determining internal consistency reliability with Cronbach's alpha. Less common methods were the use of test-retest reliability, interrater reliability, and convergent validity between validated instruments. Fourteen domains of teaching were identified and the most frequently studied domains were interpersonal and clinical-teaching skills. CONCLUSIONS: Characteristics of teacher evaluations vary between educational settings and between different learner levels, indicating that future studies should utilize more narrowly defined study populations. A variety of validation methods including temporal stability, interrater reliability, and convergent validity should be considered. Finally, existing data support the validation of instruments comprised solely of interpersonal and clinical-teaching domains.
Academic Medical Centers↗
Prediction of risk of death using 30-day outcome: a practical end point for quality auditing in intensive care.
STUDY OBJECTIVE: To validate the APACHE (acute physiology and chronic health evaluation) III unadjusted and similar hospital mortality estimate models on 30-day mortality, and to propose a simple approach to modeling local 30-day in-hospital mortality of critically ill hospitalized adults for quality management and risk-adjusted monitoring. DESIGN: Noninterventional, observational study. PATIENTS: A total of 5,278 consecutive eligible hospital admissions between January 1, 1995, and December 31, 1999. MEASUREMENTS: Prospective collection of demographic, diagnostic, physiologic, laboratory, and hospital admission and discharge data. RESULTS: The APACHE III mortality predictions exhibited excellent discrimination (receiver operating characteristic [ROC] curve area) for 30-day outcome (ROC area, 0.89) and hospital outcome (ROC area, 0.89). Calibration curves and Hosmer-Lemeshow statistics demonstrated good calibration of all models on 30-day outcome, except for the unadjusted APACHE III model. New, simplified risk adjustment models showed good discrimination and calibration on development and test data. ROC areas were 0.88 (developmental data) and 0.87 (test data), and the new model calibration was equivalent to the APACHE III model. CONCLUSION: For quality audit, 30-day in-hospital mortality can be used as an alternative outcome to survival to hospital discharge. New logistic regression models provide evidence that local models, possessing good calibration and discrimination, may be built from a few explanatory variables.
APACHE↗