Plastic surgery board review courses: to review or to reveal ... that is the question.
Explore the source record for details and available documents.
Certification↗
SEARCH · Search PubMed
Results for “REVIEW”
Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Municipal hospitals in New York City--a review of the Report of the Commission to Review the Health and Hospitals Corporation.
Explore the source record for details and available documents.
Budgets↗
Systematic reviews provide information not contained in traditional narrative reviews.
Explore the source record for details and available documents.
Anti-Inflammatory Agents, Non-Steroidal↗
Future of preschool vision screening. Review article did not separate review and implementation processes.
Explore the source record for details and available documents.
Child↗
Open peer review is feasible and does not reduce quality of reviews
Explore the source record for details and available documents.
Journal Article↗
Are reviewers suggested by authors as good as those chosen by editors? Results of a rater-blinded, retrospective study.
BACKGROUND: BioMed Central (BMC) requires authors to suggest four reviewers when making a submission. Editors searching for reviewers use these suggestions as a source. The review process of the medical journals in the BMC series is open--authors and reviewers know each other's identity--although reviewers can make confidential comments to the editor. Reviews are published alongside accepted articles so readers may see the reviewers' names and recommendations. Our objective was to compare the performance of author-nominated reviewers (ANR) with that of editor-chosen reviewers (ECR) in terms of review quality and recommendations about submissions in an online-only medical journal. METHODS: Pairs of reviews from 100 consecutive submissions to medical journals in the BMC series (with one author-nominated and one editor-chosen reviewer and a final decision) were assessed by two raters, blinded to reviewer type, using a validated review quality instrument (RQI) which rates 7 items on 5-point Likert scales. The raters discussed their ratings after the first 20 pairs (keeping reviewer type masked) and resolved major discrepancies in scoring and interpretation to improve inter-rater reliability. Reviewers' recommendations were also compared. RESULTS: Reviewer source had no impact on review quality (mean RQI score (+/- SD) 2.24 +/- 0.55 for ANR, 2.34 +/- 0.54 for ECR) or tone (mean scores on additional question 2.72 ANR vs 2.82 ECR) (maximum score = 5 in both cases). However author-nominated reviewers were significantly more likely to recommend acceptance (47 vs 35) and less likely to recommend rejection (10 vs 23) than editor-chosen reviewers after initial review (p < 0.001). However, by the final review stage (i.e. after authors had responded to reviewer comments) ANR and ECR recommendations were similar (65 vs 66 accept, 10 vs 14 reject, p = 0.47). The number of reviewers unable to decide about acceptance was similar in both groups at both review stages. CONCLUSION: Author-nominated reviewers produced reviews of similar quality to editor-chosen reviewers but were more likely to recommend acceptance during the initial stages of peer review.
Authorship↗
Peer review time: how late is late in a small medical journal?
BACKGROUND: Timeliness is an important attribute of peer review because it brings information promptly to its users. This has become even more important with the development of on-line submission and on-line peer review. Small journals usually must rely on regular mail and traditional peer review. We evaluated the review time in a small medical journal outside of mainstream science. METHODS: We analyzed 1,346 editorial requests for manuscript peer review sent from the Croatian Medical Journal to 690 reviewers from February 1998 to December 2001. RESULTS: Peer reviewer response rate was 78.6% (1,057 of 1,346 requests for review) and median review time was 29 days (95% confidence interval [95% CI]=28-31 days). Of returned reviews, 554 (52.4%) arrived on time (before the deadline established by the editor) and the others arrived late. Median delay was 12 days (95% CI=10-15 days). There was no difference in delay for Croatian- and non-Croatian-authored manuscripts, but more reviews of articles on clinical medicine were delayed than those on public health. More reviews from non-Croatian or male reviewers were never returned to the editors. For reviews that arrived late, those from male reviewers were more delayed than those from female reviewers (median, 13 days, 95% CI=11-15 days vs. median, 8 days, 95% CI=5-12 days, respectively). Logistic regression analysis revealed that odds for reviewer's positive response to an editorial request for manuscript review were greater for female Croatian reviewers, who received up to three review requests. CONCLUSIONS: Peer review time in a small general medical journal may be comparable to review times in larger and more prestigious journals. Choice of reviewers may improve the expediency of peer review: editors from small journals may profit from building and educating local peer reviewers, bearing in mind that female reviewers may provide more expedient reviews.
Female↗
[Peer review in toxicologic histopathology: its necessity, type and procedure].
A histopathology peer review already has become an integral part of industrial toxicologic pathology in the USA and Europe. Nevertheless, the review is unfamiliar to toxicologic pathologists in Japan. This report provides detailed information on convenient, useful procedures for peer review in toxicologic histopathology and describes its necessity and type. Histopathology peer review is either informal or formal. Formal review means that the target organs to be re-estimated and the reviewing pathologist's name are stated somewhere in a protocol or study report. The resolution of all diagnostic discrepancies between studies and reviewing pathologists must be clarified, and the review records need to be preserved. No audit trials need to be kept for informal peer review. Based on the purpose of the review, histopathology peer reviews may be classified into 3 major types: "complete", "problem-solving", and a "quality-monitoring" reviews. For a "Complete" review, all histopathologic findings crucial to the conclusion of the study are re-examined. A "problem-solving" review serves to re-evaluate anomalous findings. A "quality-monitoring" review is a periodical monitor of the quality of histopathologic diagnoses at pathology laboratories, particularly in toxicology contract research organizations. The procedures for histopathology peer review consist of 5 stages. At the first stage, the target organs to be reviewed are selected from the draft summary tables of the original findings diagnosed by the study pathologist, and the reviewing pathologist evaluates all selected organs. At the second stage, the study pathologist re-examines all the findings with diagnoses not accepted by the reviewing pathologist and re-considers the appropriateness of these original diagnoses. At the third stage, the study pathologist and the reviewing pathologists discuss all diagnostic differences until they reach a consensus opinion. At the fourth stage, the whole process (stage 1 to 3) of peer review of all target organs is tabulated, including the study pathologist's and reviewing pathologist's diagnoses, the study pathologist's opinions, and consensus diagnoses. At the fifth stage, Peer Review Certification is issued with the study pathologists' and reviewing pathologists' signatures. The reviewing pathologist must share the responsibility for the final diagnoses with the study pathologist. In order to gain greater credibility from regulatory agencies, a peer review performance is best made by an independent external pathologist.
Humans↗
The inclusion of reports of randomised trials published in languages other than English in systematic reviews.
OBJECTIVE: To assemble a large dataset of language restricted and language inclusive systematic reviews, including both conventional medicinal (CM) and complementary and alternative medicine (CAM) interventions. To then assess the quality of these reports by considering and comparing different types of systematic reviews and their associated RCTs; CM and CAM interventions; the effect of language restrictions compared with language inclusions, and whether these results are influenced by other issues, including statistical heterogeneity and publication bias, in the systematic review process. DATA SOURCES: MEDLINE, EMBASE, the Cochrane Database of Systematic Reviews and the Centralised Information Service for Complementary Medicine. REVIEW METHODS: Three types of systematic reviews were included: language restricted; language inclusive/English language (EL) reviews that searched RCTs in languages other than English (LOE) but did not find any and, hence, could not include any, in the quantitative data synthesis; and systematic reviews that searched for RCTs in LOE and included them in the quantitative data synthesis. Fisher's exact test was applied to compare the three different types of systematic reviews with respect to their reporting characteristics and the systematic review quality assessment tool. The odds ratio of LOE trials versus EL trials was computed for each review and this information was pooled across the reviews to examine the influence that language of publication and type of intervention (CM, CAM) have on the estimates of intervention effect. Several sensitivity analyses were performed. RESULTS: The LOE RCTs were predominantly in French and German. Language inclusive/LOE systematic reviews were of the highest quality compared with the other types of reviews. The CAM reviews were of higher quality compared with the CM reviews. There were only minor differences in the quality of reports of EL RCTs compared with the eight other languages considered. However, there are inconsistent differences in the quality of LOE reports depending on the intervention type. The results, and those reported previously, suggest that excluding reports of RCTs in LOE from the analytical part of a systematic review is reasonable. Because the present research and previous efforts have not included every type of CM RCT and the resulting possibility of the uncertainty as to when bias will be present by excluding LOE, it is always prudent to perform a comprehensive search for all evidence. This result only applies to reviews investigating the benefits of CM interventions. This does not imply that systematic reviewers should neglect reports in LOE. We recommend that systematic reviewers search for reports regardless of the language. There may be merit in including them in some aspects of the review process although this decision is likely to depend on several factors, including fiscal and other resources being available. Language restrictions significantly shift the estimates of an intervention's effectiveness when the intervention is CAM. Here, excluding trials reported in LOE, compared with their inclusion, resulted in a reduced intervention effect. The present results do not appear to be influenced by statistical heterogeneity and publication bias. CONCLUSIONS: With the exception of CAM systematic reviews, the quality of recently published systematic reviews is less than optimal. Language inclusive/LOE systematic reviews appear to be a marker for a better quality systematic review. Language restrictions do not appear to bias the estimates of a conventional intervention's effectiveness. However, there is substantial bias in the results of a CAM systematic review if LOE reports are excluded from it.
Complementary Therapies↗
Reliability of editors' subjective quality ratings of peer reviews of manuscripts.
CONTEXT: Quality of reviewers is crucial to journal quality, but there are usually too many for editors to know them all personally. A reliable method of rating them (for education and monitoring) is needed. OBJECTIVE: Whether editors' quality ratings of peer reviewers are reliable and how they compare with other performance measures. DESIGN: A 3.5-year prospective observational study. SETTING: Peer-reviewed journal. PARTICIPANTS: All editors and peer reviewers who reviewed at least 3 manuscripts. MAIN OUTCOME MEASURES: Reviewer quality ratings, individual reviewer rate of recommendation for acceptance, congruence between reviewer recommendation and editorial decision (decision congruence), and accuracy in reporting flaws in a masked test manuscript. INTERVENTIONS: Editors rated the quality of each review on a subjective 1 to 5 scale. RESULTS: A total of 4161 reviews of 973 manuscripts by 395 reviewers were studied. The within-reviewer intraclass correlation was 0.44 (P<.001), indicating that 20% of the variance seen in the review ratings was attributable to the reviewer. Intraclass correlations for editor and manuscript were only 0.24 and 0.12, respectively. Reviewer average quality ratings correlated poorly with the rate of recommendation for acceptance (R=-0.34) and congruence with editorial decision (R=0.26). Among 124 reviewers of the fictitious manuscript, the mean quality rating for each reviewer was modestly correlated with the number of flaws they reported (R=0.53). Highly rated reviewers reported twice as many flaws as poorly rated reviewers. CONCLUSIONS: Subjective editor ratings of individual reviewers were moderately reliable and correlated with reviewer ability to report manuscript flaws. Individual reviewer rate of recommendation for acceptance and decision congruence might be thought to be markers of a discriminating (ie, high-quality) reviewer, but these variables were poorly correlated with editors' ratings of review quality or the reviewer's ability to detect flaws in a fictitious manuscript. Therefore, they cannot be substituted for actual quality ratings by editors.
Peer Review↗
Effect of structured workshop training on subsequent performance of journal peer reviewers.
STUDY OBJECTIVE: We sought to determine whether peer reviewers who attend a formal interactive training session produce better reviews. METHODS: Peer reviewers were invited to attend a formal, 4-hour, highly interactive workshop on peer review. Attendees received a sample manuscript to read and review in writing in advance. The workshop included presentations on analyzing a study and the journal's expectations for a quality review, discussion of the sample manuscript's flaws and how to address them in a review, discussion of the reviews written by the attendees, and discussion of real reviews of other manuscripts illustrating key points. The performance of attendees on the basis of standard editor quality ratings (1 to 5) was assessed for the 2 years after workshop attendance. Control reviewers matched for previous review quality and volume were selected from nonattendees of the workshop. In study 1, all average reviewers received a standard written invitation. In study 2, 75 randomly selected average reviewers were personally and actively recruited with intensive follow-up by means of e-mail and telephone calls in an effort to reduce self-selection bias. RESULTS: In study 1, 25 reviewers volunteered for the course, were eligible for study, attended, and were compared with 25 matched control reviewers. Of attendees filling out evaluations, 19% thought it somewhat and 81% thought it very helpful. All thought it would improve their subsequent reviews, and 85% thought it would improve their review ratings. The mean change in rating after the workshop was 0.11 (95% confidence interval [CI] -0.25 to 0.48) for control reviewers and 0.10 (95% CI -0.20 to 0.39) for attendees. In study 2, of 75 reviewers intensively recruited, only 12 (41%) of those who said they would attend did. All of the participants thought the workshop would improve their performance and ratings. Test scores at the end of the workshop improved in 73% of participants compared with scores on pretests. The control reviewers' average rating changed by -0.10 (95% CI -0.49 to 0.29) versus 0.06 (95% CI -0.34 to 0.23) for attendees. CONCLUSION: Among invited peer reviewers, voluntary attendance at a highly structured and interactive workshop was low and did not improve the quality of subsequent reviews, contrary to the predictions of attendees. Efforts to aggressively recruit average reviewers to a second workshop were time consuming, had low success rates, and showed a similar lack of effect on ratings, despite improvement in scores on a test instrument. Workshop teaching formats, although traditional, are of unproven efficacy.
Case-Control Studies↗
The invited review ? or, my field, from my standpoint, written by me using only my data and my ideas, and citing only my publications.
Do you feel scientifically isolated? Do you find yourself sitting on the side-line while others take the field by the nose and lead it? Are you unable to publish a model that summarizes your data and ideas because reviewers label it as being too speculative and unsupported? Can't get those experiments published in any regular journal? Do you find that nobody is citing your papers? Haven't published in your field for some time, but want to show that you are still a player? Well, no need to worry! There is a special category of publication for you, 'the invited review', and even specialized journals, the 'review journals', that cater to every part of your neurosis. The major difference between many invited reviews and regular articles in journals ? this journal being an exception ? is that the invited review is generally not critiqued by outside, independent referees. In those rare cases when a review is critiqued, the letter from the editorial staff that accompanies the manuscript usually contains a disclaimer: "While we are interested in your critical evaluation of this manuscript, you should know that this review was solicited by the Editorial Board, and, therefore, we allow some latitude in the content and opinion expressed." What's the point of reviewing it when the writer has been given some sort of carte blanche to say what they want? Are you surprised? I have critiqued reviews to no affect (but then that's true also for research papers!), suggested that the writing was too biased, that the appropriate references were not cited and that the proposed model must have been conjured up while the writer was inebriated. Of course, all the critiques for my reviews were taken seriously, and I made all the suggested changes ? right! So, what kinds of review are there? As a postdoc, I got introduced to two extremes in one conversation. I was asked to write a review by my mentor. Not being very opinionated (at the time), I read hundreds of papers and cited them, and wrote a discussion of the results without providing a, how shall I say, personal opinion. My mentor scanned the manuscript, and said: "Great! Now start again. This time, write a review that contains lots of ideas and models, minimal referencing, and by all means speculate wildly on where we think the field is moving. Oh, don't throw away this one. We can use it another time for one of those 'Annual Reviews' journals." Great mentoring, huh? So, what's the point of an invited review? Well, for the lazy ? sorry I meant busy ? it is a way to keep up with publications, ideas, trends, and models in a specific field. But of course, this is exactly where the problem lies. Unless the reviews are balanced (i.e. unbiased), complete in citing publications (i.e. not just the author's papers), scholarly in the presentation of models (i.e. present competing models), what sort of intelligent way is this to 'keep up with a field'? What we need is a more critical approach to reading published papers. That's right, we should discuss the fact that a certain experiment doesn't make sense, a control was missing, and an interpretation based on the data presented is crazy. Ah, but that would be so impolitic, wouldn't it! I have two suggestions. First, let's get people to write reviews outside their own specific field of interest. There are many advantages to this. There would not be a bias, or a limited interpretation based on 'reading between the lines' or on unsupported experiments. The review would be a critical evaluation of data presented (only) in peer-reviewed research papers available to the average reader and comprise a balanced discussion of all the data from competing labs. There is of course a major disadvantage: who has the time or gumption to do this (just wait for the reprisals)? Well, most of us participate in some sort of journal club in which papers outside our field are presented. Perhaps, these discussions should be published (on the Internet?). [Some journals provide 'sound-bites' of papers (Paper Alert, or some such heading), but these tend to regurgitate the paper's abstract and are hardly critical.] Second, without implementation of the first suggestion, I think that we should add a warning sign to invited reviews: "WARNING. THE CONTENTS OF THIS REVIEW MAY BE DANGEROUS TO YOUR SCIENCE. THIS REVIEW CONTAINS STATEMENTS, IDEAS AND OPINIONS THAT MAY BE UNSUBSTANTIATED. THE MODELS HEREIN REFLECT THE STATE OF THE AUTHOR'S LOOSE GRASP OF REALITY, AND SHOULD NOT BE CONFUSED WITH FACTS. PROCEED WITH CAUTION." Invited reviews are no substitute for reading the primary literature and critically forming your own opinion. An occasional column, in which Caveman and other troglodytes involved in cell science emerge to share their views on various aspects of life-science research. Messages for Caveman and other contributors can be left at caveman@biologists.com. Any correspondence may be published in forthcoming issues. Previous Sticky Wickets can be viewed at: www.biologists.com/JCS/caveman/index.html jcs6024
Journal Article↗
[Volume and health outcomes: an overview of systematic reviews].
BACKGROUND: Improving quality and effectiveness of health care is one of the priorities of health policies. Hospital or physician volume of activity may be a measurable variable with a relevant impact on effectiveness of health care. There are several studies and systematic reviews evaluating the association between volume and outcome of health care. The aim of this review is to identify: areas, clinical conditions or interventions (prevention, diagnostic, therapeutic, surgical or clinical) for which an association between volume and outcome has been investigated; those for which an association between volume and outcome has been proved METHODS: Overview of systematic reviews and Health Technology Assessment reports; search of MEDLINE, EMBASE, The Cochrane Library, Web sites of Health Technology Assessment, other HTA Agencies, National guideline Clearinghouse, National Health Care quality tools (1995-february 2005). For each studied area results are described separately for each review due to the heterogeneity of outcomes, volume thresholds and results reported. No metanalysis has been conducted. Completeness of reporting of the systematic reviews has been evaluated using the QUOROM statement. For each review we evaluated the number of studies included and the proportion of studies with statistically significant results (p < 0,05). As far as in-hospital mortality is concerned, the different areas have been classified in the following groups: Strong evidence ofpositive association: areas with > or =10 studies included in the reviews, and high prevalence (> or =50%) of positive studies (p <0. 05) in the majority of reviews. Weak evidence of association: areas with 5 to 9 studies included in the reviews and high prevalence (> or =50%) of positive studies (p <0.05) in the majority of reviews. Weak evidence of lack of association: areas with 5 to 9 studies included in the reviews and high prevalence (>50%) of not statistically significant studies (p >0.05) in the majority of reviews. No suficient evidence of association: areas with less than 5 studies included in the reviews. No evidence of association: areas with > or =10 studies included in the reviews, and high prevalence (>50%) of not statistically significant studies (p >0.05) in the majority of reviews. The same literature search was then applied to identify primary studies published in each considered area following the most recent systematic review published. RESULTS AND DISCUSSION: We identified 21 systematic reviews and included 11 of them analysing 46 different areas. The majority of studies evaluate the effect of specific surgical procedures; the main outcomes considered are hospital mortality and 5 year survival for cancers. Considering in-hospital mortalilty as outcome, in 11 areas there is strong evidence ofassociation between volume of activity and outcome: abdominalaortic aneurysm (unruptured), percutaneous transluminal coronary angioplasty knee arthroplasty coronary artery bypass, surgery for oesophageal and pancreatic cancer, surgery for prostate cancer, colecistectomy, carotid endarterectomy, myocardial infarction, neonatal intensive care. It is never possible however to identify a unique volume threshold. For some of these areas, particularly coronary angioplasty and coronary artery bypass, there are many new studies published following the last systematic review; some specific aspects are being investigated such as the role of temporal changes in the association, the effect of different risk adjustment procedures and the separate role of physician or hospital volume. In some cases, for example knee arthroplasty in-hospital mortality could be an inadequate outcome on which judging the strength of association, in fact, the few studies evaluating other outcomes such as complications provide inconsistent results. For a range of areas the evidence of association is weak: AIDS, appendicectomy, cardiac catheterization, surgery for breast, lung, stomach cancer, hernia repair, hip fracture, hysterectomy and injuries. As far as AIDS is concerned, the few number of studies found is probably due to the lack of studies published after the introduction of effective therapies. All the included studies show an evidence of association between volume and in-hospital mortality. In no case we found weak evidence of lack of association while we identified three conditions for which the number of studies included in the reviews together with the prevalence of non significant studies do suggest lack of association; these are abdominal aortic aneurysm (ruptured), hip arthroplasty and surgery for colorectal cancer. In the case of hip arthroplasty as well, inhospital mortality could be an inadequate outcome, but only one old study found a positive association with risk of complications. Eventually there is a group of areas (n=22) for which there is not enough evidence to draw conclusions about the association between volume and outcome due to a small number of studies. In some cases, such as transplants, this could be due to the low rate of events; in this case all the few published studies show positive results. There are some limitations which should be taken into account in the interpretation of these results: despite the overall good completeness of reporting of the included reviews, the majority of studies included in the reviews themselves are cross-sectional studies representing a very weak study design to evaluate causality of the investigated association. Moreover the methodology of risk adjustment applied is heterogenous among studies and it is difficult to know the extent to which this can affect the observed results. It is eventually necessary to consider the possible occurrence of publication bias which could lead to an overestimation of the positive effect of volume on health care outcomes attributable to the lack of publication of negative studies. CONCLUSIONS: In some areas the evidence seems strong enough to guide health care organizational choices, although it is not possible to identify well defined volume thresholds. In other areas, particularly for non surgical conditions, where there is not enough evidence, it seems necessary to conduct proper epidemiological studies. Also the evaluation of effectiveness of using volume as an instrument of health policy requires further research. Taking into account the rapid and continuing process of technology development, the definition of standard and prerequisite volumes of care should be specific of each temporal period and health care system. It is therefore a dynamic process requiring a continuous review of the available evidence. In the area of evidence based public health, the limited available evidence should not impair the choice of actions based on limited evidence, but rather it should lead to the application of thefew available evidence on one side and to the planning of proper research in the areas of lack of evidence.
Evidence-Based Medicine↗
Effect of written feedback by editors on quality of reviews: two randomized trials.
CONTEXT: Better peer review is needed, but proven methods to improve quality are unknown. Our objective was to determine whether written feedback to reviewers improves subsequent reviews. METHODS: Eligible reviewers were randomized to intervention or control (receiving other reviewers' unscored reviews and the editor's decision letter). Study 1 (September 1998-September 2000) included reviewers with a median quality score of 3 or lower; study 2 (April 2000-January 2002), reviewers with median score of 4 or lower. Study 1 was designed with a power of 0.80 to detect a difference in score of 1; study 2, with a power of 0.80 to detect a difference of 0.5. All reviewers were at a peer-reviewed journal (Annals of Emergency Medicine). The main outcome measure was the editor's routine quality rating (1-5) of all reviews (blinded to study enrollment). RESULTS: For study 1, 51 reviewers were eligible and randomized and 35 had sufficient data (182 reviews) for analysis. The mean individual reviewer rating change was 0.16 (95% confidence interval [CI], -0.26 to 0.58) for control and -0.13 (-0.49 to 0.23) for intervention. For study 2, 127 reviewers were eligible and randomized, and 95 had sufficient data (324 reviews). Controls had a mean individual rating change of 0.12 (95% CI, -0.20 to 0.26) and intervention reviewers, 0.06 (-0.19 to 0.31). CONCLUSIONS: In study 1, minimal feedback from editors on review quality had no effect on subsequent performance of poor-quality reviewers, and the trend was toward a negative effect. In study 2, feedback to average reviewers was more extensive and supportive but produced no improvement in reviewer performance. Simple written feedback to reviewers seems to be an ineffective educational tool.
Evaluation Studies as Topic↗
Utility of a software assistant in critical care case review.
A computer program (the Audit Assistant) was developed to help physicians review the care of critically ill emergency department (ED) patients. The program is an example of a new class of decision aids that serves to remind physicians to consider possibilities, not an artificial intelligence program that actually attempts to simulate clinical reasoning. The goal of such programs is to enable physicians to reduce errors--in this case to enable reviewers to notice more of the errors in care in the cases they are reviewing. The objective of this study was to demonstrate on a small set of complex cases that the tested computer program enables the physician to perform a better quality review. The issue of what constitutes improved case review is addressed. In the first part of the study, reviewers reviewed two mock charts without using the Audit Assistant and then immediately reviewed the charts again with the assistance of the program. The reviews were compared. In the second part of the study, a second reviewer also compared the utility of the review of the first reviewer alone and the Audit Assistant output as an aid to review, using an additional mock chart. Six emergency physicians participated; each was a quality assurance director for the ED of one Cleveland area hospital. For the physicians reviewing without the Audit Assistant, 41% of critical actions were listed by three or four reviewers. For those using the Audit Assistant, 83% of critical actions were listed by three or four reviewers. All reviewers preferred the Audit Assistant-suggested list to the critical action list generated by a previous reviewer not using the Audit Assistant (P < .02). Use of the Audit Assistant improved the completeness and the consistency of physician review of mock charts of critically ill ED patients in a small series of cases. The critical actions added for review were important, as demonstrated by the preferential addition of critical actions chosen by other reviewers who were not using the computer program.
Critical Care↗
Implementing a systematic course/clerkship peer review process.
OBJECTIVE: A systematic course/clerkship peer-review process was developed to meet several objectives: improvement of quality of course/clerkship, enhancement of understanding of individual course and overall curricular content, improvement of communication and collaboration between basic science and clinical disciplines across campuses, provision of forum to address curricular concerns of students and faculty, facilitation of data collection for LCME reviews and the AAMC CurrMIT project; and monitoring curricular equivalency at multiple clinical sites. DESCRIPTION: Previously each course had carried out an internal review process. These reviews varied considerably in terms of the data collected and how those data were used for quality improvement. Data and outcomes of the internal reviews were seldom shared with colleagues, students, or administrators. This limited communication discouraged collaboration and led to disparity in curricular content between campuses. A collaborative effort between the medical school's Office of Medical Education (OME) and the Education Council, the faculty governance committee composed of faculty from each basic science and clinical department on each campus, has led to the development and implementation of a systematic course/clerkship peer-review process. Two standing committees of the Education Council oversee the systematic review process. One committee is charged with oversight of the first two years of the curriculum and a second committee with oversight of the third and fourth years. A two-tiered course/clerkship review process was designed with all courses undergoing a limited annual review. A comprehensive, "nuts and bolts" review for required courses was to occur every three years. The limited annual review was based on course evaluations completed by students, collected and summarized by the OME, and presented to the Education Council. The comprehensive review of a required course/clerkship occurred over a six-to-eight-month period. Course directors on each campus completed a questionnaire and submitted materials. To facilitate cross-discipline communication and synergy, a subcommittee composed of both basic science and clinical faculty, principally course directors, and at least one medical student who had completed the course at each campus reviewed submitted materials and had a series of meetings with the course directors, support personnel, and department chairs. Final review reports presented to the Education Council identify issues, concerns, and recommendations for action. To date 11 required courses have completed the comprehensive review process. Simple oversights, significant curricular disparity between campuses, and opportunities for multidisciplinary collaboration have been identified and addressed. Student-rated satisfaction has improved for those courses that have been reviewed. DISCUSSION: The systematic course/clerkship peer-review process has been a success, although there was initial resistance to "outside review." We have not yet completed one cycle of comprehensive course reviews but already faculty and administration have a better understanding of individual course and overall curriculum content. Faculty have developed working relationships and are sharing educational strategies across disciplines and campuses, and identifying innovative collaborations. The annual review process is now perceived to lack depth and is under reconsideration.
Clinical Clerkship↗
Impact of blinded versus unblinded abstract review on scientific program content.
PURPOSE: A procedure whereby reviewers are not informed of the author or institutional identity for submitted abstracts is sometimes considered a more equitable and impartial process for selection of the content for a scientific program. We performed a prospective randomized study to evaluate the impact of a reviewer blinding process on scientific program content. MATERIALS AND METHODS: A total of 234 abstracts submitted for presentation at the 2001 meeting of the Southeastern Section of the American Urological Association were distributed for review and grading to 42 reviewers who were randomly assigned to either a blinded or unblinded category. Acceptance for presentation was based on combined raw scores for blinded and unblinded reviews. However, multiple statistical comparisons were performed to evaluate the program content if only the blinded or unblinded reviews had been used as criteria for program acceptance. The abstracts were divided into pediatrics; prostate cancer, bladder cancer and urinary diversion, female urology and urodynamics, endourology and laparoscopy, benign and malignant diseases of the kidney, and penis/erectile dysfunction and miscellaneous for review. RESULTS: Statistically significant differences were observed in the variability of scores for blinded versus unblinded reviewers in the bladder cancer and prostate cancer groups but not in the other categories. For the pediatrics and urodynamics/female urology groups unblinded reviewers had statistically significant higher mean scores than blinded reviewers but this was not observed in other categories. Overall there was no clear pattern showing greater or less variability between reviewers for blinded versus unblinded reviews, nor was there a consistently observed difference in mean raw scores. Of the papers included in the program 60% would have been accepted for presentation by either blinded or unblinded review alone. This figure increased to 80% when considering the papers which were most highly scored by reviewers in each category. CONCLUSIONS: Although this study cannot be used to determine the quality of reviews based on blinding versus unblinding, it does not demonstrate any consistent increase in variability between reviewers whether they have been informed of abstract authors and institution. Furthermore, there was no identifiable tendency for blinded or unblinded reviewers to assign higher or lower raw scores to abstracts. However, blinded versus nonblinded review may have a substantial impact on program content. Only 60% of the presented abstracts would have been chosen by either method used alone. Additionally, 20% of the 5 abstracts in each category graded most highly by 1 review process would not have been accepted for inclusion on the program by the other process. These data may not be applicable to other circumstances, but the blinding process for abstract review may impact substantially on program content for section meetings.
Abstracting and Indexing↗