Quality of Cochrane reviews. Quality of Cochrane reviews is better than that of non-Cochrane reviews.
Explore the source record for details and available documents.
SEARCH · Search PubMed
Search indexed PubMed citations on genomics, clinical trials, systematic reviews and public health. Explore titles, authors and supplied subject terms, then open the PubMed record.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
BACKGROUND: Changing the organization of patient care should contribute to improved patient outcomes as functioning of clinical teams and organizational structures are important enablers for improvement. OBJECTIVE: To provide an overview of the research evidence on effects of organizational strategies to implement improvements in patient care. DESIGN: Structured review of published reviews of rigorous evaluations. DATA SOURCES: Published reviews of studies on organizational interventions. REVIEW METHODS: Searches were conducted in two data-bases (Pubmed, Cochrane Library) and in selected journals. Reviews were included, if these were based on a systematic search, focused on rigorous evaluations of organizational changes, and were published between 1995 and 2003. Two investigators independently extracted information from the reviews regarding their clinical focus, methodological quality and main quantitative findings. RESULTS: A total of 36 reviews were included, but not all were high-quality reviews. The reviews were too heterogeneous for quantitative synthesis. None of the strategies produced consistent effects. Professional performance was generally improved by revision of professional roles and computer systems for knowledge management. Patient outcomes was generally improved by multidisciplinary teams, integrated care services, and computer systems. Cost savings were reported from integrated care services. The benefits of quality management remained uncertain. CONCLUSION: There is a growing evidence base of rigorous evaluations of organizational strategies, but the evidence underlying some strategies is limited and for no strategy can the effects be predicted with high certainty.
BACKGROUND AND OBJECTIVE: To review existing quality assessment tools for diagnostic accuracy studies and to examine to what extent quality was assessed and incorporated in diagnostic systematic reviews. METHODS: Electronic databases were searched for tools to assess the quality of studies of diagnostic accuracy or guides for conducting, reporting or interpreting such studies. The Database of Abstracts of Reviews of Effects (DARE; 1995-2001) was used to identify systematic reviews of diagnostic studies to examine the practice of quality assessment of primary studies. RESULTS: Ninety-one quality assessment tools were identified. Only two provided details of tool development, and only a small proportion provided any indication of the aspects of quality they aimed to assess. None of the tools had been systematically evaluated. We identified 114 systematic reviews, of which 58 (51%) had performed an explicit quality assessment and were further examined. The majority of reviews used more than one method of incorporating quality. CONCLUSION: Most tools to assess the quality of diagnostic accuracy studies do not start from a well-defined definition of quality. None has been systematically evaluated. The majority of existing systematic reviews fail to take differences in quality into account. Reviewers should consider quality as a possible source of heterogeneity.
BACKGROUND: The peer review system faces increasing strain from rising manuscript volumes, reviewer fatigue, and well-documented interreviewer disagreement. Large language models (LLMs) have shown potential to support the peer review process, but their ability to replicate editorial decisions at high-impact medical journals and their utility as manuscript screening tools remain unknown. PURPOSE: To compare the agreement between an LLM and the final editorial decision on manuscripts submitted to the American Journal of Sports Medicine and to evaluate the potential of LLMs as a manuscript screening tool. STUDY DESIGN: Cross-sectional agreement study. METHODS: Fifty-four manuscripts randomly selected from submissions to the American Journal of Sports Medicine (September 2024-October 2024) were reviewed by a locally deployed LLM (Ministral 3 14B; Mistral AI) using a standardized prompt. The artificial intelligence (AI) produced a categorical recommendation (reject, cascade, revision, or accept) and a numerical score (0-100) for each manuscript. Agreement with the final editorial decision was assessed by Cohen kappa (4-category model) for pooled human reviewers (n = 139 reviews) and the AI (n = 54). Screening performance was evaluated by positive predictive value (PPV), sensitivity, and specificity. RESULTS: Pooled human reviewers demonstrated fair agreement with the final decision (κ = 0.181 [P < .001]; 42.4% agreement), while the AI demonstrated slight, nonsignificant agreement (κ = 0.126 [P = .099]; 37.0% agreement). The AI recommended revision for 61.1% of manuscripts, of which 72.7% were ultimately rejected or cascaded, demonstrating systematic "revision bias." When the AI recommended rejection, 54.5% of those manuscripts were ultimately rejected and 27.3% were cascaded; when the AI recommended cascade, 50% were rejected and 50% were cascaded. However, when the AI recommended rejection or cascade (n = 21), 90.5% received a final decision of rejection or cascade (PPV, 90.5%; specificity, 81.8%). Manuscripts with an AI score <70 were rejected or cascaded 88.0% of the time (PPV, 88.0%). CONCLUSION: AI cannot replicate the nuanced judgment of human peer reviewers at a high-impact sports medicine journal. When AI recommended rejection or cascade, 90.5% of manuscripts received that final decision (descriptive PPV, 90.5%; 95% CI, 71.1%-97.3%), suggesting potential utility as an exploratory first-pass screening tool warranting further validation in larger cohorts. However, AI could not reliably distinguish manuscripts destined for outright rejection from those that would be cascaded to a sister journal-an important limitation for editorial triage applications.
Requirements for ethical review of research involving human subjects are based on widely accepted international standards that are implemented by various national regulations and institutional policies. In the United States, most human research is reviewed by institutional review boards (IRBs) applying federal standards. Researchers may not realize the variety of studies that should be submitted for review. Studies involving interventions must be reviewed regardless of whether these are standard interventions, nonmedical interventions, or retrospective studies. Quality improvement activities require IRB review if the results may be published. Research use of information from medical records or of biomedical specimens requires review unless no identifiers are recorded. Review must be sought at least annually and whenever the research changes. Even human research that is technically exempt from Federal regulations usually needs to have that exemption certified by someone other than the researcher.
This paper describes the implementation of both peer review and record review systems as initiated at the University Hospitals of Cleveland occupational therapy department. Peer review evaluations can be used to ascertain a therapist's need for continuing education in a specific area, to evaluate and improve standards of practice, to gather statistics on professional objectives, and to develop Professional Standards Review Organization criteria. The system of peer review stimulated an improvement in documentation of services provided by occupational therapists, as well as in critical self-awareness of individual strengths and weaknesses. A separate method of record review was instituted to provide a basis for evaluating individual performance for either recognition or disciplinary purposes. Unlike peer review, record review results become incorporated into the therapist's personnel record.
In the Netherlands over 20% of dentists participate in peer review groups. The peer review method is highly structured and consists of: setting standards for good dental care, measuring and evaluating performances against the standards and making improvements if necessary. An independent research institute (Nivel) investigated the perceived effects and advantages of and barriers to peer review. A postal questionnaire was sent tot 278 dentists who participate in peer review (response 60%). The results showed many effects of peer review. The most frequently reported effects were: more insight into the performances of colleagues, increased inter-professional contacts, a more critical attitude to performances, increased professional expertise and new inspiration and motivation. The respondents reported improvements in practice-organisation as well as in technical aspects of dental care. However, many dentists also mentioned some barriers to peer review. The method was evaluated as rather complex and it appeared to be difficult to reach consensus about standards for good dental care. In general, respondents perceived a positive balance between advantages and disadvantages of peer review, however many dentists were dissatisfied with the lack of financial compensation for participation in peer review.
To make sound health care decisions, policy makers, providers and researchers need access to relevant research findings. The role of systematic reviews is increasingly acknowledged as an important contribution in evidence-based health care decision making, and several review efforts, including that of the international Cochrane Collaboration, are under way. The Cochrane Collaborative Review Group on HIV Infection and AIDS (CRG on HIV/AIDS), conducts systematic reviews on the prevention and the treatment of HIV infection and AIDS and is guided by the Cochrane Collaboration's principles, which include minimizing potential bias, ensuring quality in the review process, keeping reviews up to date, and enhancing collaboration. The CDC HIV/AIDS Prevention Research Synthesis (PRS) project is working closely with the CRG on HIV/AIDS to produce Cochrane reviews of behavioral prevention interventions and on development and maintenance of a centralized, cumulative electronic database of HIV/AIDS behavioral prevention studies. Systematic reviews can play an important role in advancing evidence-based policy and practice in HIV/AIDS prevention and care.
The Medicare Peer Review Organization (PRO) program includes preprocedure review using explicit criteria to assess the appropriateness of specific procedures. This study evaluates the variability in the PRO preprocedure criteria for the three procedures most frequently reviewed by PROs: carotid endarterectomy, cataract removal, and cardiac pacemaker implants. In August 1989, the PRO review criteria were received from the Health Care Financing Administration. To provide a reference point for reviewing the PRO criteria, national practice guidelines for these three procedures were identified. Wide variability was demonstrated in the PRO procedure-specific carotid endarterectomy and cataract removal review criteria among PROs, and the criteria differed significantly from the identified practice guidelines. The criteria for cardiac pacemaker implants were somewhat less variable, and were based, to varying degrees, on practice guidelines developed by the American College of Cardiology (ACC). Greater attention is needed to improve the development of review criteria, including the use of relevant practice guidelines, to ensure that review criteria are optimal.
The above material represents a detailed discussion of the GMCF quality review plan, the quality intervention plan, and the sanction plan. It should be clear that every effort is being made by GMCF-PRO to provide fair, realistic, and commonsense quality of care reviews to the physician and hospital community of Georgia. It should also be evident that GMCF interventions will focus on education. Practicing physicians of the appropriate specialty are involved at every level of the review, intervention, and sanction process. Due process is afforded to physicians and hospitals in each plan as described above. The practicing physician community should recognize the obvious need for active participation in the PRO review process to provide proper balance and perspective to quality of care reviews. The GMCF 29-member board (23 physicians) is committed to the concept of maintaining practicing physician involvement and participation in the HCFA-PRO program as mandated by Congressional legislation. It is necessary also for GMCF to continue with close communication to MAG and all specialty societies in order to ensure the recruitment of specialty physician consultants for a quality chart review program, review of screening criteria, and review committee participation.
A mail survey was conducted to document the experience, critical comments, and recommendations of a sample of applicants and peer reviewers who participated in the 1983 grantee selection process conducted by the National Institute of Handicapped Research. Questionnaires were sent to 46 applicants and 36 peer reviewers who participated in seven priority areas involving competition for either a research and training center or a rehabilitation engineering center. Questionnaires were returned by 37 (80%) of the applicants and 27 (75%) of the reviewers. The peer reviewers were generally more satisfied with their experience. Their negative criticism was concerned largely with the excessiveness of the work load. The reviewers were unanimous in stating that federal personnel made no effort to influence their judgments. The majority of applicants agreed that the criteria for evaluating proposals were stated clearly, but they disagreed with how some of the criteria were weighted. The applicants' strongest dissatisfaction was with the time allowed to prepare applications and with the selection of peer reviewers. Analysis of the collective publication record of successful applicants and of the peer reviewers indicated that the reviewers had contributed significantly less to the literature of the relevant priority area.