Proficiency-based training and evidence-based methodology: a systematic review and meta-analysis.

Abstract

OBJECTIVE: To assess adherence of self-labelled proficiency-based progression (PBP) studies to evidence-based PBP criteria and examine associations with training outcomes. METHODS: A systematic review and meta-analysis were conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and registered in the International Prospective Register of Systematic Reviews. PubMed, CENTRAL, EMBASE, MEDLINE, and Scopus were searched from inception to 1 March 2023. Prospective English-language studies on healthcare procedural training reporting objective performance outcomes were included; non-prospective, non-quantitative, non-procedural, non-English studies, and reviews were excluded. Pre-specified outcomes included adherence to 18 evidence-based PBP criteria and objective performance metrics (errors, steps, time); secondary outcomes included proficiency benchmark achievement and Likert ratings. Data extraction was performed independently by multiple reviewers. Study quality was assessed using the Medical Education Research Study Quality Instrument and risk of bias by two investigators. Effect sizes were pooled using random-effects models (DerSimonian-Laird), expressed as the ratio of means (ROM) for continuous outcomes and bias-corrected odds ratios for dichotomous outcomes. RESULTS: Of 646 studies identified 175 met inclusion criteria. In the PBP studies (n&#x2009;=&#x2009;18), 94% fulfilled minimum criteria (use of a proficiency benchmark, its quantitative definition, and requirement for demonstration prior to progression) vs 36% of non-PBP studies (n&#x2009;=&#x2009;157). If all PBP criteria were included, 83% of PBP studies used these criteria vs only 2% of non-PBP-studies. In quantitative analysis (27 randomised clinical studies, 761 participants), ROM results showed that PBP training reduced the number of performance errors by 58% (P&#x2009;<&#x2009;0.001) and procedural time by 28% (P&#x2009;=&#x2009;0.006), increasing number of steps performed by 22% (P&#x2009;=&#x2009;0.03). When stratified based on number of criteria fulfilled, meta-regression demonstrated that increasing the number of PBP criteria fulfilled was associated with progressive and systematic trainee performance improvement. CONCLUSIONS: The more training methodologies adhere to established PBP criteria, the better training outcome will be.