From Isolated Exams to Longitudinal Judgment: Implementing Programmatic Assessment with Multisource Data and Portfolio-Based Educational Decisions in Anesthesiology Residency

Abstract

Background: Competency-based medical education (CBME) emphasizes longitudinal development of professional competence and has driven a shift away from isolated high-stakes examinations toward programmatic assessment (PA). Programmatic assessment integrates multiple low-stakes assessments, continuous feedback, and portfolio-based decision-making to support learning and ensure defensible judgments of competence. Evidence regarding the implementation of PA in anesthesiology residency remains limited. Objectives: The main objective of this study was to assess the efficiency of PA in anesthesiology residency in Iran for improving the quality of anesthesiology training. Methods: This mixed-methods descriptive-analytic study evaluated the implementation of a PA system in an anesthesiology residency program at a university-affiliated teaching hospital in Iran (2020 - 2025). The assessment framework incorporated entrustable professional activities (EPAs), workplace-based assessments [mini-clinical evaluation exercise (mini-CEX) and direct observation of procedural skills (DOPS)], bi-monthly multiple-choice question (MCQ) examinations with individualized feedback, multisource feedback, global faculty assessments, and an electronic portfolio. Longitudinal resident performance data were analyzed using descriptive statistics and linear mixed-effects models. Faculty mentors’ attitudes were assessed using a validated questionnaire. Results: Sixty-one residents contributed 134 resident-year observations across four years of training. Mean scores for direct observational performance assessments, EPAs, and global faculty assessments increased progressively with advancing training year. Mixed-effects analyses demonstrated a significant effect of training year on all outcomes (P < 0.001). Faculty mentors reported positive attitudes toward the PA system, high satisfaction, and strong willingness to continue participation, indicating acceptability and perceived educational value. Conclusions: Programmatic assessment was feasible to implement in anesthesiology residency and was associated with longitudinal improvement in resident performance across competency domains. Faculty acceptance further supports the educational value of PA as an effective strategy for operationalizing CBME and supporting evidence-informed, learner-centered educational decision-making.