BMC Medical Education Article in Press
Assessment methods in graduate medical education in critical care: a scoping review
By Claudia Jaramillo-Villegas, Erwin Hernando Hernandez-Rincon, Claudia Marcela Poveda-Henao & Yahira Rossini Guzman-Sabogal
Received: 23 August 2025, Accepted: 9 December 2025, Published online: 10 January 2026.
This scoping review provides a comprehensive mapping of the literature on innovative assessment practices within graduate and fellowship critical care medical education, identifying key trends, challenges, and future directions.
Executive Impact: Key Findings at a Glance
Our analysis distills critical insights into the evolving landscape of assessment in critical care medical education, highlighting significant trends and areas for strategic development.
0
Articles Analyzed
0
From North America
0
Highest Publication Year
0
Key Domains Explored
0
Top Study Design (Validation)
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Assessment Methods and Tools in Critical Care
The review highlights a significant shift towards a structured, multimodal, formative, and longitudinal assessment model in critical care medical education. Commonly employed tools include workplace-based assessments (WBAs) and various simulation modalities, which offer real-time, context-sensitive feedback.
Techniques such as Direct Observation of Procedural Skills (DOPS), Objective Structured Assessment of Technical Skills (OSATS), and Mini-Clinical Evaluation Exercises (Mini-CEX) are crucial for evaluating both technical and non-technical skills. Ultrasound-specific assessments, like Focused Critical Care Echocardiography (FCCE), are also gaining prominence in specialized training.
| Method | Purpose | Key Benefits for Enterprise |
|---|---|---|
| Workplace-Based Assessments (WBAs) | Evaluate clinical competence and non-technical skills in real environments. |
|
| High-Fidelity Simulation | Assess procedural skills, communication, teamwork, and critical thinking in complex scenarios. |
|
| Objective Structured Clinical Exams (OSCEs) | Standardized assessment of technical skills, communication, and professionalism. |
|
Competency Frameworks and EPAs
Competency-Based Medical Education (CBME) frameworks are increasingly central to graduate medical education in critical care. Milestones and Entrustable Professional Activities (EPAs) serve as key tools for evaluating and monitoring learner progress, guiding personalized feedback, and informing crucial entrustment decisions.
International models such as ACGME Milestones, CanMEDS, and CoBaTrICE are widely implemented to standardize assessments, ensuring global alignment and ICU-specific competencies. Structured frameworks and consensus-driven processes are essential for enhancing faculty engagement, certification clarity, and consistent documentation.
Impact of Structured Frameworks
The adoption of frameworks like ACGME Milestones and Entrustable Professional Activities (EPAs) has enabled critical care programs to standardize assessment criteria. This move supports more equitable and transparent evaluations, leading to improved trainee development and patient safety outcomes. Milestones offer granularity for tracking learner progression and personalizing feedback, while EPAs integrate real-world clinical responsibilities into assessment, influencing entrustment decisions based on context, task complexity, and team dynamics.
Technology and Innovation in Assessment
Technological innovations are reshaping assessment in graduate critical care education, enabling more authentic, scalable, and data-rich evaluations. High-fidelity simulations continue to be impactful, improving resuscitation performance and diagnostic accuracy in complex scenarios.
Electronic platforms integrate digital Workplace-Based Assessments (WBAs) with clinical tasks, facilitating nuanced evaluations and real-time entrustment decisions. Mobile applications and smartphone-based tools streamline documentation, enhance transparency, and provide data-driven insights for curriculum redesign.
| Technology | Description | Enterprise Value (Advantages) |
|---|---|---|
| Eye Tracking Systems | Measures eye metrics (fixation frequency, dwell time, pupil dilation) during tasks. |
|
| SIMPL Platform | Facilitates workplace-based assessment with immediate, structured feedback post-clinical encounters. |
|
| Firefly Lab (EHR-integrated) | An electronic health record-integrated surgical education platform for automated EPA assessments. |
|
Ethical Dimensions of Assessment
Ethical challenges persist in critical care assessment, including rater bias, underassessment of professionalism, and systemic inequities. The literature emphasizes the crucial importance of fairness, transparency, and trust in evaluation processes.
There is a strong call for comprehensive, equity-focused evaluation approaches that balance character, professionalism, and competence. Integrating ethical reasoning into teaching, using workshops and reflective practices, enhances ethical awareness. Mobile platforms, while efficient, require careful implementation to avoid superficial evaluations and maintain alignment with educational values.
Fairness & Transparency
Crucial for trustworthy evaluations and addressing systemic biases, especially for marginalized learners.
Professionalism, once considered difficult to assess, is now being integrated into frameworks using observational tools and multisource feedback that go beyond simple checklists to evaluate traits like humility and integrity. Language and cultural competence are also recognized as essential for ethical assessment and patient safety.
Enterprise Process Flow: Scoping Review Methodology
Initial Records Identified (n=969)
→
After Duplicate Removal (n=811)
→
Full-Text Articles Considered (n=103)
→
Studies Included in Review (n=68)
Calculate Your Potential AI-Driven ROI
Estimate the efficiency gains and cost savings your enterprise could achieve by optimizing assessment processes with AI and advanced technologies.
Estimated Annual Savings
$0
Annual Hours Reclaimed
0
Your AI Implementation Roadmap for Assessment Innovation
A strategic phased approach to integrate advanced assessment methods and AI into your graduate medical education programs.
Phase 01: Needs Assessment & Pilot Design (1-3 Months)
Conduct a thorough review of current assessment practices, identify key areas for improvement in critical care education, and design pilot programs for new methods like AI-enhanced WBAs or high-fidelity simulations. Secure stakeholder buy-in.
Phase 02: Faculty Development & Tool Integration (3-6 Months)
Develop and deliver comprehensive training for faculty on new assessment tools, data interpretation, and ethical considerations. Integrate selected digital platforms and technologies into existing curricula, focusing on seamless workflow. Establish initial data collection protocols.
Phase 03: Pilot Deployment & Iterative Refinement (6-12 Months)
Roll out pilot programs for a select group of trainees and faculty. Collect and analyze performance data, feedback, and engagement metrics. Use insights to refine tools, training, and processes, ensuring alignment with competency frameworks and ethical guidelines.
Phase 04: Scaled Implementation & Continuous Optimization (12+ Months)
Expand successful pilot initiatives across all relevant programs. Establish robust data infrastructure for longitudinal tracking and programmatic assessment. Implement ongoing faculty development, regular audits, and adapt to emerging technologies and educational research findings for continuous improvement.
Ready to Transform Your Critical Care Assessment?
Connect with our experts to discuss a tailored strategy for implementing cutting-edge assessment methods and AI in your medical education programs.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat