AI-BASED CLINICAL DECISION SUPPORT
AI-based Clinical Decision Support for Primary Care: A Real-World Study
Evaluating LLM-based clinical decision support in live primary care, demonstrating significant reductions in diagnostic and treatment errors while improving care quality and clinician satisfaction.
Key Takeaways
- Significant Error Reduction: Clinicians using AI Consult made 16% fewer diagnostic errors and 13% fewer treatment errors.
- High Clinician Satisfaction: 100% of surveyed clinicians reported improved quality of care, with 75% calling the effect "substantial."
- Real-World Impact: Introduction of AI Consult could avert 22,000 diagnostic errors and 29,000 treatment errors annually at Penda Health alone.
- No Active Harm Caused: No patient safety reports indicated AI Consult advice actively caused harm.
- Crucial Implementation Factors: Workflow-aligned design and active deployment strategies were key to successful uptake and impact.
Executive Impact
Revolutionizing Primary Care with AI-Powered Safety Nets
In a real-world study at Penda Health clinics in Nairobi, Kenya, an LLM-based clinical decision support tool, AI Consult, served as a safety net for clinicians, identifying potential documentation and clinical decision-making errors. This initiative aimed to bridge the model-implementation gap by integrating advanced AI directly into live care workflows, preserving clinician autonomy while enhancing care quality.
0
Diagnostic Errors Averted Annually
0
Treatment Errors Averted Annually
0
Diagnostic Error Reduction
0
Treatment Error Reduction
0
For Diagnostic Errors
0
For Treatment Errors
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Reduced Clinical Error Rates
AI Consult significantly reduced clinically meaningful errors across multiple categories, acting as an effective safety net for primary care clinicians.
| Category | Non-AI Error Rate (%) | AI Error Rate (%) | RRR (%) |
|---|---|---|---|
| History & Examination | 32% | 22% | 31.8% |
| Investigations | 30% | 27% | 10.3% |
| Diagnosis | 48% | 40% | 16.0% |
| Treatment | 70% | 60% | 12.7% |
0
Absolute percentage point drop in 'started red' rate for treatments in AI group, indicating clinician learning and proactive error avoidance.
Case Study: Preventing Missed Diagnoses (Table 9)
A 2-year-old presented with vomiting and abnormal blood work (HGB 9.90, MCV 58.30), indicating microcytic anemia. The clinician initially diagnosed Tonsillitis, acute bacterial, but missed the anemia diagnosis. AI Consult flagged this as a RED FLAG, highlighting the unaddressed microcytic anemia and recommending reevaluation and further investigation for iron deficiency. The clinician subsequently added "Iron deficiency anemia" to the diagnoses, enabling appropriate treatment.
Case Study: Optimizing Treatment Plans (Table 10)
A 1-year-old child presented with gastroenteritis. The clinician prescribed Metronidazole in addition to ORS and zinc. AI Consult issued a RED FLAG, stating that Metronidazole was not indicated for uncomplicated gastroenteritis without identified protozoal infection, potentially leading to unnecessary antibiotic exposure. Following AI Consult's recommendation, the clinician removed the Metronidazole prescription, focusing on hydration and zinc supplementation.
Case Study: Enhancing Medication Safety (Table 11)
A 26-year-old patient presented with dry cough, dizziness, and headache, diagnosed with Upper Respiratory Tract Infection (URTI). The clinician prescribed Betamethasone/Dexchlorpheniramine, a combination steroid/sedating antihistamine. AI Consult raised a RED FLAG, noting these medications are not recommended for URTI due to side effects and lack of efficacy. It recommended discontinuing and considering non-sedating antihistamines. The clinician replaced the inappropriate prescription with Cetrizine and Xylometazoline, improving medication safety.
Clinician Feedback and Adoption
Clinicians found AI Consult helpful and easy to use, leading to high satisfaction scores and improved clinical documentation practices.
0
Net Promoter Score (NPS) for AI Consult
0
Of AI group clinicians reported AI Consult improved quality of care
Median clinician attending time, suggesting dedicated engagement with AI insights.
| Group | Median Attending Time (minutes) |
|---|---|
| AI Group | 16.43 |
| Non-AI Group | 13.01 |
Clinicians in the AI group demonstrated longer attending times, particularly with more AI Consult triggers, indicating they spent time responding to alerts and made fewer treatment errors.
Patient-Reported Outcomes and Safety
While patient-reported outcomes showed no statistically significant difference, AI Consult demonstrated clear potential in reducing patient harm events.
| Patient Outcome (8-day follow-up) | Non-AI Group (%) | AI Group (%) |
|---|---|---|
| Rate of patients not feeling better | 4.3% | 3.8% |
| Saw a pharmacist | 3.5% | 3.4% |
| Self-referred to another clinic or hospital | 2.9% | 3.0% |
| Unplanned visit at Penda | 6.2% | 6.0% |
Note: No statistically significant difference was detected for these patient-reported outcomes.
Half
Of reviewed patient safety reports in the non-AI group indicated AI Consult could have prevented harm if visible and heeded.
No Cases
Of patient safety reports showed AI Consult actively caused harm.
Iterative Deployment and Learning
The success of AI Consult was rooted in iterative design, close integration with clinical workflows, and active deployment strategies to foster clinician adoption and learning.
Enterprise Process Flow
AI Consult v1 (Feb 2024): On-demand LLM copilot
→
AI Consult v2 (Jan 2025): Silent, event-driven safety net
→
Induction Period (Jan-Feb 2025): Training & Initial Feedback
→
Active Deployment (Mar-Apr 2025): Coaching & Incentives
| Metric | AI Group (Start of Study, %) | AI Group (End of Study, %) | Non-AI Group (Steady State, %) |
|---|---|---|---|
| "Left in Red" Rate (Final call red) | ~35-40% | ~20% | ~40% |
| "Started Red" Rate (First call red) | ~45% | ~35% | ~45-50% |
The reduction in "left in red" and "started red" rates for the AI group demonstrates effective clinician engagement and learning over time, particularly during active deployment.
Design Rationale: Safety Net Approach
AI Consult was engineered with three core objectives:
- Maximize coverage: Reviews every visit and major decision point without active clinician requests.
- Minimize cognitive load: Provides feedback via a tiered traffic-light interface (Green, Yellow, Red) only when material risk is identified, preventing alert fatigue.
- Maintain clinician autonomy: Issues recommendations, but all final clinical decisions remain with the clinician.
This design allows AI Consult to function as a lightweight supervision tool, enhancing quality without undermining professional judgment.
Calculate Your Potential AI Impact
Estimate the efficiency gains and cost savings your enterprise could achieve by implementing AI-powered solutions, tailored to your specific operational context.
Estimated Annual Savings
$0
Estimated Annual Hours Reclaimed
0
Your AI Implementation Roadmap
Leveraging insights from successful real-world deployments, here’s a phased approach to integrating AI for maximum impact and sustained success in your organization.
Phase 1: Rule-Based CDS Foundation (2019-2020)
Established an early non-AI Clinical Decision Support (CDS) system using decision trees embedded in the EMR. This significantly improved guideline adherence from 40% to over 90% for specific conditions, laying the groundwork for digital quality improvement.
Phase 2: AI Consult v1 Pilot (Feb 2024)
Deployed an early LLM copilot, providing on-demand feedback to clinicians. Achieved approximately 60% adoption and demonstrated qualitative improvements in care, confirming safety and potential for positive impact with no observed harm.
Phase 3: AI Consult v2 & Induction (Jan-Feb 2025)
Re-engineered AI Consult as a silent, event-driven safety net with a tiered traffic-light interface for real-time feedback. Initial induction period focused on clinician training, workflow integration, and addressing performance bottlenecks like system latency.
Phase 4: Active Deployment & Learning (Mar-Apr 2025)
Implemented active change management strategies, including peer champions, personalized coaching, and performance incentives. This period saw a substantial drop in "left in red" rates and proactive error avoidance by clinicians, demonstrating successful uptake and learning.
Phase 5: Continuous Iteration & Expansion
Future development aims to further reduce documentation burden, enhance contextual relevance and localization, explore voice-first interfaces, and integrate AI agents for clinician-confirmed actions within the EMR. The goal is to establish AI systems as standard for safer, more consistent, and accessible care.
Ready to Transform Your Enterprise with AI?
Connect with our experts to explore how AI-powered decision support can enhance efficiency, reduce errors, and drive innovation within your organization.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat