Enterprise AI Analysis
Beneficial Mistrust in Generative AI? The Role of AI Literacy in Programmers' Handling of Bad Coding Advice
Generative AI (GenAI) systems, particularly Large Language Models (LLMs), are increasingly common, but their 'hallucinations' or misleading information pose significant risks. Our study, involving 542 U.S. programmers, reveals that higher AI literacy leads to less reliance on GenAI advice, especially when it's flawed. This beneficial mistrust is crucial for mitigating negative outcomes. We also find evidence for correspondence bias, where users attribute bad advice to inherent AI flaws rather than situational factors, a bias lessened by higher AI literacy.
Key Findings at a Glance
Our rigorous mixed-methods study reveals critical insights into human-AI interaction in programming contexts.
0
Programmers Surveyed
0.0
Immediate reduction in advice-taking with bad GenAI advice (Task t)
0.0
Continued reduction in advice-taking with bad GenAI advice (Task t+1)
0.0
WOA Reduction per SD AI Literacy
0.0
WOA Reduction due to Bad Advice x AI Literacy (Task t)
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Correspondence Bias & AI
AI Literacy Effects
Bad Advice & Trust
Understanding Correspondence Bias in Human-AI Interaction
The Judge-Advisor System (JAS) framework is used to evaluate how individuals incorporate GenAI guidance. Attributional tendencies, specifically correspondence bias, influence reasoning. Correspondence bias is the tendency to attribute observed behavior to dispositional factors (e.g., AI's reliability) while underestimating situational influences (e.g., prompt context). This bias is often driven by System 1 thinking, with System 2 correction requiring more effort. In human-AI interaction, individuals increasingly attribute human-like responsibilities to AI. Insufficient context can lead to GenAI hallucinations and bad advice, which users may attribute to the GenAI's inherent model rather than situational factors like missing prompt details. This impacts advice-taking, both immediately and in future tasks.
The Modulating Role of AI Literacy
AI literacy, defined as the competencies to critically evaluate AI, communicate, collaborate, and use AI effectively, is crucial. It encompasses understanding AI strengths, weaknesses, limitations, and ethical implications. Higher AI literacy is expected to mitigate correspondence bias by increasing awareness of how GenAI operates and the situational factors influencing its advice, such as hallucinations due to insufficient context. Conversely, low AI literacy can lead to overestimating AI intelligence and unrealistic expectations, thereby amplifying correspondence bias and increasing advice-taking, even when advice is flawed.
Impact of Bad Advice on Trust and Reliance
GenAI's probabilistic nature means it sometimes generates hallucinations and misleading advice. Our research hypothesizes that individuals receiving bad advice are less likely to follow it immediately (H1a) and less likely to take new good advice in future tasks (H1b). This 'algorithm aversion' or 'beneficial mistrust' stems from attributing bad advice to dispositional factors of the AI, damaging its perceived reputation. This effect is often more pronounced for algorithms than for humans and can be influenced by recent negative experiences, leading to reduced future reliance.
5.6%
Immediate reduction in advice-taking with bad GenAI advice (Task t)
9.4%
Continued reduction in advice-taking with bad GenAI advice (Task t+1)
-3.5%
Reduction in advice-taking for each standard deviation increase in AI literacy
| AI Literacy Level | Attribution of Bad Advice |
|---|---|
| Higher AI Literacy |
|
| Lower AI Literacy |
|
Enterprise Process Flow
User provides prompt
→
GenAI generates code (may lack context)
→
Potential for hallucinated advice
→
User evaluates advice
→
AI Literacy moderates attribution & reliance
Case Study: Programmers' Responses to Flawed GenAI Advice
In our experiment, programmers encountered 'hallucinated' variables in GenAI's code suggestions. Participants with higher AI literacy were more likely to identify missing contextual information in the prompt as the cause, stating, 'I think the AI didn't know the variable names' or 'the AI's performance degraded because it included the nonexistent columns'. They adjusted their trust in future tasks more dynamically. In contrast, participants with lower AI literacy often blamed the AI's 'poor habits in learning' or 'not having good understanding of programming language', displaying less awareness of situational factors and a persistent overestimation of AI capabilities, even after errors. This highlights how AI literacy fosters a more discerning 'beneficial mistrust'.
Advanced ROI Calculator
Estimate your potential efficiency gains and cost savings by strategically implementing AI in your enterprise workflows.
Strategic AI Implementation Roadmap
A phased approach to integrate GenAI responsibly and effectively, building on our research insights.
Phase 1: AI Literacy Assessment & Training
Evaluate existing AI literacy across teams and implement targeted training programs to foster critical evaluation skills and reduce correspondence bias.
Phase 2: Context-Aware System Design
Design GenAI interfaces that highlight situational uncertainties, prompt for missing context, and explicitly display confidence scores to prevent unwarranted dispositional attributions.
Phase 3: Expectation Calibration & Trust Repair
Implement mechanisms to align user expectations with GenAI capabilities, and develop trust-repair strategies that differentiate isolated hallucinations from systematic failures.
Phase 4: Continuous Monitoring & Adaptive Integration
Establish ongoing monitoring of human-AI interaction, collect performance feedback, and adapt integration strategies to promote calibrated reliance and resilient collaboration.
Ready to Implement Beneficial AI Mistrust?
Leverage our insights to design AI systems that empower your team with calibrated reliance and robust error handling. Our experts are ready to help you navigate the complexities of GenAI integration.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat