Enterprise AI Analysis

Executive Impact & Key Metrics

Generative artificial intelligence (AI) systems in higher education are now being integrated into learning environments as conversational tutors, writing and feedback assistants, and components of adaptive learning systems [24,28]. Students' adoption of AI tools, both regulated and unregulated by classroom policy, has become widespread, with prior work finding a majority of undergraduates report prior use and frequent engagement with large language models for coursework-related tasks [26,28]. As these tools become part of students' learning routines, attention has begun to focus on how students cognitively engage with AI-generated outputs during learning.

In open-ended problem-solving contexts such as programming, AI systems may provide suggestions, explanations, or proposed solution steps (for example, code) that appear coherent and confident while still being incomplete or incorrect [5,16]. This creates an immediate judgment task for learners: deciding how to interpret the output and whether, when, and how to incorporate it into their work. Prior work documents students' use of generative AI to produce explanations, suggest solution strategies, debug code, and clarify errors [19,24,26]. Such support can reduce friction and help learners make progress when human help is unavailable. Yet, without guardrails, students may begin to rely on AI output in place of their own reasoning [16,22,29].

These concerns have drawn attention to how students interpret and use AI-generated responses. When learners accept model outputs at face value, they may bypass sense-making, reduce self-explanation, and copy solutions without fully reasoning through them [16,27,29]. Studies of AI-supported programming learning show that while AI assistance can increase engagement and positive affect, it can coincide with confusion, frustration, and variable depth of cognitive processing depending on the usefulness and quality of the feedback [20]. Prior work also cautions that when AI functions primarily as an information provider, learners may accept responses with limited evaluation, which can reduce deeper engagement and higher-order thinking [12,29]. This matters for instruction, since effective learning relies on sustaining productive struggle, metacognitive engagement, and active participation during AI-supported tasks [23]. At the same time, a growing body of research shows that generative AI systems can produce misleading outputs that are difficult for users to detect [5,38]. These failures are consequential in educational contexts, where students may adopt flawed solution paths or internalize incorrect explanations [29]. Because AI-generated responses are often fluent and confident, they can exert a persuasive influence, increasing the likelihood that users accept them without sufficient verification [2].

Understanding how students regulate their reliance on AI assistance is therefore critical. In this work, we examine how students' trust in an AI assistant relates to appropriate reliance during AI-assisted programming problem-solving. We test whether trust predicts reliance behavior, and whether this relationship is moderated by learner characteristics that may influence evaluative judgment. Following prior conceptualizations of human-AI reliance, we distinguish among appropriate reliance, overreliance, and underreliance [36]. Appropriate reliance occurs when students accept correct AI recommendations and reject incorrect ones, demonstrating calibrated trust and critical evaluation. Overreliance occurs when students accept flawed AI-generated guidance without sufficient verification, while underreliance occurs when they dismiss accurate and potentially helpful AI support. Accordingly, we address the following research questions: (RQ1) What is the relationship between students' trust in an AI assistant and appropriate reliance during AI-assisted problem-solving?, and (RQ2) Do individual learner characteristics moderate the relationship between trust and appropriate reliance during AI-assisted problem-solving?

Your Implementation Roadmap

A phased approach to integrate these insights into your enterprise.