Enterprise AI Analysis Report
AI in the classroom: Exploring students' interaction with ChatGPT in programming learning
This report analyzes student interaction with ChatGPT in programming education, identifying key profiles, the impact of interventions, prior knowledge, and performance associations.
Executive Impact & Key Metrics
Our analysis distills critical findings for educational institutions and AI solution providers:
0
Distinct AI Interaction Profiles Identified
0%
Students became AI-Collaborative Coders after interventions
0
Impact of Training on Prompting Skills
0 / 2
Highest Post-Test Score for AI-Assisted Code Refiners
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
AI Interaction Profiles (RQ1)
Impact of Interventions (RQ2)
Previous Knowledge Impact (RQ3)
Profiles & Performance (RQ4)
Five distinct AI interaction profiles were identified based on students' prompting behaviors: AI-Reliant Code Generators, AI-Reliant Code Generator & Refiners, AI-Collaborative Coders, AI-Assisted Code Refiners, and AI-Independent Coders. These profiles reveal a spectrum of engagement, from direct code generation to independent problem-solving with AI refinement.
0%
AI-Reliant Code Generators (Session 1)
0%
AI-Assisted Code Refiners (Session 1)
0%
AI-Independent Coders (Session 1)
0%
AI-Collaborative Coders (Session 1)
0%
AI-Reliant Code Generator & Refiners (Session 1)
| Profile Type | Description |
|---|---|
| AI-Reliant Code Generator | Directly pastes task instructions, seeks quick, direct answers without much engagement in problem-solving. |
| AI-Reliant Code Generator & Refiner | Pastes instructions for direct answers, then refines the answer with additional prompts to meet requirements or asks for explanation/verification. |
| AI-Collaborative Coder | Inquires about specific coding knowledge/guidance, focusing on acquiring task-specific information to meet objectives, trying to understand and solve tasks themselves. |
| AI-Assisted Code Refiner | Attempts tasks first, then seeks ChatGPT's help for revision, error fixing, or optimization. Provides feedback/clarification for tailored answers. |
| AI-Independent Coder | Demonstrates high independence, completes activity without relying on ChatGPT, or asks only general, non-task-related questions. |
Significant shifts in student AI interaction profiles were observed across three sessions, indicating the influence of different AI-integration strategies. The McNemar-Bowker tests revealed significant differences between sessions (Session 1 vs 2: x² (10, N=135)=34.478, p<.001; Session 2 vs 3: x² (10, N=136)=57.099, p<.001).
Enterprise Process Flow
Session 1: No Guidance (Baseline Profiles)
→
Session 2: Prompt Writing Training
→
Session 3: Lab Guide with Sample Prompts
→
Evolved AI Interaction Profiles
From Session 1 to 2, a shift occurred from 'AI-Reliant Code Generator' towards 'AI-Reliant Code Generator & Refiner', and a decrease in 'AI-Independent Coders' coupled with an increase in 'AI-Assisted Code Refiner'. This suggests increased engagement in refining AI-generated code. From Session 2 to 3, there was a substantial rise in the 'AI-Collaborative Coder' profile (reaching 51.37%), indicating a growing preference for active collaboration, and a sharp decline in 'AI-Reliant Code Generator'.
Previous programming knowledge significantly influenced AI interaction profiles in the initial activity session. Students with higher prior knowledge were more likely to adopt the AI-Assisted Code Refiner profile, while those with lower knowledge tended to be AI-Reliant Code Generators.
p = .008
Significant difference in prior knowledge scores (AI-Assisted Code Refiners vs. AI-Reliant Code Generators)
Specifically, AI-Assisted Code Refiners (M=8.35, SD=3.22) had significantly higher mean prior knowledge test scores compared to AI-Reliant Code Generators (M=5.5, SD=3.49), supporting the idea that foundational understanding shapes AI engagement.
Significant differences were observed in students' post-test scores across different AI interaction profiles, highlighting the importance of active and critical engagement with AI tools for better learning outcomes. The post-test scores were out of 2 points.
| Profile | Session 1 (Mean ± SD) | Session 2 (Mean ± SD) | Session 3 (Mean ± SD) |
|---|---|---|---|
| AI-Reliant Code Generator | 1.17 ± 0.36 | 1.28 ± 0.32 | 1.14 ± 0.24 |
| AI-Reliant Code Generator & Refiner | 1.14 ± 0.26 | 1.26 ± 0.33 | 1.23 ± 0.33 |
| AI-Collaborative Coder | 1.17 ± 0.30 | 1.35 ± 0.36 | 1.37 ± 0.32 |
| AI-Assisted Code Refiner | 1.37 ± 0.30 | 1.51 ± 0.40 | 1.44 ± 0.41 |
| AI-Independent Coder | 1.46 ± 0.32 | 1.47 ± 0.33 | 1.47 ± 0.40 |
In Session 1, AI-Assisted Code Refiners (p = .047) and AI-Independent Coders (p = .003) significantly outperformed AI-Reliant Code Generators. AI-Independent Coders also scored higher than AI-Reliant Code Generator & Refiners (p = .016) and AI-Collaborative Coders (p = .034). In Session 2, AI-Assisted Code Refiners had significantly higher scores than AI-Reliant Code Generator & Refiners (p = .010). For Session 3, a significant difference was found overall, but no specific pairs were identified.
Advanced ROI Calculator
Estimate the potential return on investment for integrating advanced AI tools into your enterprise operations.
Estimated Annual Savings
$0
Annual Hours Reclaimed
0
Your AI Implementation Roadmap
A phased approach ensures seamless integration and maximum impact across your organization, tailored to your unique needs identified in this analysis.
Phase 01: Strategic Assessment & Planning
Initial consultation to deeply understand your current educational workflows, student demographics, and specific learning objectives. We will identify critical integration points for AI tools like ChatGPT and define measurable success metrics.
Phase 02: Pilot Program Design & Development
Develop a tailored pilot program based on the identified interaction profiles and performance insights. This includes custom prompt engineering training, instructional material adaptation, and setting up a controlled environment for initial deployment.
Phase 03: Phased Rollout & Educator Training
Gradual introduction of AI tools across specific courses or departments, accompanied by comprehensive training for educators on AI literacy, effective prompting strategies, and monitoring student-AI interactions to foster collaborative learning.
Phase 04: Performance Monitoring & Optimization
Continuous tracking of student engagement, learning outcomes, and AI tool effectiveness. Regular feedback loops will inform iterative improvements to the AI integration strategy, ensuring alignment with pedagogical goals and maximizing student success.
Ready to Transform Your Educational Approach?
Unlock the full potential of AI in your classroom. Schedule a personalized consultation to discuss how these insights can be applied to your institution.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat