Jaehoon Kim et al. | CHI '26, Barcelona, Spain
LAPS: Automating Hypothesis-Driven Statistical Analysis of Public Survey Using Large Language Models
Public surveys are indispensable resources for understanding social dynamics, yet their analysis often imposes a high cognitive load due to structural complexity. In this paper, we present LAPS, a Large Language Model (LLM)-assisted automated framework that supports end-to-end, hypothesis-driven statistical analysis of survey data. LAPS consists of four modules (i.e., Operationalization, Planning, Execution, and Reporting) with human-in-the-loop mechanisms.
Key Contributions & Impact
LAPS significantly streamlines complex survey analysis, empowering social science researchers with enhanced agency, reduced cognitive load, and trustworthy results through human-AI collaboration.
0
Integrated Modules
0
Researchers in User Study
0
Avg. Workload Reduction (NASA-TLX)
0
Avg. Usability Improvement (SUS)
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Overview
Methodology
Results
Discussion
Introduction to LAPS
LAPS (LLM-assisted Automated framework for Public Survey data analysis) is a system designed to support social science researchers through a hypothesis-driven statistical analysis. It addresses the high cognitive load and structural complexity typically associated with analyzing public survey data, offering a streamlined and transparent workflow.
LAPS Enterprise Process Flow
Operationalization
→
Planning
→
Execution
→
Reporting
LAPS Framework & Design Goals
LAPS is grounded in design goals derived through iterative discussions with domain experts, ensuring it aligns with social science researchers' analytical workflows. It features four core modules that translate conceptual hypotheses into testable measures, structure statistical plans, generate and execute analysis code, and compile comprehensive reports.
| Feature | LAPS Advantage | Traditional Tools / General LLMs |
|---|---|---|
| Conceptual Operationalization |
|
|
| Statistical Planning |
|
|
| Workflow Alignment |
|
|
Key Findings from User Study
A user study with 12 social science researchers evaluated LAPS against traditional statistical tools (BE) and general-purpose LLMs (GLE). LAPS demonstrated significant improvements across key metrics, enhancing the research workflow and output reliability.
Reduced Cognitive Load
LAPS substantially reduced cognitive burden (NASA-TLX) by 1.94 points compared to traditional tools (p < 0.001), making analysis smoother and more efficient.
| Tutorial Exercise | LAPS Selected Variable(s) | LAPS Statistical Method | LAPS p-value | Original Tutorial p-value |
|---|---|---|---|---|
| T1: >50% U.S. households use air conditioning? | ACUsed | One-sample z-test | < 0.001 | < 0.001 |
| T2: Avg. temp differ day/night in winter? | WinterTempDay; WinterTempNight | Paired t-test | < 0.001 | < 0.001 |
| T3: Relationship between housing unit type & year built? | HousingUnitType; YearMade | Pearson chi-square test | < 0.001 | < 0.001 |
| T4: Avg. age differ for Biden voters vs. others? | (DV) Age; (IV) VotedPres2020_selection | Two-sample t-test | < 0.001 | < 0.001 |
| T5: Difference in gender distribution across early voting status? | Gender, EarlyVote2020 | Rao-Scott chi-square test | 0.03 | 0.03 |
Discussion & Future Implications
LAPS demonstrated its ability to preserve researcher agency, reduce cognitive burden, and produce trustworthy outputs. Key discussion points revolve around supporting early-stage researchers without undermining data literacy, integrating knowledge resources, and managing interpretive variability.
Enhancing Researcher Agency
Participants perceived LAPS as a collaborative tool that preserved their control over key analytical decisions, offering structured alternatives while leaving interpretive and strategic choices to the researcher. One participant noted, "I could revise and adjust variable selection and the analysis plan [...] and it always felt like the tool left the decision to me. I could use the suggested method as it was or combine multiple options, so it really felt like it was giving me choices.” (P3). This highlights how LAPS empowers users to guide the analytic process effectively.
Calculate Your Potential ROI with Enterprise AI
See how much time and cost your organization could reclaim by automating repetitive analytical tasks with custom AI solutions.
Annual Cost Savings
$0
Annual Hours Reclaimed
0
Your AI Implementation Roadmap
Our proven process guides your enterprise from initial strategy to fully optimized AI operations, ensuring seamless integration and measurable success.
Discovery & Strategy
We begin with a comprehensive analysis of your existing workflows, identifying key challenges and opportunities where AI can deliver maximum impact and align with your business objectives.
Solution Design & Prototyping
Based on insights, we design a tailored AI solution, including architecture, models, and integration points. Prototypes are developed for early validation and feedback.
Development & Integration
Our expert team builds and integrates the AI solution into your enterprise systems, ensuring robust performance, scalability, and security across your infrastructure.
Deployment & Optimization
The AI system is deployed with continuous monitoring and optimization. We provide ongoing support, training, and fine-tuning to maximize efficiency and ROI.
Ready to Transform Your Enterprise with AI?
Connect with our AI specialists to discuss your unique needs and explore how custom LLM solutions can drive significant efficiency and innovation in your organization.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat