Enterprise AI Analysis
Diagnosing Korean-Language LLM Political Bias via Census-Grounded Agent Simulation
This paper introduces Dynamo-K, a census-grounded LLM agent-simulation framework used to diagnose political bias in Korean-language LLMs. We identify three systematic failure modes: progressive bias in moderate agents, model-dependent third-party salience collapse, and regional polarization collapse. The framework demonstrates that scenario reframing significantly reduces MAE and that opposing-valence models can be calibrated. The simulation accurately predicts presidential winners and dominant parties in held-out local elections at a low cost ($0.25 per 5,000-agent run).
Executive Impact & Strategic Imperatives
This research provides critical insights for enterprise leaders leveraging LLMs in politically sensitive domains. Understanding and mitigating these biases can significantly improve the reliability and trustworthiness of AI-driven simulations and predictions.
Key Strategic Implications:
- Understanding and mitigating LLM political biases is crucial for reliable AI-driven political simulations, especially in diverse linguistic and cultural contexts like Korea.
- Prompt engineering (explicit party cues, balanced descriptions, 'not from AI's perspective' instruction) is highly effective in reducing progressive bias in moderate agents, cutting MAE by 5.2x.
- The identification of salience failure vs. decision bias in third-party candidate collapse offers nuanced insights for improving model accuracy, suggesting that explicit candidate surfacing is more critical than mere knowledge.
- Regional polarization collapse highlights the need for granular geographical error decomposition, moving beyond simple 'conservative bias' labels.
- The success of scenario reframing underscores the powerful impact of prompt design on LLM outputs, suggesting it as a stronger lever than relying solely on intrinsic model knowledge.
- Opposite-valence models suggest the importance of ensemble methods or learned reweighting adapters (OSLR) for robust, model-agnostic bias correction in production deployments.
- Low operational cost allows for rapid scenario exploration, sensitivity analysis, and continuous monitoring, making LLM simulations a powerful diagnostic tool for political analysis, complementing traditional polling.
0x
MAE Reduction (Progressive Bias Mitigation)
0%
2017 MAE Recovery (Scenario Reframing)
0/3
Presidential Winner Accuracy
0
Simulation Cost (per 5,000 agents)
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Dynamo-K Agent Simulation Pipeline
Dynamo-K is a six-layer, census-grounded LLM agent-simulation framework for diagnosing Korean-language LLM political behavior. It involves data collection, preprocessing, agent synthesis, belief seeding and calibration, LLM-based vote simulation, and result aggregation. This pipeline ensures a robust and verifiable approach to political simulation.
Enterprise Process Flow
Data Collection
→
Preprocessing
→
Agent Factory
→
Belief Seeding & Calibration
→
ORC Simulation
→
Aggregation & Evaluation
Progressive Bias in Moderate Agents
LLMs exhibit a strong progressive bias, with moderate agents voting 97% progressive under vanilla prompts. Explicit mitigation strategies, including detailed orientation descriptions and 'not from AI's perspective' instructions, reduced this to 59%, cutting overall MAE by 5.2x (36.8 to 7.1%p).
5.2x
MAE Reduction from Bias Mitigation
Third-Party Salience Collapse
The simulation struggles to distribute votes among multiple viable candidates, leading to third-party candidate collapse. This decomposes into salience-failure (model doesn't mention candidate, e.g., EXAONE on Ahn Cheol-soo in 2017) and decision-bias (model mentions but doesn't convert to vote, e.g., Qwen3 on Ahn Cheol-soo).
| Mechanism | Description | Model Example |
|---|---|---|
| Salience Failure | LLM fails to acknowledge or mention the third-party candidate in reasoning. | EXAONE (0.7% mention rate for Ahn Cheol-soo in 2017) |
| Decision Bias | LLM mentions the third-party candidate but funnels votes to major parties. | Qwen3 (10.6% mention, 7.4% conversion to vote for Ahn Cheol-soo) |
Scenario Reframing Effectiveness
Reframing the election scenario prompt (e.g., reordering candidates, expanding pledges, emphasizing three-way narratives) significantly recovers collapsed third-party shares. For the 2017 election, reframing boosted Ahn Cheol-soo's predicted share from 0.9% to 18.8%, reducing MAE by 62%.
62%
2017 MAE Reduction by Reframing
Opposite Political Valences Across Models
Different LLMs exhibit opposing political biases on the same contested race. For the 2025 presidential election, Qwen3 showed an +11.8%p progressive bias for Lee Jae-myung, while EXAONE showed a -17.2%p conservative bias for Kim Moon-soo. This highlights the need for ensembling or learned calibration adapters.
Cross-Model Bias Divergence (2025 Election)
Qwen3-30B-A3B Bias
Predicted Lee Jae-myung (progressive) at 61.3% vs. actual 49.4%, showing a +11.8%p progressive bias.
EXAONE-4.0-32B Bias
Predicted Kim Moon-soo (conservative) at 58.4% vs. actual 41.2%, showing a -17.2%p conservative bias.
Implication
This opposite directional bias underscores the importance of ensemble methods or learned reweighting adapters (OSLR) for robust, model-agnostic bias correction in production deployments.
Predictive Accuracy & Cost-Effectiveness
Explore the potential savings and reclaimed hours by leveraging LLM-based political simulations for rapid diagnosis and strategic analysis.
Annual Cost Savings
$0
Annual Hours Reclaimed
0
Phased Implementation for Bias Diagnosis & Mitigation
A strategic roadmap to integrate LLM-based political bias diagnosis into your workflow, ensuring robust and accurate simulations.
Phase 1: Bias Characterization & Baseline
Establish a baseline by deploying Dynamo-K with vanilla prompts on historical elections. Quantify initial progressive bias and third-party collapse across different LLMs.
Phase 2: Prompt Engineering & Mitigation
Implement and test advanced prompt engineering techniques (explicit party cues, balanced moderate descriptions, scenario reframing) to mitigate identified biases and measure MAE reduction.
Phase 3: Cross-Model Calibration & Ensembling
Develop and apply learned calibration adapters (OSLR) to correct for model-dependent directional biases and explore ensemble methods to leverage diverse LLM valences for improved accuracy.
Phase 4: Continuous Monitoring & Refinement
Integrate the simulation pipeline for continuous monitoring of political dynamics, enabling rapid scenario exploration and sensitivity analysis. Expand to additional elections and language contexts.
Ready to Transform Your Political Analysis with AI?
Our expert team is here to help you navigate the complexities of LLM political bias and deploy robust, accurate simulation frameworks tailored to your needs.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat