AI ANALYSIS REPORT
COMPOUND DECEPTION IN ELITE PEER REVIEW: A FAILURE MODE TAXONOMY OF 100 FABRICATED CITATIONS AT NEURIPS 2025
Large language models (LLMs) are increasingly used in academic writing workflows, yet they frequently hallucinate by generating citations to sources that do not exist. This study analyzes 100 AI-generated hallucinated citations that appeared in papers accepted by the 2025 Conference on Neural Information Processing Systems (NeurIPS), one of the world's most prestigious AI conferences. Despite review by 3–5 expert researchers per paper, these fabricated citations evaded detection, appearing in 53 published papers (≈1% of all accepted papers). Our analysis reveals a critical finding: every hallucination (100%) exhibited compound failure modes, explaining why peer review fails to detect them. These findings demonstrate that current peer review processes do not include effective citation verification and that the problem extends beyond NeurIPS to other major conferences, government reports, and professional consulting. We propose mandatory automated citation verification at submission as an implementable solution to prevent fabricated citations from becoming normalized in scientific literature.
Executive Impact: Identifying & Mitigating AI-Generated Deception
AI-generated fabricated citations represent a fundamental challenge to research integrity, wasting time, corrupting knowledge graphs, and posing financial risks across academia and industry. Our findings reveal a systematic failure in current verification processes.
53
Contaminated Papers
100%
Detection Evasion Rate
2,500,000
Estimated Annual Wasted Effort
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
AI Hallucination Categories
Our study developed a five-category taxonomy to classify AI-generated hallucinated citations: Total Fabrication (66%), Partial Attribute Corruption (27%), Identifier Hijacking (4%), Placeholder Hallucination (2%), and Semantic Hallucination (1%).
Total Fabrication involves wholesale invention of authors, titles, venues, and identifiers. Partial Attribute Corruption blends real and fabricated elements, exploiting pattern recognition. Identifier Hijacking uses valid IDs linking to unrelated papers, creating false verifiability. Semantic Hallucination invents plausible-sounding titles that fit the domain, and Placeholder Hallucination involves obvious generation failures like template variables.
Multi-Layered Deception Uncovered
A critical finding: every single hallucination (100%) exhibited compound failure modes. This means AI-generated fabrications are multi-layered, exploiting multiple verification heuristics simultaneously.
The dominant pattern was Total Fabrication (66% primary) layered with Semantic Hallucination (63% secondary), meaning invented citations often had plausible-sounding titles. Identifier Hijacking (29% secondary) created false verifiability by providing working links to unrelated papers, misleading reviewers who perform superficial checks.
Implementing Robust Verification
Current peer review fails because it lacks systematic citation verification. To detect compound fabrications, a multi-attribute verification strategy is essential, moving beyond superficial checks like 'does the link work?' or 'do I recognize this author?'.
We propose a four-step verification process: 1) Existence check (web/database), 2) Metadata consistency check (authors, title, venue, date match), 3) Identifier validation (IDs point to claimed papers), and 4) Semantic plausibility check (flag suspicious titles for human review).
The Broader Contamination Landscape
The issue of AI-generated fabricated citations extends far beyond a single conference. These cases illustrate that the problem is systemic, affecting prestigious academic venues, government agencies, and professional consulting firms globally. The common factor is reliance on human verification in an environment where AI tools generate plausible-looking but fabricated citations at scale.
The contamination is driven by systemic factors like tool accessibility and publication pressure, rather than individual misconduct. Mandatory automated citation verification at submission is proposed as a solution, similar to plagiarism detection, to prevent normalization of fabricated citations.
66%
Total Fabrications: Wholesale Invention
The overwhelming prevalence of Total Fabrication (66%) indicates that AI language models primarily involve wholesale invention when generating citations, creating complete fictional sources rather than attempting to corrupt real scholarly metadata. This makes them challenging to detect without a comprehensive existence check.
| Verification Approach | Efficacy Against Simple Errors | Efficacy Against Compound Errors |
|---|---|---|
| Superficial Check (e.g., 'Link works') | ✓ Effective for obvious errors, fails for IH | ✗ Fails for IH where link works but metadata is mismatched |
| Author/Venue Recognition | ✓ Effective if names/venues are fabricated | ✗ Fails for PAC where real names are corrupted or used incorrectly |
| Semantic Plausibility (Title matches topic) | ✓ Effective if title is nonsensical | ✗ Fails for SH where titles are plausible but fabricated |
| Comprehensive Cross-Verification | ✓ Detects all errors | ✓ Detects all errors by checking multiple attributes |
Our analysis reveals that every hallucination exhibited compound failure modes, meaning they exploit multiple superficial checks simultaneously. This table illustrates why simple verification heuristics are insufficient against these multi-layered deceptions.
Enterprise Process Flow
Existence Check (Web/Databases)
→
Metadata Consistency Check
→
Identifier Validation Check
→
Semantic Plausibility Flagging
To combat the systemic failure of peer review, we propose mandatory automated citation verification at the point of submission. This four-step process ensures comprehensive checking against all known failure modes, preventing fabricated citations from entering scientific literature.
Beyond NeurIPS: A Systemic Problem
ICLR 2026 Submissions
GPTZero's analysis found over 50 additional hallucinations in ICLR 2026 papers under review, with some papers receiving average ratings of 8/10 despite containing fabricated citations. This suggests that without intervention, contaminated papers would have been accepted.
U.S. Government Reports
The U.S. government's 'Make America Healthy Again' report contained fabricated citations identified only after publication, requiring corrections and undermining public trust.
Deloitte Australia
Deloitte Australia was forced to refund (AUD)$98,000 after fabricated citations were discovered in a government report they produced. This highlights the financial and reputational risks of unverified AI-generated content in professional services.
The issue of AI-generated fabricated citations extends far beyond a single conference. These cases illustrate that the problem is systemic, affecting prestigious academic venues, government agencies, and professional consulting firms globally. The common factor is reliance on human verification in an environment where AI tools generate plausible-looking but fabricated citations at scale.
Calculate Your Potential Efficiency Gains
Understand the direct impact of automating citation verification and other AI-driven content integrity checks within your organization.
Annual Savings
$0
Hours Reclaimed Annually
0
Your AI Content Integrity Roadmap
A phased approach to integrating advanced AI content verification and integrity solutions into your enterprise workflow.
Phase 1: Discovery & Assessment
Initial consultation to understand current content generation and verification workflows, identify key risk areas, and define objectives for AI content integrity.
Phase 2: Solution Design & Integration
Customization and integration of AI verification tools (e.g., citation checkers, factual consistency engines) into existing authoring and review platforms.
Phase 3: Pilot & Optimization
Deployment of the solution in a controlled pilot, gathering feedback, and optimizing verification parameters for accuracy and efficiency. Training for content teams and reviewers.
Phase 4: Full-Scale Rollout & Continuous Improvement
Company-wide implementation, ongoing monitoring of AI-generated content integrity, and adaptive updates to verification strategies as AI models evolve.
Protect Your Enterprise from AI Deception
The integrity of your content is paramount. Don't let AI-generated hallucinations undermine your research, reports, or reputation. Proactive verification is the new standard.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat