ENTERPRISE AI ANALYSIS
Optimizing open-domain question answering with graph-based retrieval augmented generation
This research introduces TREX (Truncated RAPTOR Expanded IndeX), a novel, cost-effective graph-based Retrieval Augmented Generation (RAG) system designed to optimize open-domain question answering (QA) for both OLTP-style (fact-based) and OLAP-style (thematic) queries. By integrating hierarchical clustering, recursive summarization, and Reciprocal Rank Fusion (RRF), TREX enhances semantic depth and retrieval accuracy. Benchmarking across diverse datasets demonstrates TREX's superior performance and cost-efficiency compared to existing graph-based RAG methodologies, highlighting its potential to augment large language models with advanced and scalable retrieval capabilities.
Executive Impact & Key Metrics
The findings from this extensive benchmarking underscore TREX's capability to deliver high accuracy and significant cost savings across various query types. It demonstrates a marked improvement in retrieval quality, especially for multi-document synthesis tasks, and offers a balanced solution suitable for enterprise-scale LLM applications. TREX's proven effectiveness in real-world scenarios, such as the DRI Copilot system, solidifies its role as a key advancement in scalable and efficient AI-driven information retrieval.
0
HotPotQA Accuracy (TREX)
0
Avg. Query Cost (TREX)
0
Indexing Cost Reduction vs. GraphRAG
0
DRI Copilot Accuracy (TREX)
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Open-domain question answering (QA) requires processing vast amounts of unstructured data. Traditional RAG methods often struggle with nuanced, multi-document synthesis. Graph-based RAG solutions like GraphRAG and RAPTOR have emerged to improve retrieval by structuring knowledge and synthesizing information across multiple documents, but often face high costs and complexity. The paper highlights the distinction between OLTP-style (fact-based, direct lookups) and OLAP-style (open-ended, thematic, synthesis-required) queries, noting that specialized LLM applications are needed for each.
$297.9B
Projected AI Software Spending by 2027
Enterprise Process Flow
Raw Text Corpus (D)
→
Text Chunks
→
Embeddings & Dim. Reduction (UMAP)
→
GMM Clustering
→
LLM Summarization (Recursive)
→
Vector DB (Summaries & Leaf Nodes)
→
User Query
→
Query Embedding
→
Dense Retrieval (Cosine Similarity)
→
Keyword Retrieval (Summaries & Leaf Nodes)
→
Reciprocal Rank Fusion (RRF)
→
Select Top-k Summaries
→
Answer Generation
TREX (Truncated RAPTOR Expanded IndeX) constructs a hierarchical tree from text chunks using Gaussian Mixture Models (GMM) for clustering and UMAP for dimensionality reduction. LLMs recursively summarize these clusters to create higher-level nodes, culminating in a root node that summarizes the entire document. At query time, TREX combines dense (vector-based) and keyword retrieval, re-ranking results using Reciprocal Rank Fusion (RRF) from Azure AI Search to ensure robust and relevant context selection. This approach is designed to be query-agnostic, handling both OLTP and OLAP tasks without an external query router.
| Methodology | MSMARCO Accuracy | HotPotQA Accuracy |
|---|---|---|
| TREX | 50.0% | 80.9% |
| RAPTOR | 50.7% | 77.6% |
| HybridSearch | 49.1% | 76.3% |
| GraphRAG (GlobalSearch) | 39.0% | 73.1% |
TREX achieves high accuracy in fact-based OLTP queries, outperforming other methods in HotPotQA and closely matching RAPTOR in MSMARCO.
| Benchmark | GraphRAG Indexing Cost | TREX Indexing Cost | TREX Query Cost |
|---|---|---|---|
| MSMARCO | $51.37 | $5.31 | $0.01 |
| HotPotQA | $389.12 | $36.51 | $0.01 |
| Kevin Scott | $63.27 | $7.03 | $0.01 |
| Earnings | $116.71 | $4.19 | $0.09 |
TREX consistently demonstrates significantly lower indexing and querying costs, making it a highly cost-effective solution compared to GraphRAG.
For OLAP-style queries, such as the Microsoft Earnings Call Transcripts, TREX outperforms RAPTOR and performs comparably to Hybrid Search across comprehensiveness, diversity, and empowerment. While GraphRAG Global Search excels for multi-document synthesis due to its structured graph representation, TREX offers a more cost-effective alternative with strong performance for single-document OLAP queries.
Enhancing Incident Resolution with TREX (DRI Copilot)
Incident resolution is a manual, time-intensive process where critical knowledge is dispersed across various siloed repositories. The internal Data DRI Copilot system aims to streamline this, but struggles with novel incidents and complex data formats leading to inaccurate responses.
Evaluating TREX alongside Azure AI Hybrid Search within the DRI Copilot system, we observed significant improvements. Hybrid Search achieved 39.1% accuracy, limited by its inability to capture intra- and inter-issue relationships. In contrast, TREX improved accuracy to 65.2%, leveraging multi-hop retrieval structures that mitigate context fragmentation issues common in standard RAG systems.
This case study demonstrates TREX's potential to enhance response quality for complex, multi-document queries in a real-world technical support setting, showcasing its utility with unstructured customer support data. Future work includes integrating GraphRAG to further improve retrieval accuracy and context synthesis.
Project Your ROI
Estimate the potential savings and reclaimed hours by implementing advanced AI solutions in your enterprise.
Projected Annual Savings
$0
Annual Hours Reclaimed
0
Your Implementation Roadmap
A structured approach to integrate advanced RAG into your existing systems for maximum impact and minimal disruption.
Discovery & Strategy
Initial consultation to understand current infrastructure, data sources, and business objectives. Define clear KPIs and a phased implementation strategy.
Data Integration & Graph Construction
Securely integrate your unstructured data. Implement graph-based indexing using TREX methodology to build the knowledge graph and vector store.
Pilot Deployment & Refinement
Deploy TREX in a pilot environment with a select group of users. Gather feedback, fine-tune retrieval parameters, and optimize LLM integration.
Full-Scale Rollout & Training
Expand deployment across relevant departments. Provide comprehensive training for your teams to maximize adoption and utilization.
Continuous Optimization & Support
Monitor performance, analyze usage patterns, and implement ongoing optimizations. Benefit from continuous support and updates to ensure sustained ROI.
Ready to Transform Your Enterprise AI?
Schedule a consultation to explore how TREX can optimize your open-domain question answering and drive significant efficiency gains.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat