AI Research Analysis
Impact of LLMs News Sentiment Analysis on Stock Price Movement Prediction
This research explores the integration of Large Language Models (LLMs) for news sentiment analysis into stock price movement prediction, addressing a critical gap in understanding the benefit of sentiment data and comparative LLM performance.
Executive Impact & Key Findings
A distilled overview of the most critical insights and their potential implications for enterprise decision-making and operational efficiency.
DeBERTa outperforms FinBERT and RoBERTa for sentiment classification (75% accuracy).
An ensemble SVM model combining all three LLMs achieves superior sentiment prediction (80% accuracy).
Sentiment features significantly improve regression performance for PatchTST and TimesNet models.
Sentiment features enhance classification accuracy for LSTM, PatchTST, and tPatchGNN models.
Specific sentiment aggregation methods (sum and count) are crucial for predictive power.
0%
Sentiment Accuracy (Ensemble)
0%
DeBERTa Accuracy
0%
LSTM Accuracy Improvement
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Sentiment Model Comparison
Sentiment Feature Aggregation
Stock Movement Classification
Stock Movement Regression
Methodology Workflow
| Model | Accuracy | Precision | Recall | F1-Score |
|---|---|---|---|---|
| FinBERT | 0.696 | 0.713 | 0.701 | 0.7 |
| ROBERTa | 0.589 | 0.744 | 0.591 | 0.585 |
| DeBERTa | 0.752 | 0.761 | 0.759 | 0.755 |
80%
Ensemble SVM Sentiment Accuracy
Combining FinBERT, RoBERTa, and DeBERTa outputs via an SVM model significantly boosts sentiment classification accuracy, demonstrating the complementarity of these models.
| Variant | AUC | F1-Score |
|---|---|---|
| LSTM (Baseline) | 0.5557 | 0.5507 |
| LSTM_wo_count | 0.5393 | 0.5361 |
| LSTM_wo_sum | 0.5468 | 0.5411 |
| LSTM_wo_majority | 0.5473 | 0.5495 |
| LSTM_wo_count_sum | 0.5318 | 0.5251 |
The study reveals that both news count and sentiment sum features play a crucial role in accurate classification, as their removal significantly degrades performance. Majority vote, however, has a minimal impact.
| Model | LSTM F1 (avg, sd) | PatchTST F1 (avg, sd) | TimesNET F1 (avg, sd) | tPatchGNN F1 (avg, sd) |
|---|---|---|---|---|
| NS (No Sentiment) | 0.541 ± 0.078 | 0.519 ± 0.074 | 0.540 ± 0.068 | 0.430 ± 0.114 |
| FinBERT | 0.564 ± 0.057 | 0.507 ± 0.074 | 0.538 ± 0.064 | 0.431 ± 0.112 |
| ROBERTa | 0.561 ± 0.039 | 0.485 ± 0.079 | 0.534 ± 0.087 | 0.443 ± 0.108 |
| DeBERTa | 0.534 ± 0.075 | 0.502 ± 0.087 | 0.544 ± 0.065 | 0.491 ± 0.049 |
| SVM (Ensemble) | 0.554 ± 0.049 | 0.519 ± 0.076 | 0.528 ± 0.061 | 0.411 ± 0.109 |
Sentiment features improve the accuracy of LSTM, PatchTST, and tPatchGNN classifiers. Notably, LSTM and PatchTST are more suited for classification tasks with sentiment integration than TimesNET and tPatchGNN.
0.562
Highest AUC with FinBERT on LSTM
FinBERT-enhanced LSTM achieves the highest AUC in stock movement classification, demonstrating the value of domain-adapted LLMs.
| Model | LSTM MAE (avg, sd) | PatchTST MAE (avg, sd) | TimesNET MAE (avg, sd) | tPatchGNN MAE (avg, sd) |
|---|---|---|---|---|
| NS (No Sentiment) | 0.032 ± 0.009 | 0.391 ± 0.186 | 0.483 ± 0.215 | 0.170 ± 0.085 |
| FinBERT | 0.033 ± 0.010 | 0.251 ± 0.166 | 0.274 ± 0.145 | 0.176 ± 0.088 |
| ROBERTa | 0.034 ± 0.011 | 0.237 ± 0.149 | 0.254 ± 0.158 | 0.176 ± 0.087 |
| DeBERTa | 0.033 ± 0.010 | 0.208 ± 0.148 | 0.291 ± 0.146 | 0.175 ± 0.087 |
| SVM (Ensemble) | 0.033 ± 0.010 | 0.205 ± 0.122 | 0.306 ± 0.165 | 0.178 ± 0.088 |
PatchTST and TimesNET show substantial performance gains from sentiment incorporation for regression tasks. TimesNET benefits most, reducing MAE by 0.146-0.229 and RSE by 2.067-2.779 compared to the no-sentiment baseline.
2.881
Lowest RSE (PatchTST w/ SVM Sentiment)
SVM-based sentiment integration into PatchTST achieves the largest improvement in RSE for regression, highlighting its efficacy.
LLM-Enhanced Stock Prediction Methodology
News Sentiment Analysis (LLMs)
→
Sentiment Feature Aggregation
→
Time-Series Data Integration
→
Stock Movement Prediction (Regression/Classification)
→
Performance Evaluation
Our methodology involves a multi-stage process, from leveraging specialized LLMs for sentiment extraction to integrating these insights with advanced time-series models for robust stock price movement prediction.
Calculate Your Potential AI-Driven ROI
Estimate the financial impact of integrating advanced LLM-based sentiment analysis into your stock prediction strategies.
Your Path to Smarter Stock Predictions
A phased approach to integrating LLM-powered sentiment analysis for enhanced financial forecasting.
Phase 1: Sentiment Model Selection & Customization
Identify optimal LLMs (e.g., DeBERTa, FinBERT) for your financial data, fine-tuning for domain-specific nuances and ensemble integration.
Phase 2: Data Integration & Feature Engineering
Develop robust pipelines for aggregating news sentiment with market data, exploring various aggregation methods (e.g., sum, count) for predictive feature creation.
Phase 3: Predictive Model Development & Training
Integrate sentiment features with state-of-the-art time-series models (LSTM, PatchTST, TimesNet, tPatchGNN) for classification and regression tasks, ensuring rigorous temporal validation.
Phase 4: Performance Evaluation & Optimization
Benchmark models using appropriate financial metrics (AUC, F1-score, RMSE, MAE) and iterate on model architectures and sentiment integration strategies for peak performance.
Phase 5: Deployment & Continuous Monitoring
Implement the validated models in a real-time environment, establishing monitoring systems to track performance and adapt to evolving market dynamics and news landscapes.
Ready to Transform Your Financial Forecasting?
Book a free 30-minute consultation with our AI specialists to discuss how LLM-powered sentiment analysis can revolutionize your investment strategies.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat