Enterprise AI Analysis

Deep Learning Models Utilized

Deep learning models are highly effective at handling complex nonlinear and time-dependent problems. Compared to traditional machine learning, deep learning can automatically extract high-level features through multi-layer networks, improving prediction accuracy. The main architectures used in this study include: (1) Transformer [10], which uses a self-attention mechanism to capture long-term dependencies and overcomes the sequential bottleneck of RNNs, making it ideal for long-term series data and stock market trends; (2) BiLSTM-Attention [11], which combines bidirectional LSTM with multi-head attention to capture both forward and backward dependencies, enhancing response to sudden market fluctuations; (3) GRU [12], which efficiently models sequences through update and reset gates, with attention to focus on key time points for improved prediction; (4) TCN [13], which uses causal dilated convolution to capture long-term dependencies, making it suitable for large-scale data and both short-term and long-term trends; (5) TabNet [14], designed for tabular data, using sparse activation for feature selection and offering strong interpretability and automatic key feature extraction; (6) LSTM-CNN [15], which combines LSTM for long-term trends and CNN for short-term fluctuations, further improving prediction accuracy and robustness.

Traditional Machine Learning Models Utilized

Traditional machine learning models perform well on low-dimensional and small-sample stock data. They can model nonlinear relationships, remain efficient with limited features, and reduce overfitting through feature selection and regularization. The main models used in this study include:(1) XGBoost [5], a GBDT-based model that integrates decision trees with regularization, improving performance on complex and noisy stock market data;(2) Extra-Trees, which uses strong randomness in feature and threshold selection to reduce overfitting and enhance efficiency, making it suitable for volatile financial data;(3) Lasso Regression [6], which applies L1 regularization for embedded feature selection, ideal for high-dimensional stock features; (4) ElasticNet Regression [7], combining L1 and L2 regularization to handle multicollinearity and improve stability;(5) Bayesian Ridge Regression, which adopts Bayesian inference to mitigate noise effects, providing robust predictions in financial environments; (6) Support Vector Regression (SVR), which searches for the optimal hyperplane to model nonlinear relationships and offers strong generalization; (7) K-Nearest Neighbors Regression (KNN) [8], which predicts using nearby sample patterns, suitable for capturing local market behaviors;(8) Multilayer Perceptron (MLP) [9], which learns complex nonlinear structures through multi-layer networks but may overfit on high-dimensional financial data.

Ensemble Learning Strategies

When dealing with stock price data, a single prediction model often faces significant challenges due to the high noise and nonlinear time series characteristics of the market data. This typically leads to poor generalization and high prediction bias. Ensemble learning effectively overcomes these issues by combining the predictions of multiple base models, improving prediction accuracy and robustness. Specifically, ensemble learning constructs a fusion function F(ŷt,1,..., Ŷt,k), which combines the predictions of multiple base models through weighted averaging or other methods to minimize common prediction error metrics like Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE), thus enhancing prediction performance. Below, we introduce three common ensemble strategies and explore their applications in stock price prediction.

Weighted Averaging: Assigns different weights to base models based on their historical performance, with better-performing models receiving higher weights. This includes Basic Weighted Average and Inverse Error Weighting.

Voting Mechanism: Extends to regression tasks by statistically evaluating central tendency to filter extreme values. Includes Median Voting (robust to noise) and Truncated Mean Voting (removes extreme values).

Dynamic Weight Allocation: Introduces a time window mechanism that dynamically adjusts model weights based on recent errors, improving long-term prediction accuracy and adaptability to changing market conditions.

Performance Evaluation Metrics

To assess the models' effectiveness, several key performance metrics were used:

• Mean Absolute Error (MAE): Measures the average absolute difference between predicted and actual values, reflecting overall error magnitude.
• Root Mean Square Error (RMSE): The square root of MSE, giving greater weight to large errors while keeping the same units as the target variable.
• Mean Absolute Percentage Error (MAPE): Expresses prediction error as a percentage, removing scale effects.
• Coefficient of Determination (R2): Measures the model's capacity to explain the variance of the data, ranging from -∞ to 1.
• Directional Accuracy: Measures correctness in predicting price movement (rise/fall), crucial in quantitative trading.

Discussion of Ablation Experiment Results

This experiment evaluated 16 prediction models, including deep learning, traditional machine learning, and ensemble methods. The results show that ensemble learning methods outperformed others in stock price prediction, while deep learning and traditional models offered distinct advantages. The BayesianRidge and Ensemble models achieved the best performance across all metrics, with perfect predictions (MAE=0.00, RMSE=0.00, MAPE=0.00%, R2=1.0000). Notably, the Ensemble model also reached 100% in directional prediction accuracy, confirming the strength of ensemble learning for complex problems. Among traditional machine learning models, Lasso (MAE=2.36, MAPE=0.73%) and ElasticNet (MAE=3.38, MAPE=1.05%) performed well, particularly in directional accuracy, achieving 81.7% and 87.0%, respectively. While XGBoost (MAE=42.03) and ExtraTrees (MAE=41.33) were less accurate in numerical prediction, they excelled in directional accuracy, with 65.7% and 84.0%. In contrast, SVR_RBF and SVR_Poly performed poorly in both numerical and directional predictions, with SVR_RBF (MAE=100.76, MAPE=30.77%) failing to produce valuable forecasts. Among deep learning models, TabNet (MAE=35.82, MAPE=10.84%) and TCN (MAE=38.05, MAPE=11.48%) showed strong performance in accuracy, particularly with complex time-series data. However, other models like Transformer (MAE=51.58), GRU (MAE=60.75), and BiLSTM (MAE=77.29) underperformed, with low directional prediction accuracy (46.6% for Transformer and 48.6% for GRU). This may be due to stock market data noise and the dependency of deep learning models on large datasets. Ensemble learning methods demonstrate strong predictive performance, especially in directional forecasting, as shown in Figure 3. By combining multiple base models, the ensemble approach effectively reduces bias and variance, thereby improving overall accuracy and stability. Among the top-performing models, Bayesian-Ridge achieves near-perfect fitting, indicating a clear linear relationship between extracted features and stock price movements. Lasso and ElasticNet perform slightly weaker but still maintain high accuracy and strong regularization, making them suitable for high-dimensional financial data and effective in mitigating overfitting. In contrast, ExtraTrees and TabNet show weaker fitting ability and higher errors, performing even worse than a simple mean baseline, though they still contribute to directional prediction. To further highlight model differences, this study includes a set of visual evaluations. The prediction visualization shows high consistency between the ensemble model and actual stock trends. The model comparison chart illustrates the distinct fitting behaviors of the top five models. The error metric chart presents MAPE rankings, emphasizing the advantages of BayesianRidge and the ensemble approach. The explanatory-power chart compares R2 scores, with higher-performing models showing stronger interpretability. The directional-accuracy chart confirms the ensemble model's 100% success rate in predicting price movements. Finally, the residual analysis reveals stable error fluctuations, demonstrating the robustness of the ensemble model. Overall, these visual results show that ensemble learning provides substantial advantages over single models in fitting accuracy, directional prediction, and stability.

Conclusion

This paper proposes a multi-model ensemble framework that integrates deep learning with traditional machine learning to enhance stock price prediction accuracy and stability. In quantitative investment, accurate stock price forecasting is essential, but single models often suffer from insufficient generalization. To address this, three ensemble strategies are introduced: weighted averaging, voting, and dynamic weight allocation, where model weights are adjusted in real-time based on historical errors, improving adaptability to market fluctuations. Experimental results show that the Bayesian Ridge model excels in certain data scenarios, capturing linear relationships and achieving near-perfect predictions. The ensemble model further optimizes performance, particularly in directional prediction, achieving 100% accuracy and enhancing stability and precision. This study demonstrates the power of ensemble learning in stock price prediction, offering an efficient decision support solution for quantitative investment and suggesting further optimization of ensemble strategies for better prediction accuracy across various data scenarios.