AI RESEARCH PAPER ANALYSIS

LLM-driven hybrid architecture for multi-variate and multi-horizon forecasting of consumption patterns using graphs, recurrent units, and transformers

With growing urbanization, climate change, and increasing dependence on electricity, energy load forecasting has become essential for modern power systems. Predicting power demand precisely is essential to ensure the sustainability, stability, and efficiency of electrical infrastructure. Nevertheless, energy consumption patterns are characterized by intricate and non-linear information that is related to both space and time. Consequently, it is difficult for conventional forecasting models to accurately capture the spatial and temporal dependencies that are essential for short- and long-term prediction tasks. The self-attention mechanism of Large Language Models (LLM) has recently gained popularity, allowing them to analyze and train the model by recognizing long-range linkages and complex patterns. Leveraging the idea of transformers, which form the basis of LLMs, this work presents a hybrid TimeGPT model that integrates Graph Convolution Network (GCN), which is a type of Graph Neural Networks (GNNs), Gated Recurrent Units (GRUs), and a Transformer Decoder to effectively capture long-term dependencies for accurate future predictions and thus address the aforementioned challenges. This design effectively encapsulates the spatial relationships as well as the temporal variations in energy consumption patterns. In contrast to foundational large pre-trained models, such as TimeGPT, the proposed model is designed as a task-specific alternative, enabling efficient training on individual resident smart-meter time series, even with limited data availability. Incorporating environmental factors such as temperature, humidity, pressure, and weather conditions, the proposed model is evaluated using real-world power consumption data from domestic households. Experiments are conducted to evaluate the performance, taking into account resident-specific analysis across various forecast horizons of 1H, 4H, 6H, 12H, and 24H. Furthermore, the results demonstrate that the proposed approach outperforms baseline models such as TimeGPT, Chronos, Moirai, Linear Regression, LSTM, MLP, and XGBoost across various forecast horizons, achieving lower MSE, RMSE, and MAE. The model demonstrates strong generalization ability, interpretability, and suitability for deployment in smart energy systems.