In-Context and Few-Shots Learning for Forecasting Time Series Data based on Large Language Models

In-context learning (ICL), introduced with GPT-3, allows large language models to generalize to new tasks by implicitly learning task structure from a few demonstrations within the input prompt. This approach is particularly powerful for time series, where re-structuring data as text prompts enables LLMs to analyze and predict sequential patterns without explicit training or re-training. While general-purpose LLMs like OpenAI o4-mini and Gemini 2.5 Flash Lite show varying degrees of success with ICL for time series, specialized foundation models like TimesFM inherently leverage principles similar to ICL through their extensive pre-training. ICL's adaptability is crucial for dynamic time series forecasting where system parameters change frequently, offering remarkable generalization and pattern recognition capabilities.

Zero-shot learning allows models to generalize to entirely new tasks or domains without any task-specific examples during training, relying purely on instructions. For time series, this involves reformatting temporal signals into textual tokens. Few-shot learning extends this by providing a small number of annotated input-output instances (demonstrations) within the prompt, helping the model infer underlying task structures. OpenAI o4-mini demonstrated competitive zero-shot performance, confirming the effectiveness of contextual reasoning in time series prediction. However, its high computational latency limits its use in edge devices. Gemini 2.5 Flash Lite showed poorer accuracy, suggesting that lightweight language reasoning alone doesn't always translate to strong time series forecasting without domain adaptation.

Traditional deep learning models like Long Short-Term Memory (LSTM) and Temporal Convolutional Networks (TCN), despite their past success, exhibited significantly poorer performance in this study compared to foundation models. Their RMSE values were considerably higher, indicating an inability to fully capture nonlinear volatility patterns without substantial datasets or extensive architecture engineering. These models typically require large amounts of labeled training data, domain-specific feature engineering, and long optimization periods. Their reliance on local temporal context and lack of global priors embedded through vast pre-training limit their ability to detect fast-evolving process disruptions and generalize across diverse time series environments.