Transfer Learning with Foundational Models for Time Series Forecasting using Low-Rank Adaptations
Foundational Models are an emerging widely used technique of Generative Artificial Intelligence. These models are distinguished by their scalability and the ease with which they can be adapted through the exploitation of Transfer Learning. The availability of high computational power and large datas...
| Autores: | , , , |
|---|---|
| Tipo de recurso: | artículo |
| Estado: | Versión publicada |
| Fecha de publicación: | 2025 |
| País: | España |
| Institución: | Universidad de Jaén |
| Repositorio: | RUJA. Repositorio Institucional de la Producción Científica de la Universidad de Jaén |
| OAI Identifier: | oai:ruja.ujaen.es:10953/6952 |
| Acceso en línea: | https://www.sciencedirect.com/science/article/pii/S1566253525003203 https://hdl.handle.net/10953/6952 |
| Access Level: | acceso abierto |
| Palabra clave: | Time series forecasting Transfer learning Foundational models Large lenguaje models Low-rank adaptations 004 004.4 004.6 004.8 |
| Sumario: | Foundational Models are an emerging widely used technique of Generative Artificial Intelligence. These models are distinguished by their scalability and the ease with which they can be adapted through the exploitation of Transfer Learning. The availability of high computational power and large datasets have supported their development, achieving a high generalization capacity due to the enormous and heterogeneous amounts of data used in their initial training. These characteristics contribute to a solid base that can be adapted or adjusted to a wide range of tasks, increasing their applicability. This study proposes the methodology LLIAM, a straightforward adaptation of a kind of Foundational Models, Large Language Models, for the Time Series Forecasting task. An adequate time-series prompting schema and Low-Rank Adaptations are used to enhance the knowledge of the model with diverse time series datasets, known as the fine-tuning phase. A study divided in two stages has been performed for evaluating the effectiveness of the proposed methodology. Initially, a comparison was made between the performance of LLIAM and different state-of-the-art Deep Learning algorithms, including Recurrent Neural Networks and Temporal Convolutional Networks, as well as a LLMbased method, TimeLLM. Following this, a zero-shot study is presented in order to evaluate the generalization capacity of the proposed methodology with time series datasets from unknown domains not considered in the model training. The outcomes of this investigation demonstrate the efficacy of LLIAM, highlighting that this straightforward and general approach can attain competent results without the necessity for applying complex modifications. This work also encourages the use of available esources (such as these pre-trained models) and efficient fine-tuning techniques to avoid unnecessary and costly training, arrowing the gap between the goals of traditional Artificial Intelligence and Green Artificial Intelligence. |
|---|