TY - GEN

T1 - LTBoost

T2 - 33rd ACM International Conference on Information and Knowledge Management, CIKM 2024

AU - Truchan, Hubert

AU - Kalfar, Christian

AU - Ahmadi, Zahra

N1 - Publisher Copyright: © 2024 Owner/Author.

PY - 2024/10/21

Y1 - 2024/10/21

N2 - The progress of deep-learning-based forecasting architectures is evident through their expanding parameter configurations. However, the need for rapid online decision making in practical scenarios calls for an alternative strategy, highlighting the necessity for networks that are not only adaptive but also efficient in real-time operations. This shift is critical as we confront three principal challenges in deep-learning-based forecasting frameworks: (i) the inherent limitations of transformers, which, despite their attempts to preserve ordering information, the temporal information loss due to the permutation-invariant nature of self-attention mechanisms is inevitable, (ii) the inefficacy of linear models in capturing the dynamic interactions within swiftly evolving signals; and (iii) the incapacity of tree-based approaches to extrapolating beyond values present in the training set. In response to these challenges, we introduce LTBoost, an innovative boosted hybrid of linear and tree-based ensemble gradient algorithms tailored for long-term time series forecasting (LTSF) tasks, scalable to high data dimensions. LTBoost employs a dual strategy, beginning with a linear regression model to capture trends and extrapolate beyond known data, complemented by a robust nonlinear tree-based model that focuses on the residuals. This boosted hybrid approach not only addresses the challenges posed by existing models but also significantly improves forecast accuracy. The effectiveness of LTBoost is validated through empirical experiments conducted on nine well-established benchmark datasets, demonstrating superior performance and achieving state-of-the-art results in 32 out of 36 cases, measured by mean absolute error (MAE). Our findings also explore the impact of lag features and signal normalization techniques, demonstrating further improvements in predictive accuracy. This hybrid and highly effective approach highlights LTBoost's innovation and its resolution of specific forecasting challenges, setting the stage for its contribution to the field of time series forecasting, paving the way for its application in diverse real-world scenarios.

AB - The progress of deep-learning-based forecasting architectures is evident through their expanding parameter configurations. However, the need for rapid online decision making in practical scenarios calls for an alternative strategy, highlighting the necessity for networks that are not only adaptive but also efficient in real-time operations. This shift is critical as we confront three principal challenges in deep-learning-based forecasting frameworks: (i) the inherent limitations of transformers, which, despite their attempts to preserve ordering information, the temporal information loss due to the permutation-invariant nature of self-attention mechanisms is inevitable, (ii) the inefficacy of linear models in capturing the dynamic interactions within swiftly evolving signals; and (iii) the incapacity of tree-based approaches to extrapolating beyond values present in the training set. In response to these challenges, we introduce LTBoost, an innovative boosted hybrid of linear and tree-based ensemble gradient algorithms tailored for long-term time series forecasting (LTSF) tasks, scalable to high data dimensions. LTBoost employs a dual strategy, beginning with a linear regression model to capture trends and extrapolate beyond known data, complemented by a robust nonlinear tree-based model that focuses on the residuals. This boosted hybrid approach not only addresses the challenges posed by existing models but also significantly improves forecast accuracy. The effectiveness of LTBoost is validated through empirical experiments conducted on nine well-established benchmark datasets, demonstrating superior performance and achieving state-of-the-art results in 32 out of 36 cases, measured by mean absolute error (MAE). Our findings also explore the impact of lag features and signal normalization techniques, demonstrating further improvements in predictive accuracy. This hybrid and highly effective approach highlights LTBoost's innovation and its resolution of specific forecasting challenges, setting the stage for its contribution to the field of time series forecasting, paving the way for its application in diverse real-world scenarios.

KW - boosted hybrids

KW - ensemble trees

KW - forecasting

KW - time series

UR - http://www.scopus.com/inward/record.url?scp=85210037577&partnerID=8YFLogxK

U2 - 10.1145/3627673.3679527

DO - 10.1145/3627673.3679527

M3 - Conference contribution

AN - SCOPUS:85210037577

SP - 2271

EP - 2281

BT - CIKM 2024

Y2 - 21 October 2024 through 25 October 2024

ER -