Abstract
Forecasting methods are affected by data quality issues in two ways: 1. they are hard to predict, and 2. they may affect the model negatively when it is updated with new data. The latter issue is usually addressed by pre-processing the data to remove those issues. An alternative approach has recently been proposed, Corrector LSTM (cLSTM), which is a Read & Write Machine Learning (RW-ML) algorithm that changes the data while learning to improve its predictions. Despite promising results being reported, cLSTM is computationally expensive, as it uses a meta-learner to monitor the hidden states of the LSTM. We propose a new RW-ML algorithm, Kernel Corrector LSTM (KcLSTM), that replaces the meta-learner of cLSTM with a simpler method: Kernel Smoothing. We empirically evaluate the forecasting accuracy and the training time of the new algorithm and compare it with cLSTM and LSTM. Results indicate that it is able to decrease the training time while maintaining a competitive forecasting accuracy.
This work was partially funded by projects AISym4Med (101095387) supported by Horizon Europe Cluster 1: Health, ConnectedHealth (n.\(^{\underline{\text {o}}}\) 46858), supported by Competitiveness and Internationalisation Operational Programme (POCI) and Lisbon Regional Operational Programme (LISBOA 2020), under the PORTUGAL 2020 Partnership Agreement, through the European Regional Development Fund (ERDF); NextGenAI - Center for Responsible AI (2022-C05i0102-02), supported by IAPMEI, and also by FCT plurianual funding for 2020–2023 of LIACC (UIDB/00027/2020_UIDP/00027/2020) and SONAE IM Labs@FEUP.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Baghoussi, Y., Soares, C., Mendes-Moreira, J.: Corrector LSTM: built-in training data correction for improved time series forecasting. In: Proceedings of the 8th SIGKDD International Workshop on Mining and Learning from Time Series–Deep Forecasting: Models, Interpretability, and Applications, Washington DC, USA, pp. 1–8. ACM (2022)
Bailer-Jones, C., MacKay, D.J.C., Withers, P.J.A.: A recurrent neural network for modelling dynamical systems. Network 9(4), 531–47 (1998). https://api.semanticscholar.org/CorpusID:653765
Baptista, A., Baghoussi, Y., Soares, C., Mendes-Moreira, J., Arantes, M.: Pastprop-RNN: improved predictions of the future by correcting the past (2021)
Bhowmik, P., Partha, A.S.: A data-centric approach to improve machine learning model’s performance in production. Int. J. Eng. Adv. Technol. (2021). https://api.semanticscholar.org/CorpusID:240328155
Bowman, S.R., Potts, C., Manning, C.D.: Recursive neural networks can learn logical semantics. In: Workshop on Continuous Vector Space Models and their Compositionality (2014). https://api.semanticscholar.org/CorpusID:15618372
Castro, J., Achanccaray Diaz, P., Sanches, I., Cue La Rosa, L., Nigri Happ, P., Feitosa, R.: Evaluation of recurrent neural networks for crop recognition from multitemporal remote sensing images (2017)
Cerqueira, V., Torgo, L., Mozetič, I.: Evaluating time series forecasting models: an empirical study on performance estimation methods. Mach. Learn. 109(11), 1997–2028 (2020). https://doi.org/10.1007/s10994-020-05910-7
Cerqueira, V., Torgo, L., Soares, C.: Machine learning vs statistical methods for time series forecasting: size matters (2019)
Dave, A., Russakovsky, O., Ramanan, D.: Predictive-corrective networks for action detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2067–2076 (2017). https://api.semanticscholar.org/CorpusID:2466592
Dehaene, S., Changeux, J.P., Nadal, J.P.: Neural networks that learn temporal sequences by selection. Proc. Natl. Acad. Sci. USA 84(9), 2727–31 (1987). https://api.semanticscholar.org/CorpusID:7423734
Denker, J.S.: Neural network models of learning and adaptation. Phys. D Nonlinear Phenom. 2, 216–232 (1986). https://api.semanticscholar.org/CorpusID:119988262
Diebold, F., Mariano, R.: Comparing predictive accuracy. J. Bus. Econ. Stat. 13(3), 253–63 (1995). https://EconPapers.repec.org/RePEc:bes:jnlbes:v:13:y:1995:i:3:p:253-63
Gelenbe, E.: Learning in the recurrent random neural network. Neural Comput. 5, 154–164 (1992). https://api.semanticscholar.org/CorpusID:38667978
Ghahramani, Z.: Probabilistic machine learning and artificial intelligence. Nature 521, 452–459 (2015). https://api.semanticscholar.org/CorpusID:216356
Hochreiter, S.: The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int. J. Uncertain. Fuzziness Knowl.-Based Syst. 6, 107–116 (1998). https://doi.org/10.1142/S0218488598000094
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–80 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
Kanarachos, S., Christopoulos, S.R.G., Chroneos, A., Fitzpatrick, M.E.: Detecting anomalies in time series data via a deep learning algorithm combining wavelets, neural networks and hilbert transform. Expert Syst. Appl. 85, 292–304 (2017). https://doi.org/10.1016/j.eswa.2017.04.028. https://www.sciencedirect.com/science/article/pii/S0957417417302737
Makridakis, S., Spiliotis, E., Assimakopoulos, V.: The m4 competition: results, findings, conclusion and way forward. Int. J. Forecast. 34(4), 802–808 (2018). https://doi.org/10.1016/j.ijforecast.2018.06.001. https://www.sciencedirect.com/science/article/pii/S0169207018300785
Maya, S., Ueno, K., Nishikawa, T.: dLSTM: a new approach for anomaly detection using deep learning with delayed prediction. Int. J. Data Sci. Anal. 8, 137–164 (2019). https://doi.org/10.1007/s41060-019-00186-0
Otte, C.: Safe and interpretable machine learning: a methodological review (2013). https://api.semanticscholar.org/CorpusID:56899177
Sherstinsky, A.: Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Phys. D Nonlinear Phenom. 404, 132306 (2020). https://doi.org/10.1016/j.physd.2019.132306. https://www.sciencedirect.com/science/article/pii/S0167278919305974
Song, C., Ristenpart, T., Shmatikov, V.: Machine learning models that remember too much. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (2017). https://api.semanticscholar.org/CorpusID:2904063
Staudemeyer, R.C., Morris, E.R.: Understanding LSTM – a tutorial into long short-term memory recurrent neural networks (2019)
Strobelt, H., Gehrmann, S., Pfister, H., Rush, A.M.: LSTMVis: a tool for visual analysis of hidden state dynamics in recurrent neural networks (2017)
Zha, D., et al.: Data-centric artificial intelligence: a survey (2023)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tuna, R., Baghoussi, Y., Soares, C., Mendes-Moreira, J. (2024). Kernel Corrector LSTM. In: Miliou, I., Piatkowski, N., Papapetrou, P. (eds) Advances in Intelligent Data Analysis XXII. IDA 2024. Lecture Notes in Computer Science, vol 14642. Springer, Cham. https://doi.org/10.1007/978-3-031-58553-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-58553-1_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-58555-5
Online ISBN: 978-3-031-58553-1
eBook Packages: Computer ScienceComputer Science (R0)