Abstract
Recurrent and convolutional neural networks are the most common architectures used for time-series forecasting in deep learning literature. Owing to parameter sharing and repeating architecture, these models are time-invariant (shift-invariant in the spatial domain). We demonstrate how time-invariance in such models can reduce the capacity to model time-varying dynamics in the data. We propose ForecastNet which uses a deep feed-forward architecture and interleaved outputs to provide a time-variant model. ForecastNet is demonstrated to model time varying dynamics in data and outperform statistical and deep learning benchmark models on several seasonal time-series datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Code is available at https://github.com/jjdabr/forecastNet.
- 2.
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: Proceedings of the International Conference on Learning Representations (2015). http://arxiv.org/abs/1409.0473
Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018)
Bishop, C.M.: Mixture density networks. Technical report NCRG94004, Aston University (1994)
Chang, S., et al.: Dilated recurrent neural networks. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30, pp. 77–87. Curran Associates, Inc. (2017). http://papers.nips.cc/paper/6613-dilated-recurrent-neural-networks.pdf
Dabrowski, J.J., Rahman, A., George, A., Arnold, S., McCulloch, J.: State space models for forecasting water quality variables: an application in aquaculture prawn farming. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and #38; Data Mining. (KDD 2018), pp. 177–185, ACM, New York (2018). https://doi.org/10.1145/3219819.3219841
Du, S., Li, T., Yang, Y., Horng, S.J.: Multivariate time series forecasting via attention-based encoder-decoder framework. Neurocomputing (2020). https://doi.org/10.1016/j.neucom.2019.12.118, http://www.sciencedirect.com/science/article/pii/S0925231220300606
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1026–1034, December 2015. https://doi.org/10.1109/ICCV.2015.123
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385 (2015)
Hewamalage, H., Bergmeir, C., Bandara, K.: Recurrent neural networks for time series forecasting: current status and future directions. arXiv preprint arXiv:1909.00590 (2019)
Mariet, Z., Kuznetsov, V.: Foundations of sequence-to-sequence modeling for time series. In: Chaudhuri, K., Sugiyama, M. (eds.) Proceedings of Machine Learning Research. Proceedings of Machine Learning Research, vol. 89, pp. 408–417. PMLR, 16–18 April 2019. http://proceedings.mlr.press/v89/mariet19a.html
Oord, A.V.D., et al.: WaveNet: a generative model for raw audio. arXiv preprint arXiv:1609.03499 (2016)
Oppenheim, A.V., Schafer, R.W.: Discrete-Time Signal Processing. Pearson education signal processing series, 3rd edn. Pearson, London (2009)
Salinas, D., Flunkert, V., Gasthaus, J., Januschowski, T.: DeepAR: probabilistic forecasting with autoregressive recurrent networks. Int. J. Forecast. (2019). https://doi.org/10.1016/j.ijforecast.2019.07.001, http://www.sciencedirect.com/science/article/pii/S0169207019301888
Srivastava, R.K., Greff, K., Schmidhuber, J.: Training very deep networks. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems 28, pp. 2377–2385. Curran Associates, Inc. (2015). http://papers.nips.cc/paper/5850-training-very-deep-networks.pdf
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 3104–3112. Curran Associates, Inc. (2014). http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf
Vaswani, A., et al.: Attention is all you need. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems 30, pp. 5998–6008. Curran Associates, Inc. (2017). http://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf
West, M., Harrison, J.: Bayesian Forecasting and Dynamic Models. Springer Series in Statistics, Springer, New York (1997). https://doi.org/10.1007/0-387-22777-6
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Dabrowski, J.J., Zhang, Y., Rahman, A. (2020). ForecastNet: A Time-Variant Deep Feed-Forward Neural Network Architecture for Multi-step-Ahead Time-Series Forecasting. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12534. Springer, Cham. https://doi.org/10.1007/978-3-030-63836-8_48
Download citation
DOI: https://doi.org/10.1007/978-3-030-63836-8_48
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63835-1
Online ISBN: 978-3-030-63836-8
eBook Packages: Computer ScienceComputer Science (R0)