Abstract
Time series are usually complicated in nature and contains many complex patterns. As such, many researchers work on different ways to pick up such patterns. In this paper, we explore using Residual Networks (a Convolutional Neural Network) as a feature extractor for Oblique Random Forest. Here, we extract features using Residual Networks, and pass the extracted feature set to Oblique Random Forest for classification of time series. Based on the experiments on 85 UCR datasets, we found that using features extracted from Residual Network significantly improves the performance of Oblique Random Forest. In addition, using including intermediate features from Residual Networks significantly improves the performance of Oblique Random Forests.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bagnall, A., Lines, J., Bostrom, A., Large, J., Keogh, E.: The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances. Data Min. Knowl. Disc. 31(3), 606–660 (2016). https://doi.org/10.1007/s10618-016-0483-9
Baydogan, M.G., Runger, G., Tuv, E.: A bag-of-features framework to classify time series. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2796–2802 (2013)
Bostrom, A., Bagnall, A.: Binary shapelet transform for multiclass time series classification. In: Hameurlain, A., Küng, J., Wagner, R., Madria, S., Hara, T. (eds.) Transactions on Large-Scale Data- and Knowledge-Centered Systems XXXII. LNCS, vol. 10420, pp. 24–46. Springer, Heidelberg (2017). https://doi.org/10.1007/978-3-662-55608-5_2
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Chen, Y., et al.: The UCR time series classification archive, July 2015. www.cs.ucr.edu/~eamonn/time_series_data/
Deng, H., Runger, G., Tuv, E., Vladimir, M.: A time series forest for classification and feature extraction. Inf. Sci. 239, 142–153 (2013). https://doi.org/10.1016/j.ins.2013.02.030
Gamboa, J.C.B.: Deep learning for time-series analysis. arXiv preprint arXiv:1701.01887 (2017)
Górecki, T., Łuczak, M.: Using derivatives in time series classification. Data Min. Knowl. Disc. 26(2), 310–331 (2013)
Grabocka, J., Schilling, N., Wistuba, M., Schmidt-Thieme, L.: Learning time-series shapelets. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2014, pp. 392–401. Association for Computing Machinery, New York (2014). https://doi.org/10.1145/2623330.2623613
Gãrecki, T., Åuczak, M.: Non-isometric transforms in time series classification using DTW. Knowl. Based Syst. 61, 98–108 (2014). https://doi.org/10.1016/j.knosys.2014.02.011
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778, June 2016
Hills, J., Lines, J., Baranauskas, E., Mapp, J., Bagnall, A.: Classification of time series by shapelet transformation. Data Min. Knowl. Disc. 28(4), 851–881 (2013). https://doi.org/10.1007/s10618-013-0322-1
Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)
Karim, F., Majumdar, S., Darabi, H., Chen, S.: Lstm fully convolutional networks for time series classification. IEEE Access 6, 1662–1669 (2018)
Katuwal, R., Suganthan, P.N.: Enhancing multi-class classification of random forest using random vector functional neural network and oblique decision surfaces. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2018)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Lin, M., Chen, Q., Yan, S.: Network in network. arXiv preprint arXiv:1312.4400 (2013)
Lines, J., Bagnall, A.: Time series classification with ensembles of elastic distance measures. Data Min. Knowl. Disc. 29(3), 565–592 (2014). https://doi.org/10.1007/s10618-014-0361-2
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440, June 2015
Ma, Q., Zhuang, W., Shen, L., Cottrell, G.W.: Time series classification with echo memory networks. Neural Networks 117, 225–239 (2019). https://doi.org/10.1016/j.neunet.2019.05.008
Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), pp. 807–814 (2010)
Nweke, H.F., Teh, Y.W., Al-garadi, M.A., Alo, U.R.: Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: state of the art and research challenges. Expert Syst. Appl. 105, 233–261 (2018). https://doi.org/10.1016/j.eswa.2018.03.056
Rajkomar, A., et al.: Scalable and accurate deep learning with electronic health records. NPJ. Digital Med. 1(1), 18 (2018)
Schäfer, P.: The boss is concerned with time series classification in the presence of noise. Data Min. Knowl. Disc. 29(6), 1505–1530 (2015)
Susto, G.A., Cenedese, A., Terzi, M.: Chapter 9 - time-series classification methods: Review and applications to power systems data. In: Arghandeh, R., Zhou, Y. (eds.) Big Data Application in Power Systems, pp. 179–220. Elsevier (2018). https://doi.org/10.1016/B978-0-12-811968-6.00009-7
Ho, T.K.: The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20(8), 832–844 (1998)
Wang, J., Chen, Y., Hao, S., Peng, X., Hu, L.: Deep learning for sensor-based activity recognition: a survey. Pattern Recogn. Lett. 119, 3–11 (2019). https://doi.org/10.1016/j.patrec.2018.02.010, deep Learning for Pattern Recognition
Wang, Z., Yan, W., Oates, T.: Time series classification from scratch with deep neural networks: a strong baseline. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 1578–1585, May 2017. https://doi.org/10.1109/IJCNN.2017.7966039
YANG, Q., WU, X.: 10 challenging problems in data mining research. Int. J. Inf. Technol. Decis. Making 05(04), 597–604 (2006). https://doi.org/10.1142/S0219622006002258
Zhang, L., Suganthan, P.N.: Oblique decision tree ensemble via multisurface proximal support vector machine. IEEE Trans. Cybern. 45(10), 2165–2176 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Cheng, W.X., Suganthan, P.N., Katuwal, R. (2020). Oblique Random Forests on Residual Network Features. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12534. Springer, Cham. https://doi.org/10.1007/978-3-030-63836-8_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-63836-8_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63835-1
Online ISBN: 978-3-030-63836-8
eBook Packages: Computer ScienceComputer Science (R0)