Abstract
The automatic writer’s recognition from his manuscript is a topical issue handling online writing. Recurrent neural networks (RNNs) are an effective means of solving such problem. More specifically, RNN networks with Long and Short Term Memory (LSTM) represent an ideal mean for writer’s recognition. Intuitively, LSTM networks are based on the gradient method for their learning processes. In addition, an LSTM node presents a complex data processing machine.
Our hybrid approach combining LSTM and PSO (H-PSO-LSTM) presents the purpose of this paper and increases the performance of the network.
Experiments were carried out on a Biometrics Ideal Test (BIT) bilingual database (Chinese and English). The BIT deals with a large number of writers (between 130 and 188). With H-PSO-LSTM, we were able to improve the learning performance accuracy to 91.9% instead of 81.2%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18(5), 602–610 (2005)
Cai, X., Zhang, N., Venayagamoorthy, G.K., Wunsch, D.C.: Time series prediction with recurrent neural networks using a hybrid PSO-EA algorithm. Neurocomputing 70(13), 2342–2353 (2007)
Dhahri, H., Alimi, M.A.: The modified differential evolution and the RBF (MDE-RBF) neural network for time series prediction. In: IJCNN 2006, International Joint Conference on Neural Network, pp. 2938–2943. IEEE, Vancouver (2006)
Boubaker, H., Kherallah, M., Alimi, M.A.: New algorithm of straight or curved baseline detection for short Arabic handwritten writing. In: ICDAR 2009, 10th International Conference on Document Analysis and Recognition, pp. 778–782. IEEE, Barcelona (2009)
Slimane, F., Kanoun, S., Hennebert, J., Alimi, M.A., Ingold, R.: A study on font-family and font-size recognition applied to Arabic word images at ultra-low resolution. Pattern Recogn. Lett. 34(2), 209–218 (2013)
Moussa, S.B., Zahour, A., Benabdelhafid, A., Alimi, M.A.: New features using fractal multi-dimensions for generalized Arabic font recognition. Pattern Recogn. Lett. 31(5), 361–371 (2010)
Bezine, H., Alimi, M.A., Derbel, N.: Handwriting trajectory movements controlled by a beta-elliptic model. In: ICDAR 2003, Proceedings of the International Conference on Document Analysis and Recognition, Scotland, pp. 1228–1232 (2003)
Alimi, M.A.: Evolutionary computation for the recognition of on-line cursive handwriting. IETE J. Res. 48(5), 385–396 (2002)
Baccour, L., Alimi, M.A., John, R.I.: Similarity measures for intuitionistic fuzzy sets: state of the art. J. Intell. Fuzzy Syst. 24(1), 37–49 (2013)
Chen, K., Yan, Z., Huo, Q.: A context-sensitive-chunk BPTT approach to training deep LSTM/BLSTM recurrent neural networks for offline handwriting recognition. In: ICDAR 2015, 13th International Conference on Document Analysis and Recognition, France, pp. 411–415. IEEE (2015)
Gers, F.A., Schmidhuber, E.: LSTM recurrent networks learn simple context-free and context-sensitive languages. IEEE Trans. Neural Netw. 12(6), 1333–1340 (2001)
Zhang, X.Y., Xie, G.S., Liu, C.L., Bengio, Y.: End-to-end online writer identification with recurrent neural network. IEEE Trans. Hum.-Mach. Syst. 74(2), 285–292 (2017)
Elbaati, A., Boubaker, H., Kherallah, M., Ennaji, A., El Abed, H., Alimi, M.A.: Arabic handwriting recognition using restored stroke chronology. In: ICDAR 2009, 10th International Conference on Document Analysis and Recognition, Beijing, China, pp. 411–415. IEEE (2009)
Huang, T.Y., Li, C.J., Hsu, T.W.: Structure and parameter learning algorithm of Jordan type recurrent neural networks. In: IJCNN 2007, International Joint Conference Neural Networks, pp. 1819–1824. IEEE, Barcelona (2007)
Pascanu, R., Mikolov, T., Bengio, Y.: On the difficulty of training recurrent neural networks. In: ICML 2013, International Conference on Machine Learning, Atlanta, pp. 1310–1318 (2013)
Bouaziz, S., Dhahri, H., Alimi, M.A., Abraham, A.: A hybrid learning algorithm for evolving flexible beta basis function neural tree model. Neurocomputing 117, 107–117 (2013)
Zhang, J.R., Zhang, J., Lok, T.M., Lyu, M.R.: A hybrid particle swarm optimization–back-propagation algorithm for feedforward neural network training. Appl. Math. Comput. 185(2), 1026–1037 (2007)
Elloumi, W., Baklouti, N., Abraham, A., Alimi, M.A.: The multi-objective hybridization of Particle Swarm Optimization and fuzzy ant colony optimization. J. Intell. Fuzzy Syst. 27(1), 515–525 (2014)
Elloumi, W., Alimi, M.A.: A more efficient MOPSO for optimization. In: AICCSA 2010, ACS/IEEE International Conference on Computer System and Applications, Tunisia, pp. 1–7 (2010)
Greff, K., Srivastava, R.K., Koutník, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: a search space odyssey. IEEE Trans. Neural Netw. Learn. Syst. (2016)
Graves, A.: Supervised Sequence Labelling with Recurrent Neural Networks. Springer, Heidelberg (2012)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Poli, R., Kennedy, J., Blackwell, T.: Particle Swarm Optimization. Swarm Intell. 1(1), 33–57 (2007)
Bali, O., Elloumi, W., Abraham, A., Alimi, M.A.: GPU PSO and ACO applied to TSP for vehicle security tracking. J. Inf. Assur. Secur. 11(6), 369–384 (2016)
Elloumi, W., Alimi, M.A.: Combinatory optimization of ACO and PSO. In: META 2008, Second International Conference on Metaheuristics and Nature Inspired Computing, Tunisia, pp. 1–8 (2008)
Feng, M., Pan, H.: A modified PSO algorithm based on cache replacement algorithm. In: CIS 2014, Computational Intelligence and Security, China, pp. 558–562 (2014)
Elloumi, W., El Abed, H., Abraham, A., Alimi, M.A.: A comparative study of the improvement of performance using a PSO modified by ACO applied to TSP. Appl. Soft Comput. 25, 234–241 (2014)
Chouikhi, N., Ammar, B., Rokbani, N., Alimi, M.A.: PSO-based analysis of Echo State Network parameters for time series forecasting. Appl. Soft Comput. 55, 211–225 (2017)
Sanjeevi, S.G., Nikhila, A.N., Khan, T., Sumathi, G.: Hybrid PSO-SA algorithm for training a neural network for classification. Int. J. Comput. Sci. Eng. Appl. (IJCSEA 2011) 1(6), 73–83 (2011)
Dhahri, H., Alimi, M.A.: The modified differential evolution and the RBF (MDE-RBF) neural network for time series prediction. In: IJCNN 2006, International Joint Conference on Neural Networks, pp. 2938–2943. IEEE, Vancouver (2006)
Acknowledgment
The research leading to these results has received funding from the Ministry of Higher Education and Scientific Research of Tunisia under the grant agreement number LR11ES48.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Moalla, H., Elloumi, W., Alimi, A.M. (2017). H-PSO-LSTM: Hybrid LSTM Trained by PSO for Online Handwriter Identification. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10637. Springer, Cham. https://doi.org/10.1007/978-3-319-70093-9_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-70093-9_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70092-2
Online ISBN: 978-3-319-70093-9
eBook Packages: Computer ScienceComputer Science (R0)