Abstract
This paper presents a fusion framework for air-writing recognition. By modeling a hand trajectory using both spatial and temporal features, the proposed network can learn more information than the state-of-the-art techniques. The proposed network combines elements of CNN and BLSTM networks to learn the isolated air-writing characters. The performance of proposed network was evaluated by the alphabet and numeric databases in the public dataset namely 6DMG. We first evaluate the accuracy of fusion network using CNN, BLSTM, and another fusion network as the references. The results confirmed that the average accuracy of fusion network outperforms all of the references. When the BLSTM unit was set at 40, the best accuracy of proposed network is 99.27% and 99.33% in the alphabet and numeric gesture, respectively. When compared this result with another work, the accuracy of proposed network improves 0.70% and 0.34% in the alphabet and numeric gesture, respectively. We also examine the performance of the proposed network by varying the number of BLSTM units. The experiments demonstrate that while increasing the number of BLSTM units, the accuracy also improves. When the BLSTM unit is greater than 20, the accuracy maintains even though the BLSTM unit increases. Despite adding more learning features, the accuracy of proposed network insignificantly improves.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
LeCun, Y.: Neural networks and gradient-based learning in OCR. In: Proceedings of the 1997 IEEE Workshop Neural Networks for Signal Processing, USA, p. 255, September 1997
Hu, J.T., Fan, C.X., Ming, Y.: Trajectory image based dynamic gesture recognition with convolutional neural networks. In: 2015 15th International Conference on Control, Automation and Systems, Korea, pp. 1885–1889, October 2015
Xu, S., Xue, Y.: Air-writing characters modelling and recognition on modified CHMM. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics, Hungary, pp. 001510–001513, October 2016
Agarwal, C., Dogra, D.P., Saini, R., Roy, P.P.: Segmentation and recognition of text written in 3D using Leap motion interface. In: 2015 3rd IAPR Asian Conference on Pattern Recognition, Malaysia, pp. 539–543, November 2015
Hameed, M.Z., Garcia-Hernando, G.: Novel spatio-temporal features for fingertip writing recognition in egocentric viewpoint. In: 2015 14th IAPR International Conference on Machine Vision Applications, Japan, pp. 484–488, May 2015
Hsu, Y.L., Chu, C.L., Tsai, Y.J., Wang, J.S.: An inertial pen with dynamic time warping recognizer for handwriting and gesture recognition. IEEE Sens. J. 15(1), 154–163 (2015)
Yang, C., Ku, B., Han, D.K., Ko, H.: Alpha-numeric hand gesture recognition based on fusion of spatial feature modelling and temporal feature modelling. Electron. Lett. 52(20), 1679–1681 (2016)
Chen, M., AlRegib, G., Juang, B.: 6DMG: a new 6D motion gesture database. In: Proceedings of the 3rd Multimedia Systems Conference, USA, pp. 83–88, February 2012
Ma, L., Zhang, J., Wang, J.: Modified CRF algorithm for dynamic hand gesture recognition. In: 2014 33rd Chinese Control Conference, China, pp. 4763–4767, July 2014
Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM networks. In: 2005 IEEE International Joint Conference on Neural Networks, Canada, vol. 4, pp. 2047–2052, August 2005
Frinken, V., Uchida, S.: Deep BLSTM neural networks for unconstrained continuous handwritten text recognition. In: ICDAR 2015 Proceedings of the 2015 13th International Conference on Document Analysis and Recognition, USA, pp. 911–915, August 2015
Zhang, X.Y., Yin, F., Zhang, Y.M., Liu, C.L., Bengio, Y.: Drawing and recognizing Chinese characters with recurrent neural network. Computer Vision Pattern Recognition arXiv:1606.06539, June 2016
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Holzinger, A., Stocker, C., Peischl, B., Simonic, K.M.: On using entropy for enhancing handwriting preprocessing. Entropy 14(11), 2324–2350 (2012)
Jaeger, S., Manke, S., Reichert, J., Waibel, A.: Online handwriting recognition: the NPen++ recognizer. Int. J. Doc. Anal. Recogn. 3(3), 169–180 (2001)
Graves, A., Fernández, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: The 23rd International Conference on Machine Learning, New York, USA, pp. 369–376, June 2006
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Yana, B., Onoye, T. (2018). Fusion Networks for Air-Writing Recognition. In: Schoeffmann, K., et al. MultiMedia Modeling. MMM 2018. Lecture Notes in Computer Science(), vol 10705. Springer, Cham. https://doi.org/10.1007/978-3-319-73600-6_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-73600-6_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73599-3
Online ISBN: 978-3-319-73600-6
eBook Packages: Computer ScienceComputer Science (R0)