Abstract
In the framework of bag-of-visual-words, visual words are independent each other, which results in discarding spatial relations and lacking semantic information of visual words. To capture semantic information of visual words, a deep learning procedure similar to word embedding technique is used for mapping visual words to embedding vectors in a semantic space. And then, word mover’s distance (WMD) is utilized to measure similarity between two word images, which calculates the minimum traveling distance from the visual embeddings of one word image to another one. Moreover, word images are partitioned into several sub-regions with equal sizes along rows and columns in advance. After that, WMDs can be computed from the corresponding sub-regions of the two word images, separately. Thus, the similarity between the two word images is the sum of these WMDs. Experimental results show that the proposed method outperforms various baseline and state-of-the-art methods, including spatial pyramid matching, latent Dirichlet allocation, average visual word embeddings and the original word mover’s distance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Rath, T.M., Manmatha, R.: Word spotting for historical manuscripts. Int. J. Doc. Anal. Recogn. 9(2), 139–152 (2007)
Rath, T.M., Manmatha, R.: Features for word spotting in historical manuscripts. In: Proceedings of ICDAR 2003, pp. 218–222. IEEE Press, New York (2003)
Rath, T.M., Manmatha, R.: Word image matching using dynamic time warping. In: Proceedings of CVPR 2003, pp. 521–527. IEEE Press, New York (2003)
Shekhar, R., Jawahar, C.V.: Word image retrieval using bag of visual words. In: Proceedings of DAS 2012, pp. 297–301. IEEE Press, New York (2012)
Aldavert, D., Rusinol, M., Toledo, R., Llados, J.: A study of bag-of-visual-words representations for handwritten keyword spotting. Int. J. Doc. Anal. Recogn. 18(3), 223–234 (2015)
Mikolov, T., Sutskever, I., Chen, K., Coorado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of NIPS 2013, pp. 3111–3119. MIT Press, Massachusetts (2013)
Kusner, M.J., Sun, Y., Kolkin, N.I., Weinberger, K.Q.: From word embeddings to document distances. Proc. Mach. Learn. Res. 37, 957–966 (2015)
Fornes, A., Frinken, V., Fischer, A., Almazan, J., Jackson, G., Bunke, H.: A keyword spotting approach using blurred shape model-based descriptors. In: Proceedings of HIP 2011, pp. 83–89. ACM Press, New York (2011)
Aldavert, D., Rusinol, M., Toledo, R., Llados, J.: Integrating visual and textual cues for query-by-string word spotting. In: Proceedings of ICDAR 2013, pp. 511–515. IEEE Press, New York (2013)
Rothacker, L., Fink, G.A.: Segmentation-free query-by-string word spotting with bag-of-features HMMs. In: Proceedings of ICDAR 2015, pp. 661–665. IEEE Press, New York (2015)
Wei, H.X., Gao, G.L., Su, X.D.: A multiple instances approach to improving keyword spotting on historical Mongolian document images. In: Proceedings of ICDAR 2015, pp. 121–125. IEEE Press, New York (2015)
Wei, H.X., Zhang, H., Gao, G.L.: Representing word image using visual word embeddings and RNN for keyword spotting on historical document images. In: Proceedings of ICME 2017, pp. 1374–1379. IEEE Press, New York (2017)
Wei, H.X., Gao, G.L.: Visual language model for keyword spotting on historical Mongolian document images. In: Proceedings of CCDC 2017, pp. 1765–1770. IEEE Press, New York (2017)
Wei, H., Gao, G., Su, X.: LDA-based word image representation for keyword spotting on historical Mongolian documents. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds.) ICONIP 2016. LNCS, vol. 9950, pp. 432–441. Springer, Cham (2016). doi:10.1007/978-3-319-46681-1_52
Zamani, H., Croft, W.B.: Embeddings-based query language models. In: Proceedings of ICTIR 2016, pp. 147–156. ACM Press, New York (2016)
Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Proceedings of EMNLP 2014, pp. 1532–1543. ACL Press, Stroudsburg (2014)
Nalisnick, E., Mitra, B., Craswell, N., Caruana, R.: Improving document ranking with dual word embeddings. In: Proceedings of WWW 2016, pp. 83–84. ACM Press, New York (2016)
Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of CVPR 2006, pp. 2169–2178. IEEE Press, New York (2006)
Acknowledgement
This paper is supported by the National Natural Science Foundation of China under Grant 61463038.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Wei, H., Zhang, H., Gao, G., Su, X. (2017). Using Word Mover’s Distance with Spatial Constraints for Measuring Similarity Between Mongolian Word Images. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10637. Springer, Cham. https://doi.org/10.1007/978-3-319-70093-9_20
Download citation
DOI: https://doi.org/10.1007/978-3-319-70093-9_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70092-2
Online ISBN: 978-3-319-70093-9
eBook Packages: Computer ScienceComputer Science (R0)