Abstract
Existing key-word based image search engines return images whose title or immediate surrounding text contains the search term as a keyword. When the search term is ambiguous and means different things, the results often come in a mixed bag of different entities. This paper proposes a novel framework that understands the context and thus infers the most likely entity in the given image by disambiguating the terms in the context into the corresponding concepts from external knowledge in a process called conceptualization. The images can subsequently be clustered by the most likely associated entities. This approach outperforms the best competing image clustering techniques by 29.2% in NMI score. In addition, the framework automatically annotates each cluster of images by its key entities which allows users to quickly identify the images they want.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Alcic, S., Conrad, S.: Measuring performance of web image context extraction. In: MDMKDD, vol. 8, p. 8 (2010)
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)
Cai, D., He, X., Ma, W.Y., Wen, J.R., Zhang, H.: Organizing www images based on theanalysis of page layout and web link structure. In: ICME, pp. 113–116 (2004)
Cai, D., Yu, S., Wen, J.R., Ma, W.Y.: VIPS: a vision-based page segmentation algorithm. In: Microsoft Technical Report, MSR-TR-2003-79 (2003)
Cai, Z., Zhao, K., Zhu, K.Q., Wang, H.: Wikification via link co-occurrence. In: CIKM, CIKM 2013, pp. 1087–1096 (2013)
Ding, H., Liu, J., Lu, H.: Hierarchical clustering-based navigation of image search results. In: MM, pp. 741–744 (2008)
Fan, J., Gao, Y., Luo, H.: Hierarchical classification for automatic image annotation. In: SIGIR, pp. 111–118 (2007)
Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research 9, 1871–1874 (2008)
Feng, H., Shi, R., Chua, T.S.: A bootstrapping framework for annotating and retrieving www images. In: MM, pp. 960–967 (2004)
Fergus, R., Li, F.F., Perona, P., Zisserman, A.: Learning object categories from google’s image search. In: ICCV, pp. 1816–1823 (2005)
Fu, Z., Ip, H.H.S., Lu, H., Lu, Z.: Multi-modal constraint propagation for heterogeneous image clustering. In: MM, pp. 143–152 (2011)
Gao, B., Liu, T.Y., Qin, T., Zheng, X., Cheng, Q., Ma, W.Y.: Web image clustering by consistent utilization of visual features and surrounding texts. In: MM, pp. 112–121 (2005)
Gao, Y., Fan, J., Luo, H., Satoh, S.: A novel approach for filtering junk images from google search results. In: MMM, pp. 1–12 (2008)
Jing, F., Wang, C., Yao, Y., Deng, K., Zhang, L., Ma, W.Y.: IGroup: web image search results clustering. In: MM, pp. 377–384 (2006)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)
van Leuken, R.H., Pueyo, L.G., Olivares, X., van Zwol, R.: Visual diversification of image search results. In: WWW, pp. 341–350 (2009)
Li, L.J., Socher, R., Li, F.F.: Towards total scene understanding: Classification, annotation and segmentation in an automatic framework. In: CVPR, pp. 2036–2043 (2009)
Lowe, D.G.: Object recognition from local scale-invariant features. In: ICCV, pp. 1150–1157 (1999)
Song, Y., Wang, H., Wang, Z., Li, H., Chen, W.: Short text conceptualization using a probabilistic knowledgebase. In: IJCAI (2011)
Suchanek, F.M., Kasneci, G., Weikum, G.: YAGO: a core of semantic knowledge. In: WWW, pp. 697–706 (2007)
Taneva, B., Kacimi, M., Weikum, G.: Gathering and ranking photos of named entities with high precision, high recall, and diversity. In: WSDM, pp. 431–440 (2010)
Taneva, B., Kacimi, M., Weikum, G.: Finding images of difficult entities in the long tail. In: CIKM, CIKM 2011, pp. 189–194 (2011)
Tsai, D., Jing, Y., Liu, Y., Rowley, H., Ioffe, S., Rehg, J.: Large-scale image annotation using visual synset. In: ICCV, pp. 611–618 (2011)
Wang, X.J., Ma, W.Y., Zhang, L., Li, X.: Iteratively clustering web images based on link and attribute reinforcements. In: MM, pp. 122–131 (2005)
Yu, H., Li, M., Zhang, H.J., Feng, J.: Color texture moments for content-based image retrieval. In: International Conference on Image Processing, pp. 24–28 (2003)
Zeng, H.J., He, Q.C., Chen, Z., Ma, W.Y., Ma, J.: Learning to cluster web search results. In: SIGIR, pp. 210–217 (2004)
Zhong, S., Liu, Y., Liu, Y.: Bilinear deep learning for image classification. In: MM, pp. 343–352 (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhao, K., Cai, Z., Sui, Q., Wei, E., Zhu, K.Q. (2014). Clustering Image Search Results by Entity Disambiguation. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2014. Lecture Notes in Computer Science(), vol 8726. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44845-8_24
Download citation
DOI: https://doi.org/10.1007/978-3-662-44845-8_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-44844-1
Online ISBN: 978-3-662-44845-8
eBook Packages: Computer ScienceComputer Science (R0)