Efficient region of visual interests search for geo-multimedia data | Multimedia Tools and Applications Skip to main content
Log in

Efficient region of visual interests search for geo-multimedia data

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the proliferation of online social networking services and mobile smart devices equipped with mobile communications module and position sensor module, massive amount of multimedia data has been collected, stored and shared. This trend has put forward higher request on massive multimedia data retrieval. In this paper, we investigate a novel spatial query named region of visual interests query (RoVIQ), which aims to search users containing geographical information and visual words. Three baseline methods are presented to introduce how to exploit existing techniques to address this problem. Then we propose the definition of this query and related notions at the first time. To improve the performance of query, we propose a novel spatial indexing structure called quadtree based inverted visual index which is a combination of quadtree, inverted index and visual words. Based on it, we design a efficient search algorithm named region of visual interests search to support RoVIQ. Experimental evaluations on real geo-image datasets demonstrate that our solution outperforms state-of-the-art method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Beckmann N, Kriegel H, Schneider R, Seeger B (1990) The r*-tree: an efficient and robust access method for points and rectangles. In: Proceedings of the 1990 ACM SIGMOD international conference on management of data, Atlantic City, NJ, May 23-25, 1990, pp 322–331

  2. Cao X, Chen L, Cong G, Jensen CS, Qu Q, Skovsgaard A, Wu D, Yiu ML (2012) Spatial keyword querying. In: Conceptual modeling - 31st international conference ER 2012, Florence, Italy, October 15-18, 2012. Proceedings, pp 16–29

  3. Cao X, Cong G, Jensen CS, Ooi BC (2011) Collective spatial keyword querying. In: Proceedings of the ACM SIGMOD international conference on management of data, SIGMOD 2011, Athens, Greece, June 12-16, 2011, pp 373–384

  4. Charfi N, Trichili H, Alimi AM, Solaiman B (2017) Bimodal biometric system for hand shape and palmprint recognition based on SIFT sparse representation. Multimed Tools Appl 76(20):20457–20482

    Article  Google Scholar 

  5. Cong G, Jensen CS, Wu D (2009) Efficient retrieval of the top-k most relevant spatial web objects. PVLDB 2(1):337–348

    Google Scholar 

  6. Deng K, Li X, Lu J, Zhou X (2015) Best keyword cover search. IEEE Trans Knowl Data Eng 27(1):61–73

    Article  Google Scholar 

  7. Faloutsos C (1986) Multiattribute hashing using gray codes. In: ACM SIGMOD international conference on management of data, Washington, DC, pp 227–238

  8. Fan J, Li G, Zhou L, Chen S, Hu J (2012) Seal: spatio-textual similarity search. Proc Vldb Endowment 5(9):824–835

    Article  Google Scholar 

  9. Felipe ID, Hristidis V, Rishe N (2008) Keyword search on spatial databases. In: Proceedings of the 24th international conference on data engineering, ICDE 2008, April 7-12, 2008, Cancu̇n, Mexico, pp 656–665

  10. Gargantini I (1982) An effective way to represent quadtrees. Commun ACM 25(12):905–910

    Article  Google Scholar 

  11. Guo L, Shao J, Aung HH, Tan K (2015) Efficient continuous top-k spatial keyword queries on road networks. GeoInformatica 19(1):29–60

    Article  Google Scholar 

  12. Guo T, Cao X, Cong G (2015) Efficient algorithms for answering the m-closest keywords query. In: Proceedings of the 2015 ACM SIGMOD international conference on management of data, Melbourne, Victoria, Australia, May 31 - June 4, 2015, pp 405–418

  13. Guttman A (1984) R-trees: A dynamic index structure for spatial searching. In: SIGMOD’84, Proceedings of annual meeting, Boston, Massachusetts, June 18-21, 1984, pp 47–57

  14. Hariharan R, Hore B, Li C, Mehrotra S (2007) Processing spatial-keyword (SK) queries in geographic information retrieval (GIR) systems. In: 19th international conference on scientific and statistical database management, SSDBM 2007, 9-11 July 2007, Banff, Canada, Proceedings, p 16

  15. Hunter GM, Steiglitz K (1979) Operations on images using quad trees. IEEE Trans Pattern Anal Mach Intell 1(2):145–153

    Article  Google Scholar 

  16. Jing Y, Baluja S (2008) Visualrank: Applying pagerank to large-scale image search. IEEE Trans Pattern Anal Mach Intell Mach Intell 30(11):1877–1890

    Article  Google Scholar 

  17. Karakasis EG, Amanatiadis A, Gasteratos A, Chatzichristofis SA (2015) Image moment invariants as local features for content based image retrieval using the bag-of-visual-words model. Pattern Recogn Lett 55:22–27

    Article  Google Scholar 

  18. Ke Y, Sukthankar R (2004) PCA-SIFT: A more distinctive representation for local image descriptors. In: 2004 IEEE computer society conference on computer vision and pattern recognition (CVPR 2004), with CD-ROM, 27 June - 2 July 2004, Washington, DC, USA, pp 506–513

  19. Lee KCK, Lee W, Zheng B, Tian Y (2012) ROAD: A new spatial object search framework for road networks. IEEE Trans Knowl Data Eng 24(3):547–560. https://doi.org/10.1109/TKDE.2010.243

    Article  Google Scholar 

  20. Lew MS, Sebe N, Djeraba C, Jain R (2006) Content-based multimedia information retrieval: State of the art and challenges. TOMCCAP 2(1):1–19

    Article  Google Scholar 

  21. Li Z, Lee KCK, Zheng B, Lee W, Lee DL, Wang X (2011) Ir-tree: an efficient index for geographic document search. IEEE Trans Knowl Data Eng 23(4):585–599

    Article  Google Scholar 

  22. Liu Y, Liu S, Wang Z (2015) Multi-focus image fusion with dense SIFT. Inf Fusion 23:139–155

    Article  Google Scholar 

  23. Lowe DG (1999) Object recognition from local scale-invariant features. In: ICCV, pp 1150–1157

  24. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  25. Mortensen EN, Deng H, Shapiro LG (2005) A SIFT descriptor with global context. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR 2005), 20-26 June 2005, San Diego, CA, USA, pp 184–190

  26. Morton GM (2015) A computer oriented geodetic data base and a new technique in file sequencing. Phys Plasmas 24(7):159–173

    Google Scholar 

  27. Nam S, Kim W, Mun S, Hou J, Choi S, Lee H (2018) A SIFT features based blind watermarking for DIBR 3d images. Multimed Tools Appl 77(7):7811–7850

    Article  Google Scholar 

  28. Rocha-Junior JB, Gkorgkas O, Jonassen S, Nørvåg K (2011) Efficient processing of top-k spatial keyword queries. In: Advances in spatial and temporal databases - 12th international symposium, SSTD 2011, Minneapolis, MN, USA, August 24-26, 2011, Proceedings, pp 205–222

  29. Rocha-Junior JB, Nørvåg K (2012) Top-k spatial keyword queries on road networks. In: 15th international conference on extending database technology, EDBT ’12, Berlin, Germany, March 27-30, 2012, Proceedings, pp 168–179

  30. dos Santos JM, de Moura ES, da Silva AS, da Silva Torres R (2017) Color and texture applied to a signature-based bag of visual words method for image retrieval. Multimed Tools Appl 76(15):16855–16872

    Article  Google Scholar 

  31. Sivic J, Zisserman A (2003) Video google: a text retrieval approach to object matching in videos. In: 9th IEEE international conference on computer vision (ICCV 2003), 14-17 October 2003, Nice, France, pp 1470–1477

  32. Su M, Ma Y, Zhang X, Wang Y, Zhang Y (2017) MBR-SIFT: A mirror reflected invariant feature descriptor using a binary representation for image matching. Plos One 12 (5):1–16. https://doi.org/10.1371/journal.pone.0178090

    Article  Google Scholar 

  33. Wan J, Wang D, Hoi SC, Wu P, Zhu J, Zhang Y, Li J (2014) Deep learning for content-based image retrieval: A comprehensive study. In: Proceedings of the ACM international conference on multimedia, MM ’14, Orlando, FL, USA, November 03 - 07, 2014, pp 157–166

  34. Wang F, Wang H, Li H, Zhang S (2013) Large scale image retrieval with practical spatial weighting for bag-of-visual-words. In: Advances in multimedia modeling, 19th international conference, MMM 2013, Huangshan, China, January 7-9, 2013, Proceedings, Part I, pp 513–523

    Google Scholar 

  35. Wang X, Zhang Y, Zhang W, Lin X, Wang W (2015) Ap-tree: efficiently support continuous spatial-keyword queries over stream. In: 31st IEEE international conference on data engineering, ICDE 2015, Seoul, South Korea, April 13-17, 2015, pp 1107–1118

  36. Wang Y, Lin X, Wu L, Zhang W (2015) Effective multi-query expansions: Robust landmark retrieval. In: Proceedings of the 23rd annual ACM conference on multimedia conference, MM ’15, Brisbane, Australia, October 26 - 30, 2015, pp 79–88

  37. Wang Y, Lin X, Wu L, Zhang W (2017) Effective multi-query expansions: Collaborative deep networks for robust landmark retrieval. IEEE Trans Image Process 26(3):1393–1404

    Article  MathSciNet  Google Scholar 

  38. Wang Y, Lin X, Wu L, Zhang W, Zhang Q (2014) Exploiting correlation consensus: towards subspace clustering for multi-modal data. In: Proceedings of the ACM international conference on multimedia, MM ’14, Orlando, FL, USA, November 03 - 07, 2014, pp 981–984

  39. Wang Y, Lin X, Wu L, Zhang W, Zhang Q (2015) LBMCH: learning bridging mapping for cross-modal hashing. In: Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, Santiago, Chile, August 9-13, 2015, pp 999–1002

  40. Wang Y, Lin X, Wu L, Zhang W, Zhang Q, Huang X (2015) Robust subspace clustering for multi-view data by exploiting correlation consensus. IEEE Trans Image Process 24(11):3939–3949

    Article  MathSciNet  Google Scholar 

  41. Wang Y, Lin X, Zhang Q (2013) Towards metric fusion on multi-view data: a cross-view based graph random walk approach. In: 22nd ACM international conference on information and knowledge management, CIKM’13, San Francisco, CA, USA, October 27 - November 1, 2013, pp 805–810

  42. Wang Y, Lin X, Zhang Q, Wu L (2014) Shifting hypergraphs by probabilistic voting. In: Advances in knowledge discovery and data mining - 18th Pacific-Asia conference, PAKDD 2014, Tainan, Taiwan, May 13-16, 2014. Proceedings, Part II, pp 234–246

    Chapter  Google Scholar 

  43. Wang Y, Wu L (2018) Beyond low-rank representations: Orthogonal clustering basis reconstruction with optimized graph structure for multi-view spectral clustering. Neural Netw 103:1–8

    Article  Google Scholar 

  44. Wang Y, Wu L, Lin X, Gao J (2018) Multiview spectral clustering via structured low-rank matrix factorization. IEEE Trans Neural Networks and Learning Systems

  45. Wang Y, Zhang W, Wu L, Lin X, Fang M, Pan S (2016) Iterative views agreement: An iterative low-rank based structured optimization method to multi-view spectral clustering. In: Proceedings of the 25th international joint conference on artificial intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016, pp 2153–2159

  46. Wang Y, Zhang W, Wu L, Lin X, Zhao X (2017) Unsupervised metric fusion over multiview data by graph random walk-based cross-view diffusion. IEEE Trans Neural Netw Learn Syst 28(1):57–70

    Article  Google Scholar 

  47. Wu L, Wang Y (2017) Robust hashing for multi-view data: jointly learning low-rank kernelized similarity consensus and hash functions. Image Vision Comput 57:58–66

    Article  Google Scholar 

  48. Wu L, Wang Y, Gao J, Li X (2018) Deep adaptive feature embedding with local sample distributions for person re-identification. Pattern Recogn 73:275–288

    Article  Google Scholar 

  49. Wu L, Wang Y, Gao J, Li X (2018) Where-and-when to look: deep siamese attention networks for video-based person re-identification. arXiv:1808.01911

  50. Wu L, Wang Y, Ge Z, Hu Q, Li X (2018) Structured deep hashing with convolutional neural networks for fast person re-identification. Comput Vis Image Underst 167:63–73

    Article  Google Scholar 

  51. Wu L, Wang Y, Li X, Gao J (2018) Deep attention-based spatially recursive networks for fine-grained visual recognition. IEEE Trans Cybernetics

  52. Wu L, Wang Y, Li X, Gao J (2018) What-and-where to match: deep spatially multiplicative integration networks for person re-identification. Pattern Recogn 76:727–738

    Article  Google Scholar 

  53. Wu L, Wang Y, Shao L (2018) Cycle-consistent deep generative hashing for cross-modal retrieval. arXiv:1804.11013

  54. Wu L, Wang Y, Shepherd J (2013) Efficient image and tag co-ranking: a bregman divergence optimization method. In: ACM multimedia conference, MM ’13, Barcelona, Spain, October 21-25, 2013, pp 593–596

  55. Zhang C, Zhang Y, Zhang W, Lin X (2016) Inverted linear quadtree: efficient top K spatial keyword search. IEEE Trans Knowl Data Eng 28(7):1706–1721

    Article  Google Scholar 

  56. Zhang C, Zhang Y, Zhang W, Lin X, Cheema M A, Wang X (2014) Diversified spatial keyword search on road networks. In: Proceedings of the 17th international conference on extending database technology, EDBT 2014, Athens, Greece, March 24-28, 2014., pp 367–378

  57. Zhang D, Tan K, Tung A K H (2013) Scalable top-k spatial keyword search. In: Joint 2013 EDBT/ICDT conferences, EDBT ’13 Proceedings, Genoa, Italy, March 18-22, 2013, pp 359–370

  58. Zheng K, Su H, Zheng B, Shang S, Xu J, Liu J, Zhou X (2015) Interactive top-k spatial keyword queries. In: 31st IEEE international conference on data engineering, ICDE 2015, Seoul, South Korea, April 13-17, 2015, pp 423–434

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (61702560), project (2018JJ3691, 2016JC2011) of Science and Technology Plan of Hunan Province, and the Research and Innovation Project of Central South University Graduate Students(2018zzts177,2018zzts588).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Lei Zhu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, C., Lin, Y., Zhu, L. et al. Efficient region of visual interests search for geo-multimedia data. Multimed Tools Appl 78, 30839–30863 (2019). https://doi.org/10.1007/s11042-018-6750-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6750-6

Keywords

Navigation