The Emerging Field of Graph Signal Processing for Moving Object Segmentation | SpringerLink
Skip to main content
Springer Nature Link
Log in

The Emerging Field of Graph Signal Processing for Moving Object Segmentation

  • Conference paper
  • First Online:
Frontiers of Computer Vision (IW-FCV 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1405))

Included in the following conference series:

Abstract

Moving Object Segmentation (MOS) is an important topic in computer vision. MOS becomes a challenging problem in the presence of dynamic background and moving camera videos such as Pan–Tilt–Zoom cameras (PTZ). The MOS problem has been solved using unsupervised and supervised learning strategies. Recently, new ideas to solve MOS using semi-supervised learning have emerged inspired from the theory of Graph Signal Processing (GSP). These new algorithms are usually composed of several steps including: segmentation, background initialization, features extraction, graph construction, graph signal sampling, and a semi-supervised learning algorithm inspired from reconstruction of graph signals. In this work, we summarize and explain the theoretical foundations as well as the technical details of MOS using GPS. We also propose two architectures for MOS using semi-supervised learning and a new evaluation procedure for GSP-based MOS algorithms. GSP-based algorithms are evaluated in the Change Detection (CDNet2014) dataset for MOS, outperforming numerous State-Of-The-Art (SOTA) methods in several challenging conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 5719
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 7149
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/jhonygiraldo/GraphMOS.

  2. 2.

    https://sites.google.com/view/gsp-website.

References

  1. Achanta, R., et al.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)

    Article  Google Scholar 

  2. Anis, A., Gadde, A., Ortega, A.: Efficient sampling set selection for bandlimited graph signals using graph spectral proxies. IEEE Trans. Signal Process. 64(14), 3775–3789 (2016)

    Article  MathSciNet  Google Scholar 

  3. Bianco, S., Ciocca, G., Schettini, R.: Combination of video change detection algorithms by genetic programming. IEEE Trans. Evol. Comput. 21(6), 914–928 (2017)

    Article  Google Scholar 

  4. Bouwmans, T.: Traditional and recent approaches in background modeling for foreground detection: an overview. Comput. Sci. Rev. 11, 31–66 (2014)

    Article  Google Scholar 

  5. Bouwmans, T., El Baf, F., Vachon, B.: Background modeling using mixture of Gaussians for foreground detection-a survey. Recent Patents Comput. Sci. 1(3), 219–237 (2008)

    Article  Google Scholar 

  6. Bouwmans, T., Zahzah, E.H.: Robust PCA via principal component pursuit: a review for a comparative evaluation in video surveillance. Comput. Vis. Image Underst. 122, 22–34 (2014)

    Article  Google Scholar 

  7. Bouwmans, T., et al.: Decomposition into low-rank plus additive matrices for background/foreground separation: a review for a comparative evaluation with a large-scale dataset. Comput. Sci. Rev. 23, 1–71 (2017)

    Article  Google Scholar 

  8. Bouwmans, T., et al.: Deep neural network concepts for background subtraction: a systematic review and comparative evaluation. Neural Netw. 117, 8–66 (2019)

    Article  Google Scholar 

  9. Braham, M., Piérard, S., Van Droogenbroeck, M.: Semantic background subtraction. In: IEEE ICIP (2017)

    Google Scholar 

  10. Cai, Z., Vasconcelos, N.: Cascade R-CNN: high quality object detection and instance segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 43, 1483–1498 (2019)

    Article  Google Scholar 

  11. Chatfield, K., et al.: Return of the devil in the details: delving deep into convolutional nets. In: BMVC (2014)

    Google Scholar 

  12. Chen, L.C., et al.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)

    Article  Google Scholar 

  13. Chen, S., et al.: Discrete signal processing on graphs: sampling theory. IEEE Trans. Signal Process. 63(24), 6510–6523 (2015)

    Article  MathSciNet  Google Scholar 

  14. Danelljan, M., et al.: ECO: efficient convolution operators for tracking. In: IEEE CVPR (2017)

    Google Scholar 

  15. Du, S.S., et al.: How many samples are needed to estimate a convolutional neural network? In: NeurIPS (2018)

    Google Scholar 

  16. Egilmez, H.E., Ortega, A.: Spectral anomaly detection using graph-based filtering for wireless sensor networks. In: IEEE ICASSP (2014)

    Google Scholar 

  17. Garcia-Garcia, B., Bouwmans, T., Silva, A.J.: Background subtraction in real applications: challenges, current models and future directions. Comput. Sci. Rev. 35, 100204 (2020)

    Article  MathSciNet  Google Scholar 

  18. Giraldo, J.H., Bouwmans, T.: GraphBGS: background subtraction via recovery of graph signals. In: ICPR (2021)

    Google Scholar 

  19. Giraldo, J.H., Bouwmans, T.: On the minimization of Sobolev norms of time-varying graph signals: estimation of new Coronavirus disease 2019 cases. In: IEEE MLSP (2020)

    Google Scholar 

  20. Giraldo, J.H., Bouwmans, T.: Semi-supervised background subtraction of unseen videos: minimization of the total variation of graph signals. In: IEEE ICIP (2020)

    Google Scholar 

  21. Giraldo, J.H., Javed, S., Bouwmans, T.: Graph moving object segmentation. IEEE Trans. Pattern Anal. Mach. Intell. (2020)

    Google Scholar 

  22. Giraldo, J.H., Le, H.T., Bouwmans, T.: Deep learning based background subtraction: a systematic survey. In: Handbook of Pattern Recognition and Computer Vision, p. 51 (2020)

    Google Scholar 

  23. He, K., et al.: Deep residual learning for image recognition. In: IEEE CVPR (2016)

    Google Scholar 

  24. He, K., et al.: Mask R-CNN. In: IEEE CVPR (2017)

    Google Scholar 

  25. Javed, S., et al.: Spatiotemporal low-rank modeling for complex scene background initialization. IEEE Trans. Circuit Syst. Video Technol. 28(6), 1315–1329 (2016)

    Article  Google Scholar 

  26. Javed, S., et al.: Background-foreground modeling based on spatiotemporal sparse subspace clustering. IEEE Trans. Image Process. 26(12), 5840–5854 (2017)

    Article  MathSciNet  Google Scholar 

  27. Javed, S., et al.: Robust structural low-rank tracking. IEEE Trans. Image Process. 29, 4390–4405 (2020)

    Article  MathSciNet  Google Scholar 

  28. Javed, S., et al.: Moving object detection in complex scene using spatiotemporal structured-sparse RPCA. IEEE Trans. Image Process. 28(2), 1007–1022 (2018)

    Article  MathSciNet  Google Scholar 

  29. Jung, A., et al.: Semi-supervised learning in network-structured data via total variation minimization. IEEE Trans. Signal Process. 67(24), 6256–6269 (2019)

    Article  MathSciNet  Google Scholar 

  30. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: NeurIPS (2012)

    Google Scholar 

  31. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  32. Lim, L.A., Keles, H.Y.: Learning multi-scale features for foreground segmentation. Pattern Anal. Appl. 23(3), 1369–1380 (2020)

    Article  Google Scholar 

  33. Lucas, B.D., Kanade, T., et al.: An iterative image registration technique with an application to stereo vision (1981)

    Google Scholar 

  34. Mandal, M., Vipparthi, S.K.: Scene independency matters: an empirical study of scene dependent and scene independent evaluation for CNN-based change detection. IEEE Trans. Intell. Transp. Syst., 1–14 (2020)

    Google Scholar 

  35. Mandal, M., et al.: 3DCD: scene independent end-to-end spatiotemporal feature learning framework for change detection in unseen videos. IEEE Trans. Image Process. 30, 546–558 (2020)

    Article  Google Scholar 

  36. Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 7, 971–987 (2002)

    Article  Google Scholar 

  37. Ortega, A., et al.: Graph signal processing: overview, challenges, and applications. Proc. IEEE 106(5), 808–828 (2018)

    Article  Google Scholar 

  38. Pang, J., et al.: Optimal graph Laplacian regularization for natural image denoising. In: IEEE ICASSP (2015)

    Google Scholar 

  39. Parada-Mayorga, A., et al.: Blue-noise sampling on graphs. IEEE Trans. Signal Inf. Process. Netw. 5(3), 554–569 (2019)

    MathSciNet  Google Scholar 

  40. Parada-Mayorga, A., et al.: Sampling of graph signals with blue noise dithering. In: IEEE DSW (2019)

    Google Scholar 

  41. Perazzi, F., et al.: A benchmark dataset and evaluation methodology for video object segmentation. In: IEEE CVPR (2016)

    Google Scholar 

  42. Perraudin, N., et al.: UNLocBoX a Matlab convex optimization toolbox using proximal splitting methods. arXiv preprint arXiv:1402.0779

  43. Perraudin, N., et al.: GSPBOX: a toolbox for signal processing on graphs. arXiv preprint arXiv:1408.5781 (2014)

  44. Pesenson, I.: Sampling in Paley-Wiener spaces on combinatorial graphs. Trans. Amer. Math. Soc. 360(10), 5603–5627 (2008)

    Article  MathSciNet  Google Scholar 

  45. Pesenson, I.: Variational splines and Paley-Wiener spaces on combinatorial graphs. Constructive Approximation 29(1), 1–21 (2009)

    Article  MathSciNet  Google Scholar 

  46. Romero, D., Ma, M., Giannakis, G.B.: Kernel-based reconstruction of graph signals. IEEE Trans. Signal Process. 65(3), 764–778 (2016)

    Article  MathSciNet  Google Scholar 

  47. Shalev-Shwartz, S., Ben-David, S.: Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, Cambridge (2014)

    Book  Google Scholar 

  48. Shuman, D.I., et al.: The emerging field of signal processing on graphs: extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process. Mag. 30(3), 83–98 (2013)

    Article  Google Scholar 

  49. St-Charles, P.L., Bilodeau, G.A., Bergevin, R.: SuBSENSE: a universal change detection method with local adaptive sensitivity. IEEE Trans. Image Process. 24(1), 359–373 (2014)

    Article  MathSciNet  Google Scholar 

  50. Sultana, M., et al.: Unsupervised deep context prediction for background estimation and foreground segmentation. Mach. Vis. Appl. 30(3), 375–395 (2019)

    Article  Google Scholar 

  51. Tezcan, O., Ishwar, P., Konrad, J.: BSUV-Net: a fully-convolutional neural network for background subtraction of unseen videos. In: IEEE WACV (2020)

    Google Scholar 

  52. Thanou, D., Chou, P.A., Frossard, P.: Graph-based compression of dynamic 3D point cloud sequences. IEEE Trans. Image Process. 25(4), 1765–1778 (2016)

    Article  MathSciNet  Google Scholar 

  53. Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (2013). https://doi.org/10.1007/978-1-4757-2440-0

    Book  MATH  Google Scholar 

  54. Wang, Y., et al.: CDnet 2014: an expanded change detection benchmark dataset. In: IEEE CVPR-W (2014)

    Google Scholar 

  55. Xie, S., et al.: Aggregated residual transformations for deep neural networks. In: IEEE CVPR (2017)

    Google Scholar 

  56. Yang, F., et al.: Superpixel segmentation with fully convolutional networks. In: IEEE CVPR (2020)

    Google Scholar 

  57. Zhang, C., Florencio, D., Loop, C.: Point cloud attribute compression with graph transform. In: IEEE ICIP (2014)

    Google Scholar 

  58. Zhang, H., et al.: ResNeSt: split-attention networks. arXiv preprint arXiv:2004.08955 (2020)

Download references

Author information

Authors and Affiliations

  1. La Rochelle Université, 17000, La Rochelle, France

    Jhony H. Giraldo & Thierry Bouwmans

  2. Khalifa University, Abu Dhabi, 127788, United Arab Emirates

    Sajid Javed

  3. Kyungpook National University, Daegu, 41566, South Korea

    Maryam Sultana & Soon Ki Jung

Authors
  1. Jhony H. Giraldo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sajid Javed
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maryam Sultana
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Soon Ki Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Thierry Bouwmans
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jhony H. Giraldo .

Editor information

Editors and Affiliations

  1. Chonnam National University, Gwangju, Korea (Republic of)

    Hieyong Jeong

  2. Aoyama Gakuin University, Kanagawa, Japan

    Kazuhiko Sumi

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Giraldo, J.H., Javed, S., Sultana, M., Jung, S.K., Bouwmans, T. (2021). The Emerging Field of Graph Signal Processing for Moving Object Segmentation. In: Jeong, H., Sumi, K. (eds) Frontiers of Computer Vision. IW-FCV 2021. Communications in Computer and Information Science, vol 1405. Springer, Cham. https://doi.org/10.1007/978-3-030-81638-4_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-81638-4_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-81637-7

  • Online ISBN: 978-3-030-81638-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics