MOD-IR: moving objects detection from UAV-captured video sequences based on image registration | Multimedia Tools and Applications Skip to main content
Springer Nature Link
Log in

MOD-IR: moving objects detection from UAV-captured video sequences based on image registration

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The moving objects detection from freely moving camera like the one mounted on Unmanned Aerial Vehicle (UAV) stands as an important and challenging issue. This paper introduced a new MOD-IR method for moving objects detection from UAV-captured video sequences. The proposed method consists of four steps: (1) feature extraction and matching, (2) frame registration, (3) moving objects detection and (4) moving objects detection post-processing. Our method stands out from those of the literature in a number of ways. First, we enhanced the method effectiveness and robustness by handling the constraints related to this field through extracting robust features, on the one hand, and automatically defining the optimum threshold, on the other. Second, we proposed an efficient method able to deal with real-time applications by extracting keypoint features instead of pixel-to-pixel model estimation, and by simulating the search for the matching features among multiple trees. Finally, we involved the quick-shift segmentation in parallel with the three first steps, in order to enhance and accelerate the moving objects detection task. Relying on quantitative and qualitative evaluations of the proposed method on a variety of sequences extracted from several datasets (such as DARPA VIVID-EgTest05, Hopkins 155, UCF Aerial Action, etc.), we assessed the performance of our method compared to the state-of-the-art reference methods. Furthermore, the time cost evaluation has enabled us to emphasize that our MOD-IR method is the optimal choice for real-time applications, owing to its lower computational time requirement compared to the reference methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

No additional data or material has been used for this work other than the referenced papers.

References

  1. University of central florida. ucf aerial action data set. http://crcv.ucf.edu/data/UCF_Aerial_Action.php (2017) 01 March 2021

  2. Amri S, Barhoumi W, Zagrouba E (2010) A robust framework for joint background/foreground segmentation of complex video scenes filmed with freely moving camera. Multime Tools Appl 46(2–3):175–205

    Article  Google Scholar 

  3. Bay H, Tuytelaars T, Van Gool L (2006) Surf: Speeded up robust features. In: European conference on computer vision, pp 404–417. Springer

  4. Bentley JL (1975) Multidimensional binary search trees used for associative searching. Communications of the ACM 18(9):509–517

    Article  Google Scholar 

  5. Bilen H, Vedaldi A (2016) Weakly supervised deep detection networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2846–2854

  6. Bouguet JY (2001) Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm. Intel Corporation 5(1–10):4

    Google Scholar 

  7. Calonder M, Lepetit V, Strecha C, Fua P (2010) Brief: Binary robust independent elementary features. In: European conference on computer vision, pp 778–792. Springer

  8. Cheng Y (1995) Mean shift, mode seeking, and clustering. IEEE transactions on pattern analysis and machine intelligence 17(8):790–799

    Article  Google Scholar 

  9. Cho J, Jung Y, Kim D, Lee S, Jung Y (2018) Design of moving object detector based on modified gmm algorithm for uav collision avoidance. J Semicon Technol Sci 18(4):491–499

    Article  Google Scholar 

  10. Choi J, Chang HJ, Yoo YJ, Choi JY (2012) Robust moving object detection against fast illumination change. Comput Vision Image Underst 116(2):179–193

    Article  Google Scholar 

  11. Collins R, Zhou X, Teh SK (2005) An open source tracking testbed and evaluation web site. In: IEEE International workshop on performance evaluation of tracking and surveillance, vol 35

  12. Comaniciu D, Meer P (2002) Mean shift: A robust approach toward feature space analysis. IEEE Transactions on pattern analysis and machine intelligence 24(5):603–619

    Article  Google Scholar 

  13. Cucchiara R, Prati A, Vezzani R (2006) Advanced video surveillance with pan tilt zoom cameras. In: Proceedings of the 6th IEEE international workshop on visual surveillance, pp 334–352

  14. Dey S, Reilly V, Saleemi I, Shah M (2012) Detection of independently moving objects in non-planar scenes via multi-frame monocular epipolar constraint. In: European conference on computer vision, pp 860–873. Springer

  15. Dubrofsky E (2009) Homography estimation. Diplomová práce. Univerzita Britské Kolumbie, Vancouver

    Google Scholar 

  16. ElTantawy A, Shehata MS (2018) Krmaro: Aerial detection of small-size ground moving objects using kinematic regularization and matrix rank optimization. IEEE transactions on circuits and systems for video technology 29(6):1672–1686

    Article  Google Scholar 

  17. Fischler MA, Bolles RC (1981) Random sample consensus: A paradigm for model fitting with. Communications of the ACM 24:381–395

  18. Friedman JH, Bentley JL, Finkel RA (1977) An algorithm for finding best matches in logarithmic expected time. ACM Transactions on Mathematical Software (TOMS) 3(3):209–226

    Article  Google Scholar 

  19. Fulkerson B, Soatto S (2010) Really quick shift: image segmentation on a gpu. In: European conference on computer vision, pp 350–358. Springer

  20. Galar M, Fernandez A, Barrenechea E, Bustince H, Herrera F (2011) A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Transactions on systems, man, and cybernetics, part C (Applications and Reviews) 42(4):463–484

  21. Guillot C, Taron M, Sayd P, Pham QC, Tilmant C, Lavest JM (2010) Background subtraction adapted to ptz cameras by keypoint density estimation. In: Proceedings of the British machine vision conference, pp 34–1

  22. Guo H, Viktor HL (2004) Learning from imbalanced data sets with boosting and data generation: the databoost-im approach. ACM Sigkdd Explor Newsl 6(1):30–39

    Article  Google Scholar 

  23. Haralick RM (1989) Determining camera parameters from the perspective projection of a rectangle. Pattern Recog 22(3):225–230

    Article  Google Scholar 

  24. Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press

    Google Scholar 

  25. Hong G, Zhang Y (2007) Combination of feature-based and area-based image registration technique for high resolution remote sensing image. In: 2007 IEEE international geoscience and remote sensing symposium, pp 377–380. IEEE

  26. Huang J, Zou W, Zhu Z, Zhu J (2019) An efficient optical flow based motion detection method for non-stationary scenes. In: 2019 Chinese control and decision conference (CCDC), pp 5272–5277. IEEE

  27. Huang Y, Jiang Q, Qian Y (2020) A novel method for video moving object detection using improved independent component analysis. IEEE Transactions on circuits and systems for video technology 31(6):2217–2230

    Article  Google Scholar 

  28. Jackson BP, Goshtasby AA (2009) Registering aerial video images using the projective constraint. IEEE Transactions on image processing 19(3):795–804

    Article  MathSciNet  Google Scholar 

  29. Kalantar B, Mansor SB, Halin AA, Shafri HZM, Zand M (2017) Multiple moving object detection from uav videos using trajectories of matched regional adjacency graphs. IEEE Transactions on geoscience and remote sensing 55(9):5198–5213

    Article  Google Scholar 

  30. Kang J, Cohen I, Medioni G (2003) Continuous tracking within and across camera streams. In: 2003 IEEE computer society conference on computer vision and pattern recognition, 2003. Proceedings, vol 1, pp 267–272. IEEE

  31. Kapur JN, Sahoo PK, Wong AK (1985) A new method for gray-level picture thresholding using the entropy of the histogram. Computer vision, graphics, and image processing 29(3):273–285

    Article  Google Scholar 

  32. Lee DS (2008) Effective gaussian mixture learning for video background subtraction. IEEE transactions on pattern analysis and machine intelligence 27(5):827–832

    Google Scholar 

  33. Li F, Kim T, Humayun A, Tsai D, Rehg JM (2013) Video segmentation by tracking many figure-ground segments. In: Proceedings of the IEEE international conference on computer vision, pp 2192–2199

  34. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60(2):91–110

    Article  Google Scholar 

  35. Ma J, Zhou H, Zhao J, Gao Y, Jiang J, Tian J (2015) Robust feature matching for remote sensing image registration via locally linear transforming. IEEE transactions on geoscience and remote sensing 53(12):6469–6481

    Article  Google Scholar 

  36. Márquez-Neila P, López-Alberca J, Buenaposada JM, Baumela L (2013) Speeding-up homography estimation in mobile devices. J Real-Time Image Proc 11(1):141–154

    Article  Google Scholar 

  37. Minaeian S, Liu J, Son YJ (2018) Effective and efficient detection of moving targets from a uav’s camera. IEEE transactions on intelligent transportation systems 19(2):497–506

    Article  Google Scholar 

  38. Mittal A, Huttenlocher D (2000) Scene modeling for wide area surveillance and image synthesis. In: Proceedings IEEE conference on computer vision and pattern recognition. CVPR 2000 (Cat. No. PR00662), vol 2, pp 160–167. IEEE

  39. Mliki H, Bouhlel F, Hammami M (2020) Human activity recognition from uav-captured video sequences. Pattern Recognition 100:107140

    Article  Google Scholar 

  40. Muja M, Lowe DG (2009) Fast approximate nearest neighbors with automatic algorithm configuration. VISAPP (1) 2:(331-340), 2

  41. Muja M, Lowe DG (2014) Scalable nearest neighbor algorithms for high dimensional data. IEEE transactions on pattern analysis and machine intelligence 36(11):2227–2240

    Article  Google Scholar 

  42. Pharr M, Jakob W, Humphreys G (2016) Physically based rendering: From theory to implementation. Morgan Kaufmann

  43. Pun T (1981) Entropic thresholding, a new approach. Computer graphics and image processing 16(3):210–239

    Article  Google Scholar 

  44. Reilly V, Idrees H, Shah M (2010) Detection and tracking of large number of targets in wide area surveillance. In: European conference on computer vision, pp 186–199. Springer

  45. Robinault L, Bres S, Miguet S (2009) Real time foreground object detection using ptz camera. In: VISAPP (1), pp 609–614

  46. Rublee E, Rabaud V, Konolige K, Bradski G (2011) Orb: an efficient alternative to sift or surf. 2011 ie int conf on computer vision: Google Scholar Google Scholar Digital Library Digital Library 2564–2571

  47. Sand P, Teller S (2008) Particle video: Long-range motion estimation using point trajectories. Int J Comput Vision 80(1):72

    Article  Google Scholar 

  48. Sengar SS, Mukhopadhyay S (2020) Moving object detection using statistical background subtraction in wavelet compressed domain. Multimed Tools Appl 79(9):5919–5940

    Article  Google Scholar 

  49. Sheikh Y, Javed O, Kanade T (2009) Background subtraction for freely moving cameras. In: 2009 IEEE 12th international conference on computer vision, pp 1219–1225. IEEE

  50. Silpa-Anan C, Hartley R (2008) Optimised kd-trees for fast image descriptor matching. In: 2008 IEEE conference on computer vision and pattern recognition, pp 1–8. IEEE

  51. Song K, Zhang W, Lu W, Zha ZJ, Ji X, Li Y (2019) Visual object tracking via guessing and matching. IEEE transactions on circuits and systems for video technology

  52. Spagnolo P, Leo M, Distante A et al (2006) Moving object segmentation by background subtraction and temporal analysis. Image Vision Comput 24(5):411–423

    Article  Google Scholar 

  53. Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. In: Proceedings 1999 IEEE computer society conference on computer vision and pattern recognition (Cat. No PR00149), vol 2, pp 246–252. IEEE

  54. Suo P, Wang Y (2008) An improved adaptive background modeling algorithm based on gaussian mixture model. In: 2008 9th international conference on signal processing, pp 1436–1439. IEEE

  55. Takacs G, Chandrasekhar V, Tsai S, Chen D, Grzeszczuk R, Girod B (2012) Rotation invariant fast features for large-scale recognition. In: Applications of digital image processing XXXV, vol 8499, pp 84991D. Int Soc Opt Photon

  56. Tron R, Vidal R (2007) A benchmark for the comparison of 3-d motion segmentation algorithms. In: 2007 IEEE conference on computer vision and pattern recognition, pp 1–8. IEEE

  57. Vedaldi A, Soatto S (2008) Quick shift and kernel methods for mode seeking. In: European conference on computer vision, pp 705–718. Springer

  58. Vinay A, Kailasam S, Bharadwaj A, Murthy K, Natarajan S (2019) Effective utilization of whitening for person identification. S, Effective Utilization of Whitening for Person Identification (February 23, 2019)

  59. Walha A, Wali A, Alimi AM (2015) Video stabilization with moving object detecting and tracking for aerial video surveillance. Multimed Tools Appl 74(17):6745–6767

    Article  Google Scholar 

  60. Wang B, Liu J, Zhu S, Xu F, Liu C (2023) A dual-input moving object detection method in remote sensing image sequences via temporal semantics. Remote Sensing 15(9):2230

    Article  Google Scholar 

  61. Wang Q, Gao J, Yuan Y (2017) Embedding structured contour and location prior in siamesed fully convolutional networks for road detection. IEEE Transactions on intelligent transportation systems 19(1):230–241

    Article  Google Scholar 

  62. Wang Q, Gao J, Yuan Y (207) A joint convolutional neural networks and context transfer for street scenes labeling. IEEE Trans Intell Trans Syst 19(5):1457–1470

  63. Wu M, Peng X, Zhang Q (2011) Segmenting moving objects from a freely moving camera with an effective segmentation cue. Meas Sci Technol 22(2):025108

    Article  Google Scholar 

  64. Wu S, Oreifej O, Shah M (2011) Action recognition in videos acquired by a moving camera using motion decomposition of lagrangian particle trajectories. In: 2011 international conference on computer vision, pp 1419–1426. IEEE

  65. Wu Y, He X, Nguyen TQ (2017) Moving object detection with a freely moving camera via background motion subtraction. IEEE Trans Circuits Syst Video Technol 27(2):236–248

    Article  Google Scholar 

  66. Xiao J, Cheng H, Sawhney H, Han F (2010) Vehicle detection and tracking in wide field-of-view aerial video. In: 2010 IEEE computer society conference on computer vision and pattern recognition, pp 679–684. IEEE

  67. Yazdi M, Bouwmans T (2018) New trends on moving object detection in video images captured by a moving camera: a survey. Comput Sci Rev 28:157–177

    Article  MathSciNet  Google Scholar 

  68. Yu Y, Kurnianggoro L, Jo KH (2019) Moving object detection for a moving camera based on global motion compensation and adaptive background model. Int J Control Auto Syst 17(7):1866–1874

    Article  Google Scholar 

  69. Yuan Y, Lu Y, Wang Q (2017) Tracking as a whole: multi-target tracking by modeling group behavior with sequential detection. IEEE Trans Intel Transport Syst 18(12):3339–3349

    Article  Google Scholar 

  70. Yun K, Lim J, Choi JY (2017) Scene conditional background update for moving object detection in a moving camera. Pattern Recog Lett 88:57–63

    Article  Google Scholar 

  71. Zhang G, Jia J, Xiong W, Wong TT, Heng PA, Bao H (2007) Moving object extraction with a hand-held camera. In: 2007 IEEE 11th international conference on computer vision, pp 1–8. IEEE

  72. Zheng A, Zou T, Zhao Y, Jiang B, Tang J, Li C (2019) Background subtraction with multi-scale structured low-rank and sparse factorization. Neurocomput 328:113–121

    Article  Google Scholar 

  73. Zheng Y, Zheng C, Zhang X, Chen F, Chen Z, Zhao S (2022) Detection, localization, and tracking of multiple mavs with panoramic stereo camera networks. IEEE transactions on automation science and engineering

  74. Zitova B, Flusser J (2003) Image registration methods: a survey. Image Vision Comput 21(11):977–1000

    Article  Google Scholar 

Download references

Funding

This study was not funded by any organization.

Author information

Authors and Affiliations

  1. MIRACL-FSS, University of Sfax Faculty of Sciences of Sfax, Road Sokra Km 3, Sfax, 3018, Tunisia

    Fatma Bouhlel & Mohamed Hammami

  2. MIRACL Laboratory, University of Sfax, Sfax, Tunisia

    Hazar Mliki

  3. National Institute of Applied Science and Technology, University of Carthage, Tunis, Tunisia

    Hazar Mliki

Authors
  1. Fatma Bouhlel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hazar Mliki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed Hammami
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fatma Bouhlel.

Ethics declarations

Conflict of interest

The authors have no conflict of interest in relation to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bouhlel, F., Mliki, H. & Hammami, M. MOD-IR: moving objects detection from UAV-captured video sequences based on image registration. Multimed Tools Appl 83, 46779–46798 (2024). https://doi.org/10.1007/s11042-023-16667-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16667-1

Keywords

Search

Navigation