Real-Time Artificial Intelligence Based Visual Simultaneous Localization and Mapping in Dynamic Environments – a Review | Journal of Intelligent & Robotic Systems Skip to main content

Advertisement

Springer Nature Link
Log in

Real-Time Artificial Intelligence Based Visual Simultaneous Localization and Mapping in Dynamic Environments – a Review

  • Review Paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

This paper aims to review current approaches for real-time artificial intelligence-based visual Simultaneous Localization and Mapping (vSLAM) operating in dynamic environments. Through the review of several relatively recent published papers for vSLAM in dynamic environments, an attempt is made to explain the concept of Simultaneous Localization and Mapping (SLAM) and its purpose; identify the general framework for real-time AI-based vSLAM approaches in dynamic environments and highlighting the potential solutions that has been developed and their significant results. All related information regarding this topic was obtained with the intention to answer three main questions. Firstly, how do robots localize and map within an unknown environment? Secondly, how can state-of-the-art vSLAM be modified with an AI algorithm to function within a dynamic environment? And lastly, what level of success has these approaches achieved in developing methods for real-time AI-based vSLAM in dynamic environments? The paper intends to provide readers with a clearer general understanding of SLAM and acts as a road map for steps to move forward in developing viable approaches for real-time vSLAM in dynamic environments based on artificial intelligence.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Bailey, T., Durrant-Whyte, H.: Simultaneous localization and mapping (SLAM): part I. IEEE Robot. Autom. Mag. 13, 99–108 (2006)

    Article  Google Scholar 

  2. Dissanayake, G., Huang, S., Wang, Z., Ranasinghe, R.: A review of recent developments in simultaneous localization and mapping. In: 2011 6th International Conference on Industrial and Information Systems, pp. 477–482 (2011). https://doi.org/10.1109/ICIINFS.2011.6038117

    Chapter  Google Scholar 

  3. Saputra, M.R.U., Markham, A., Trigoni, N.: Visual SLAM and structure from motion in dynamic environments: A survey. ACM Comput. Surv. 51. Epub ahead of print 2018, 1–36 (2019). https://doi.org/10.1145/3177853

    Article  Google Scholar 

  4. Sun, Y., Liu, M., Meng, M.Q.H.: Improving RGB-D SLAM in dynamic environments: a motion removal approach. Robot. Auton. Syst. 89, 110–122 (2017)

    Article  Google Scholar 

  5. Zeng, F., Wang, C., Ge, S.S.: A survey on visual navigation for artificial agents with deep reinforcement learning. IEEE Access. 8, 135426–135442 (2020)

    Article  Google Scholar 

  6. Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., Leonard, J.J.: Past, present, and future of simultaneous localization and mapping: toward the robust-perception age. IEEE Trans. Robot. 32, 1309–1332 (2016)

    Article  Google Scholar 

  7. Taketomi, T., Uchiyama, H., Ikeda, S.: Visual SLAM algorithms: a survey from 2010 to 2016. IPSJ Trans. Comput. Vision Appl. 9, 1–11 (2017)

    Google Scholar 

  8. Li, A., Ruan, X., Huang, J., Zhu, X., Wang, F.: Review of vision-based simultaneous localization and mapping. In: 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), pp. 117–123 (2019). https://doi.org/10.1109/ITNEC.2019.8729285

  9. Bresson, G., Alsayed, Z., Yu, L., Glaser, S.: Simultaneous localization and mapping: a survey of current trends in autonomous driving. IEEE Trans. Intell. Veh. 2, 194–220 (2017)

    Article  Google Scholar 

  10. Singandhupe, A., La, H.M.: A review of SLAM techniques and security in autonomous driving. Third IEEE International Conference on Robotic Computing (IRC). 2019, 602–607 (2019). https://doi.org/10.1109/IRC.2019.00122

    Article  Google Scholar 

  11. Panchpor, A.A., Shue, S., Conrad, J.M.: A survey of methods for mobile robot localization and mapping in dynamic indoor environments. Conference on Signal Processing And Communication Engineering Systems (SPACES). 2018, 138–144 (2018). https://doi.org/10.1109/SPACES.2018.8316333

    Article  Google Scholar 

  12. Agunbiade, O., Zuva, T.: Simultaneous localization and mapping in application to autonomous robot. 2018 Int Conf Intell Innov Comput Appl ICONIC 2018. Epub ahead of print 2019. https://doi.org/10.1109/ICONIC.2018.8601094

  13. Azzam, R., Taha, T., Huang, S., Zweiri, Y.: Feature-based visual simultaneous localization and mapping: a survey. SN Appl Sci. 2, 1–24 (2020)

    Article  Google Scholar 

  14. Brownlee, J.: A Gentle Introduction to Maximum a Posteriori (MAP) for Machine Learning. Machine Learning Mastery, (2019) https://machinelearningmastery.com/maximum-a-posteriori-estimation/, Accessed 11 March 2021)

  15. Lu, F., Milios, E.: Globally consistent range scan alignment for environment mapping. Auton. Robot. 4, 333–349 (1997)

    Article  Google Scholar 

  16. Gutmann J-S, Konolige K. Incremental mapping of large cyclic environments. Inst. Electr. Electron. Eng. (IEEE), 2003, pp. 318–325

  17. Aulinas, J., Petillot, Y., Salvi, J., et al.: The SLAM problem: a survey. Front Artif Intell Appl. 184, 363–371 (2008)

    Google Scholar 

  18. Thrun, S.: Probabilistic Algorithms in Robotics. AI Mag. 21, 93–109 (2000)

    Google Scholar 

  19. Thrun, S.: Robotic Mapping: A Survey. Science (80). 298, 1–35 (2002)

    Google Scholar 

  20. Stachniss, C., Leonard, J.J., Thrun, S.: Simultaneous Localization and Mapping. In: Springer Handbook of Robotics, pp. 1153–1175. Springer International Publishing (2016)

    Chapter  Google Scholar 

  21. Engel, J., Schöps, T., Cremers, D.: LSD-SLAM: large-scale direct monocular SLAM. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, Vol 8690. Springer, Cham. (2014). https://doi.org/10.1007/978-3-319-10605-2_54

    Chapter  Google Scholar 

  22. Younes, G., Asmar, D., Shammas, E., Zelek, J.: Keyframe-based monocular SLAM: design, survey, and future directions. Robot. Auton. Syst. 98, 67–88 (2017)

    Article  Google Scholar 

  23. Newcombe, R.A., Lovegrove, S.J., Davison, A.J.: DTAM: dense tracking and mapping in real-time. International Conference on Computer Vision. 2011, 2320–2327 (2011). https://doi.org/10.1109/ICCV.2011.6126513

    Article  Google Scholar 

  24. Forster, C., Pizzoli, M., Scaramuzza, D.: SVO: Fast semi-direct monocular visual odometry. In: Proceedings - IEEE International Conference on Robotics and Automation. Ins. Electric. Electron. Eng. Inc., 15–22 (2014)

  25. Alismail, H., Browning, B., Lucey, S.: Photometric Bundle Adjustment for Vision-Based SLAM. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics). 10114 LNCS, 324–341 (2016)

    Google Scholar 

  26. Schops, T., Sattler, T., Pollefeys, M.: Bad SLAM: Bundle Adjusted Direct RGB-D SLAM. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. 2019, 134–144 (2019)

    Google Scholar 

  27. Zubizarreta, J., Aguinaga, I., Montiel, J.M.M.: Direct sparse mapping. IEEE Trans. Robot. 36, 1363–1370 (2020)

    Article  Google Scholar 

  28. Mao, Z., Zhao, L., Huang, S., et al.: Direct bundle adjustment for 3D image fusion with application to transesophageal echocardiography *. IEEE/RSJ Int Conf Intell Robot Syst. 2021, 548–554 (2021)

    Google Scholar 

  29. Cho, H., Kim, E.K., Kim, S.: Indoor SLAM application using geometric and ICP matching methods based on line features. Robot. Auton. Syst. 100, 206–224 (2018)

    Article  Google Scholar 

  30. Kasyanov, A., Engelmann, F., Stückler, J., Leibe, B.: Keyframe-based visual-inertial online SLAM with relocalization. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2017, 6662–6669 (2017). https://doi.org/10.1109/IROS.2017.8206581

    Article  Google Scholar 

  31. Souto, L.A.V., Nascimento, T.P.: Object subtraction planar RGB-D SLAM. In: 2016 XIII Latin American Robotics Symposium and IV Brazilian Robotics Symposium (LARS/SBR), pp. 19–24 (2016). https://doi.org/10.1109/LARS-SBR.2016.11

  32. Li, J., Meger, D., Dudek, G.: Context-coherent scenes of objects for camera pose estimation. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2017, 655–660 (2017). https://doi.org/10.1109/IROS.2017.8202221

    Article  Google Scholar 

  33. Gawel, A., Del Don, C., Siegwart, R., et al.: X-view: Graph-based semantic Multiview localization. IEEE Robot Autom Lett. 3, 1687–1694 (2018)

    Article  Google Scholar 

  34. Mu, B., Liu, S., Paull, L., Leonard, J., How, J.P.: SLAM with objects using a nonparametric pose graph. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2016, 4602–4609 (2016). https://doi.org/10.1109/IROS.2016.7759677

    Article  Google Scholar 

  35. Yang, S., Song, Y., Kaess, M., Scherer, S.: Pop-up SLAM: semantic monocular plane SLAM for low-texture environments. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2016, 1222–1229 (2016). https://doi.org/10.1109/IROS.2016.7759204

    Article  Google Scholar 

  36. Gálvez-López, D., Salas, M., Tardós, J.D., Montiel, J.M.M.: Real-time monocular object SLAM. Robot. Auton. Syst. 75, 435–449 (2016)

    Article  Google Scholar 

  37. Yang, S., Scherer, S.: Monocular object and plane slam in structured environments. IEEE Robot Autom Lett. 4, 3145–3152 (2019)

    Article  Google Scholar 

  38. Ho, T.S., Fai, Y.C.: Ming, E.S.L.: Simultaneous localization and mapping survey based on filtering techniques. In: 2015 10th Asian Control Conference: Emerging Control Techniques for a Sustainable World, ASCC 2015. Institute of Electrical and Electronics Engineers Inc., (2015). Epub ahead of print 8 September 2015. https://doi.org/10.1109/ASCC.2015.7244836

  39. Ribeiro, M.I.: Kalman and extended kalman filters: concept, derivation and properties. Institute for Systems and Robotics. 43, 46 (2004)

    Google Scholar 

  40. Huang, S., Dissanayake, G.: Convergence and consistency analysis for extended Kalman filter based SLAM. IEEE Trans. Robot. 23, 1036–1049 (2007)

    Article  Google Scholar 

  41. Huang S., Dissanayake G.: Convergence analysis for extended Kalman filter based SLAM. Proceedings 2006 IEEE International Conference on Robotics and Automation. ICRA 2006, vol. 2006, pp. 412–417 (2006). https://doi.org/10.1109/ROBOT.2006.1641746

  42. Bailey, T., Nieto, J., Guivant, J., Stevens, M., Nebot, E.: Consistency of the EKF-SLAM algorithm. IEEE/RSJ International Conference on Intelligent Robots and Systems. 2006, 3562–3568 (2006). https://doi.org/10.1109/IROS.2006.281644

    Article  Google Scholar 

  43. Solà, J.: Consistency of the monocular EKF-SLAM algorithm for three different landmark parametrizations. IEEE International Conference on Robotics and Automation. 2010, 3513–3518 (2010). https://doi.org/10.1109/ROBOT.2010.5509518

    Article  Google Scholar 

  44. Zhang, T., Wu, K., Song, J., Huang, S., Dissanayake, G.: Convergence and consistency analysis for a 3-D invariant-EKF SLAM. IEEE Robot Autom Lett. 2, 733–740 (2017)

    Article  Google Scholar 

  45. Davison, A.J., Reid, I.D., Molton, N.D., Stasse, O.: MonoSLAM: Real-time single camera SLAM. IEEE Trans. Pattern Anal. Mach. Intell. 29, 1052–1067 (2007)

    Article  Google Scholar 

  46. Esparza-Jiménez, J.O., Devy, M., Gordillo, J.L.: Visual EKF-SLAM from Heterogeneous Landmarks. Sensors. 16, 489 (2016)

    Article  Google Scholar 

  47. Quan, M., Piao, S., Tan, M., Huang, S.S.: Accurate monocular visual-inertial SLAM using a map-assisted EKF approach. IEEE Access. 7, 34289–34300 (2019)

    Article  Google Scholar 

  48. Zhou, H., Zou, D., Pei, L., Ying, R., Liu, P., Yu, W.: StructSLAM: visual SLAM with building structure lines. IEEE Trans. Veh. Technol. 64, 1364–1375 (2015)

    Article  Google Scholar 

  49. Yuan, W., Li, Z., Su, C.: RGB-D sensor-based visual SLAM for localization and navigation of indoor mobile robot. International Conference on Advanced Robotics and Mechatronics (ICARM). 2016, 82–87 (2016). https://doi.org/10.1109/ICARM.2016.7606899

    Article  Google Scholar 

  50. van der Merwe, R., Doucet, A., De Freitas, N., et al.: A tutorial on particle filtering and smoothing. Handb Nonlinear Filter. 12, 656–704 (2009)

    Google Scholar 

  51. Gustafsson, F., Gunnarsson, F., Bergman, N., Forssell, U., Jansson, J., Karlsson, R., Nordlund, P.J.: Particle filters for positioning, navigation, and tracking. IEEE Trans. Signal Process. 50, 425–437 (2002)

    Article  Google Scholar 

  52. Gustafsson, F.: Particle filter theory and practice with positioning applications. IEEE Aerosp. Electron. Syst. Mag. 25, 53–81 (2010)

    Article  Google Scholar 

  53. Karlsson, N., Di Bernardo, E., Ostrowski, J., et al.: The vSLAM algorithm for robust localization and mapping. Proc - IEEE Int Conf Robot Autom. 2005, 24–29 (2005)

    Google Scholar 

  54. Hee Won, D., Chun, S., Sung, S., et al.: INS/vSLAM system using distributed particle filter. Int. J. Control. Autom. Syst. 8, 1232–1240 (2010)

    Article  Google Scholar 

  55. Montemerlo, M., Thrun, S., Koller, D., Wegbreit, B.: FastSLAM: a factored solution to the simultaneous localization and mapping problem. Proceedings of the Eighteenth National Conference on Artificial Intelligence. 593–598 (2002)

  56. Carlone, L., Du, J., Kaouk Ng, M., et al.: ActiveSLAM and exploration with particle filters using Kullback-Leibler divergence. J Intell Robot Syst Theory Appl. 75, 291–311 (2014)

    Article  Google Scholar 

  57. Chan, S.-H., Wu, P.-T., Fu, L.-C.: Robust 2D indoor localization through laser SLAM and visual SLAM fusion. In: 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 1263–1268 (2018). https://doi.org/10.1109/SMC.2018.00221

    Chapter  Google Scholar 

  58. Farrokhsiar, M., Najjaran, H.: Rao-Blackwellized particle filter approach to monocular vSLAM with a modified initialization scheme. Proc ASME Des Eng Tech Conf. 3, 427–433 (2010)

    Google Scholar 

  59. Farrokhsiar, M., Najjaran, H.: A higher order Rao-Blackwellized particle filter for monocular vSLAM. In: Proceedings of the 2010 American Control Conference, pp. 6987–6992 (2010). https://doi.org/10.1109/ACC.2010.5531347

    Chapter  Google Scholar 

  60. Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 225–234 (2007). https://doi.org/10.1109/ISMAR.2007.4538852

    Chapter  Google Scholar 

  61. Strasdat, H., Montiel, J.M.M., Davison, A.J.: Visual SLAM: Why filter? Image Vis. Comput. 30, 65–77 (2012)

    Article  Google Scholar 

  62. Mur-Artal, R., Montiel, J.M.M., Tardos, J.D.: ORB-SLAM: A Versatile and Accurate Monocular SLAM System. IEEE Trans. Robot. 31, 1147–1163 (2015)

    Article  Google Scholar 

  63. Mur-Artal, R., Tardos, J.D.: ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras. IEEE Trans. Robot. 33, 1255–1262 (2017)

    Article  Google Scholar 

  64. Campos Martínez C, Elvira R, Gómez Rodríguez JJ, et al. ORB-SLAM3: An accurate Open-source library for visual, Visual-inertial and Multi-map SLAM. arXiv. Epub ahead of print 2020. https://doi.org/10.1109/TRO.2021.3075644

  65. Shin, Y., Park, Y.S., Kim, A.: Direct visual SLAM using sparse depth for camera-LiDAR system. IEEE International Conference on Robotics and Automation (ICRA). 2018, 5144–5151 (2018). https://doi.org/10.1109/ICRA.2018.8461102

    Article  Google Scholar 

  66. Chen, S., Zhou, B., Jiang, C., et al.: A LiDAR/Visual SLAM Backend with Loop Closure Detection and Graph Optimization. Remote Sens. 13, 2720 (2021)

    Article  Google Scholar 

  67. Grisetti, G., Kümmerle, R., Stachniss, C., et al.: A tutorial on Graph based SLAM. Aerospace. 2, 31–43 (2010)

    Google Scholar 

  68. Sünderhauf, N., Protzel, P.: Towards a robust back-end for pose graph SLAM. IEEE International Conference on Robotics and Automation. 2012, 1254–1261 (2012). https://doi.org/10.1109/ICRA.2012.6224709

    Article  Google Scholar 

  69. Mendes, E., Koch, P., Lacroix, S.. ICP-based pose-graph SLAM. In: SSRR 2016 - International Symposium on Safety, Security and Rescue Robotics. Institute of Electrical and Electronics Engineers Inc., pp. 195–200 (2016)

  70. Pumarola, A., Vakhitov, A., Agudo, A., et al.: PL-SLAM: real-time monocular visual SLAM with points and lines. Proc - IEEE Int Conf Robot Autom, 4503–4508 (2017)

  71. Tateno, K., Tombari, F., Laina, I., Navab, N.: CNN-SLAM: real-time dense monocular SLAM with learned depth prediction. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017, 6565–6574 (2017). https://doi.org/10.1109/CVPR.2017.695

    Article  Google Scholar 

  72. Wang, R., Wan, W., Wang, Y., et al.: A new RGB-D SLAM method with moving object detection for dynamic indoor scenes. Remote Sens. 11. Epub ahead of print 2019. https://doi.org/10.3390/rs11101143

  73. Wang, C., Luo, B., Zhang, Y., Zhao, Q., Yin, L., Wang, W., Su, X., Wang, Y., Li, C.: DymSLAM: 4D dynamic scene reconstruction based on geometrical motion segmentation. IEEE Robot Autom Lett. 6, 550–557 (2021)

    Article  Google Scholar 

  74. Thrun, S., Montemerlo, M.: The Graph SLAM Algorithm with applications to large-scale mapping of urban structures. Int. J. Robot. Res. 25, 403–429 (2006)

    Article  Google Scholar 

  75. Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment — a modern synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) Vision Algorithms: Theory and Practice. IWVA 1999. Lecture Notes in Computer Science, Vol 1883. Springer, Berlin, Heidelberg (2000). https://doi.org/10.1007/3-540-44480-7_21

    Chapter  Google Scholar 

  76. Thrun, S., Burgard, W., Fox, D.: Probabilistic Robotics | The MIT Press. MIT Press, https://mitpress.mit.edu/books/probabilistic-robotics.(2005), Accessed 27 March 2021) (2021)

  77. Kaess, M., Johannsson, H., Roberts, R., Ila, V., Leonard, J.J., Dellaert, F.: iSAM2: incremental smoothing and mapping using the Bayes tree. Int. J. Robot. Res. 31, 216–235 (2012)

    Article  Google Scholar 

  78. Mouragnon, E., Lhuillier, M., Dhome, M., Dekeyser, F., Sayd, P.: Real time localization and 3D reconstruction. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), pp. 363–370 (2006). https://doi.org/10.1109/CVPR.2006.236

    Chapter  Google Scholar 

  79. Kalogeiton, V.S., Ioannidis, K., Sirakoulis, G.C., et al.: Real-Time Active SLAM and Obstacle Avoidance for an Autonomous Robot Based on Stereo Vision. Cybern Syst. 50, 239–260 (2019). https://doi.org/10.1080/0196972220181541599

    Article  Google Scholar 

  80. Hachiuma, R., Pirchheim, C., Schmalstieg, D., Saito, H.: DetectFusion: detecting and segmenting both known and unknown dynamic objects in real-time slam. In: Paper Presented at 30th British Machine Vision Conference, BMVC 2019. United Kingdom, Cardiff (2020)

    Google Scholar 

  81. Kelly, J., E.: Computing, cognition and the future of knowing (whitepaper). IBM Corporation. (2015)

  82. Moore, M.: What is Industry 4.0? Everything you need to know | TechRadar. (2018). Techradar.com, https://www.techradar.com/news/what-is-industry-40-everything-you-need-to-know, (Accessed 24 August 2021)

  83. Chen C, Wang B, Xiaoxuan Lu C, et al. A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence. arXiv, http://arxiv.org/abs/2006.12567 (2020), (Accessed 29 October 2020)

  84. Wangsiripitak, S., Murray, D.W.: Avoiding moving outliers in visual SLAM by tracking moving objects. IEEE International Conference on Robotics and Automation. 2009, 375–380 (2009). https://doi.org/10.1109/ROBOT.2009.5152290

    Article  Google Scholar 

  85. Lin, K., Wang, C.: Stereo-based simultaneous localization, mapping and moving object tracking. IEEE/RSJ International Conference on Intelligent Robots and Systems. 2010, 3975–3980 (2010). https://doi.org/10.1109/IROS.2010.5649653

    Article  Google Scholar 

  86. Lin, T.-H., Wang, C.-C.: Deep learning of spatio-temporal features with geometric-based moving point detection for motion segmentation. IEEE International Conference on Robotics and Automation (ICRA). 2014, 3058–3065 (2014). https://doi.org/10.1109/ICRA.2014.6907299

    Article  Google Scholar 

  87. Sabzevari, R., Scaramuzza, D.: Monocular simultaneous multi-body motion segmentation and reconstruction from perspective views. IEEE International Conference on Robotics and Automation (ICRA). 2014, 23–30 (2014). https://doi.org/10.1109/ICRA.2014.6906585

    Article  Google Scholar 

  88. Zhang, T., Szlam, A., Lerman, G.: Median K-flats for hybrid linear modeling with many outliers. In: 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops, pp. 234–241 (2009). https://doi.org/10.1109/ICCVW.2009.5457695

    Chapter  Google Scholar 

  89. Byravan, A., Fox, D.: SE3-nets: learning rigid body motion using deep neural networks. IEEE International Conference on Robotics and Automation (ICRA). 2017, 173–180 (2017). https://doi.org/10.1109/ICRA.2017.7989023

    Article  Google Scholar 

  90. Kundu, A., Krishna, K.M., Jawahar, C.V.. Realtime multibody visual SLAM with a smoothly moving monocular camera. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 2080–2087 (2011)

  91. Park, H.S., Shiratori, T., Matthews, I., Sheikh, Y.: 3D trajectory reconstruction under perspective projection. Int. J. Comput. Vis. 115, 115–135 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  92. Zappella, L., Del Bue, A., Lladó, X., et al.: Joint estimation of segmentation and structure from motion. Comput. Vis. Image Underst. 117, 113–129 (2013)

    Article  Google Scholar 

  93. Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 2481–2495 (2017)

    Article  Google Scholar 

  94. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 386–397 (2020). https://doi.org/10.1109/TPAMI.2018.2844175

    Chapter  Google Scholar 

  95. Urzua, S., Munguía, R., Grau, A.: Vision-based SLAM system for MAVs in GPS-denied environments. Int J Micro Air Veh. 9, 283–296 (2017)

    Article  Google Scholar 

  96. Lowe, T., Kim, S., Cox, M.: Complementary perception for handheld SLAM. IEEE Robot Autom Lett. 3, 1104–1111 (2018)

    Article  Google Scholar 

  97. Maity, S., Saha, A., Bhowmick, B.: Edge SLAM: edge points based monocular visual SLAM. IEEE International Conference on Computer Vision Workshops (ICCVW). 2017, 2408–2417 (2017). https://doi.org/10.1109/ICCVW.2017.284

    Article  Google Scholar 

  98. Zhang, A.S., Liu, B.S., Zhang, C.J., Wang, D.Z., Wang, E.X.: Fast initialization for feature-based monocular slam. IEEE International Conference on Image Processing (ICIP). 2017, 2119–2123 (2017). https://doi.org/10.1109/ICIP.2017.8296656

    Article  Google Scholar 

  99. Civera, J., Galvez-Lopez, D., Riazuelo, L., et al.: Towards semantic SLAM using a monocular camera. Inst. Electr. Electron. Eng. (IEEE), 1277–1284 (2011)

  100. Brasch, N., Bozic, A., Lallemand, J., et al.: Semantic monocular SLAM for highly dynamic environments. IEEE Int Conf Intell Robot Syst, 393–400 (2018)

  101. Vogiatzis, G., Hernández, C.: Video-based, real-time multi-view stereo. Image Vis. Comput. 29, 434–441 (2011)

    Article  Google Scholar 

  102. Xiao, L., Wang, J., Qiu, X., Rong, Z., Zou, X.: Dynamic-SLAM: semantic monocular visual localization and mapping based on deep learning in dynamic environment. Robot. Auton. Syst. 117, 1–16 (2019)

    Article  Google Scholar 

  103. Bescos, B., Facil, J.M., Civera, J., Neira, J.: DynaSLAM: tracking, mapping, and Inpainting in dynamic scenes. IEEE Robot Autom Lett. 3, 4076–4083 (2018)

    Article  Google Scholar 

  104. Zhong, F., Wang, S., Zhang, Z., Chen, C., Wang, Y.: Detect-SLAM: making object detection and SLAM mutually beneficial. IEEE Winter Conference on Applications of Computer Vision (WACV). 2018, 1001–1010 (2018). https://doi.org/10.1109/WACV.2018.00115

    Article  Google Scholar 

  105. Yu, C., Liu, Z., Liu, X.J., et al.: DS-SLAM: a semantic visual SLAM towards dynamic environments. IEEE Int Conf Intell Robot Syst, 1168–1174 (2018)

  106. Cui, L., Ma, C.: SDF-SLAM: semantic depth filter SLAM for dynamic environments. IEEE Access. 8, 95301–95311 (2020)

    Article  Google Scholar 

  107. Cui, L., Ma, C.: SOF-SLAM: a semantic visual SLAM for dynamic environments. IEEE Access. 7, 166528–166539 (2019)

    Article  Google Scholar 

  108. Zhang, L., Wei, L., Shen, P., Wei, W., Zhu, G., Song, J.: Semantic SLAM based on object detection and improved octomap. IEEE Access. 6, 75545–75559 (2018)

    Article  Google Scholar 

  109. Zhao, L., Liu, Z., Chen, J., Cai, W., Wang, W., Zeng, L.: A compatible framework for RGB-D SLAM in dynamic scenes. IEEE Access. 7, 75604–75614 (2019)

    Article  Google Scholar 

  110. Long, X., Zhang, W., Zhao, B.: PSPNet-SLAM: a semantic SLAM detect dynamic object by pyramid scene parsing network. IEEE Access. 8, 214685–214695 (2020)

    Article  Google Scholar 

  111. Bao, A.Y., Rui, T., Qiang, Y.X., et al.: Visual SLAM in dynamic environments based on object detection. Def Technol. Epub ahead of print 24 September 2020. (2021). https://doi.org/10.1016/j.dt.2020.09.012

  112. Cheng, J., Wang, Z., Zhou, H., Li, L., Yao, J.: DM-SLAM: a feature-based SLAM system for rigid dynamic scenes. ISPRS Int J Geo-Inf. 9, 1–18 (2020)

    Article  Google Scholar 

  113. Liu, Y., Miura, J.: RDS-SLAM: real-time dynamic SLAM using semantic segmentation methods. IEEE Access. 9, 23772–23785 (2021)

    Article  Google Scholar 

  114. Fan, Y., Fan, Y., Zhang, Q., et al.: Semantic SLAM with more accurate point cloud map in dynamic environments. IEEE Access. 8, 112237–112252 (2020)

    Article  Google Scholar 

  115. Ji, T., Wang, C., Xie, L.: Towards Real-time Semantic RGB-D SLAM in Dynamic Environments. pp. 11175–11181

  116. Li, A., Wang, J., Xu, M., Chen, Z.: DP-SLAM: a visual SLAM with moving probability towards dynamic environments. Inf Sci (Ny). 556, 128–142 (2021)

    Article  Google Scholar 

  117. Qin, T., Li, P., Shen, S.: VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator. IEEE Trans. Robot. 34, 1004–1020 (2018)

    Article  Google Scholar 

  118. Zhao, X., Wang, C., Ang, M.H.: Real-time visual-inertial localization using semantic segmentation towards dynamic environments. IEEE Access. 8, 155047–155059 (2020)

    Article  Google Scholar 

  119. Cortes, S., Solin, A., Rahtu, E., et al.: ADVIO: An Authentic Dataset for Visual-Inertial Odometry. 419–434 (2018)

  120. Whaley, S.: What is Frame Rate and Why is it Important to PC Gaming?, (2018) https://www.hp.com/us-en/shop/tech-takes/what-is-frame-rate, (Accessed 4 September 2021)

  121. Read, P., Meyer, M.-P.: Restoration of Motion Picture Film, (2000). https://books.google.ae/books?hl=en&lr=&id=jzbUUL0xJAEC&oi=fnd&pg=PP1&dq=Read,+Paul%3B+Meyer,+Mark-Paul%3B+Gamma+Group+(2000).+Restoration+of+motion+picture+film&ots=orlihkxMBR&sig=RFItd8lo1LfvN5uLxi9puN-3FJo&redir_esc=y#v=onepage&q=Read%252C.Paul%253B Meyer%25, (Accessed 4 September 2021)

  122. Brownlow, K.: Silent Films: What Was the Right Speed?, (1980) https://web.archive.org/web/20110708155615/http://www.cinemaweb.com/silentfilm/bookshelf/18_kb_2.htm, (Accessed 4 September 2021)

  123. Armstrong, M.G., Flynn, D.J., Hammond, M.E., Jolly, S.J.E., Salmon, R.A.: High frame-rate television. SMPTE Motion Imaging J. 118, 54–58 (2009)

    Article  Google Scholar 

  124. Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y.M.: YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv, (2020) http://arxiv.org/abs/2004.10934, (Accessed 12 May 2021) (2021)

  125. Soares, J.C.V., Gattass, M., Meggiolaro, M.A.: Visual SLAM in human populated environments: exploring the trade-off between accuracy and speed of YOLO and mask R-CNN. In: 2019 19th International Conference on Advanced Robotics (ICAR), pp. 135–140 (2019). https://doi.org/10.1109/ICAR46387.2019.8981617

    Chapter  Google Scholar 

  126. Dong, G., Yan, Y., Shen, C., Wang, H.: Real-time high-performance semantic image segmentation of urban street scenes. IEEE Trans. Intell. Transp. Syst. 22, 3258–3274 (2021)

    Article  Google Scholar 

  127. Javed, Z., Kim, G.: A comparative study of recent real time semantic segmentation algorithms for visual semantic SLAM. IEEE International Conference on Big Data and Smart Computing (BigComp). 2020, 474–476 (2020). https://doi.org/10.1109/BigComp48618.2020.00-22

    Article  Google Scholar 

  128. Kyrkou, C., Theocharides, T.: EmergencyNet: efficient aerial image classification for drone-based emergency monitoring using Atrous convolutional feature fusion. IEEE J Sel Top Appl Earth Obs Remote Sens. 13, 1687–1699 (2020)

    Article  Google Scholar 

  129. Everingham, M., Van Gool, L., Williams, C.K.I., et al.: The Pascal Visual Object Classes (VOC) Challenge. Int. J. Comput. Vis. 88, 303–338 (2009)

    Article  Google Scholar 

  130. Lin, TY. et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture notes in computer science, vol 8693. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

  131. Schneider, T., Dymczyk, M., Fehr, M., Egger, K., Lynen, S., Gilitschenski, I., Siegwart, R.: Maplab: an open framework for research in visual-inertial mapping and localization. IEEE Robot Autom Lett. 3, 1418–1425 (2018)

    Article  Google Scholar 

  132. Mur-Artal, R., Tardos, J.D.: Visual-inertial monocular SLAM with map reuse. IEEE Robot Autom Lett. 2, 796–803 (2017)

    Article  Google Scholar 

  133. Zhao, H., Qi, X., Shen, X., et al.: ICNet for Real-Time Semantic Segmentation on High-Resolution Images, (2018) https://github.com/hszhao/ICNet, (Accessed 23 May 2021) (2021)

  134. Cordts M, Omran M, Ramos S, et al. The Cityscapes Dataset for Semantic Urban Scene Understanding, www.cityscapes-dataset.net (2016, accessed 23 May 2021) (2016)

  135. Gaidon, A., Wang, Q., Cabon, Y., et al. Virtual Worlds as Proxy for Multi-Object Tracking Analysis, (2016) http://www.xrce.xerox.com/, (Accessed 23 May 2021)

  136. Ros, G., Sellart, L., Materzynska, J., Vazquez, D., Lopez, A.M.: The SYNTHIA dataset: a large collection of synthetic images for semantic segmentation of urban scenes. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016, 3234–3243 (2016). https://doi.org/10.1109/CVPR.2016.352

    Article  Google Scholar 

  137. Liu, W., et al.: SSD: single shot MultiBox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds) Computer Vision – ECCV 2016. ECCV 2016. Lecture Notes in Computer Science, vol 9905. Springer, Cham (2016) https://doi.org/10.1007/978-3-319-46448-0_2

  138. Chen, L.-C., Zhu, Y., Papandreou, G., et al.: Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation, (2018) https://github.com/tensorflow/models/tree/master/ Accessed 23 May 2021)

  139. Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D SLAM systems. IEEE/RSJ International Conference on Intelligent Robots and Systems. 2012, 573–580 (2012). https://doi.org/10.1109/IROS.2012.6385773

    Article  Google Scholar 

  140. Geiger, A., Lenz, P., Stiller, C., et al.: Vision meets robotics. The KITTI dataset. https://doi.org/10.1177/0278364913491297 Epub ahead of print 23 August 2013

  141. Baker, S., Matthews, I.: Lucas-Kanade 20 years on: a unifying framework. Int. J. Comput. Vis. 56, 221–255 (2004)

    Article  MATH  Google Scholar 

Download references

Availability of Data and Material

Not Applicable.

Code Availability

Not Applicable.

Funding

This work was supported by the United Arab Emirates University's Research Start-Up Grant (G00003527).

Author information

Authors and Affiliations

  1. Mechanical and Aerospace Engineering Department, College of Engineering, United Arab Emirates University (UAEU), Al Ain, United Arab Emirates

    Wan Faris Aizat Wan Aasim & Mohamed Okasha

  2. Mechanical and Aerospace Engineering Department, College of Engineering, International Islamic University Malaysia (IIUM), Kuala Lumpur, Malaysia

    Waleed Fekry Faris

Authors
  1. Wan Faris Aizat Wan Aasim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed Okasha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Waleed Fekry Faris
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The idea for the article, its literature search and analysis along with its eventual draft was done by Aizat Aasim. While extensive critical review of the work done was handled equally between Mohamed Okasha and Waleed Fekry Faris.

Corresponding author

Correspondence to Mohamed Okasha.

Ethics declarations

Ethics Approval

Not Applicable.

Conflicts of Interest/Competing Interests

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Consent to Participate

Not Applicable.

Consent for Publication

Not Applicable.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wan Aasim, W.F.A., Okasha, M. & Faris, W.F. Real-Time Artificial Intelligence Based Visual Simultaneous Localization and Mapping in Dynamic Environments – a Review. J Intell Robot Syst 105, 15 (2022). https://doi.org/10.1007/s10846-022-01643-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-022-01643-y

Keywords

Search

Navigation