Abstract
The paper describes a system to track the body movement of a person from a video source while augmenting the labelled skeleton joints onto the body of the person. This work has endless applications in the real world especially in the physical-demanding working environment as well as in the sports industry by implementing deep learning, the techniques can recognize the joints on a person’s body. An algorithm namely Mediapipe Blazepose has been applied using PoseNet dataset to detect and estimate curated movements specifically designed for body injury during heavy workload. The propose method has been compared to IMU based motion capture and the difference accuracy is within 10% since IMU capture real data of the sensors while the deep learning method using 2D image analysis. The expected outcome from this project is a working system that is able to correctly identify and label the skeleton joints on a person’s body as well as perform various calculation such as movement velocity and the angle of joints which could be crucial for determining whether certain body movements could result in injuries either in the short- or long-term period.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Eldar, R., Fisher-Gewirtzman, D.: Ergonomic design visualization mapping- developing an assistive model for design activities. Int. J. Ind. Ergon. 74, 102859 (2019)
Maurice, P., et al.: Human movement and ergonomics: an industry-oriented dataset for collaborative robotics. Int. J. Robot. Res. 38(14), 1529–1537 (2019)
Yunus, M.N.H., Jaafar, M.H., Mohamed, A.S.A., Azraai, N.Z., Hossain, M.: Implementation of kinetic and kinematic variables in ergonomic risk assessment using motion capture simulation: a review. Int. J. Environ. Res. Public Health18, 8342 (2021)
Bortolini, M., Gamberi, M., Pilati, F., Regattieri, A.: Automatic assessment of the ergonomic risk for manual manufacturing and assembly activities through optical motion capture technology. Procedia CIRP 72, 81–86 (2018)
Zhang, Z., Fang, Q., Gu, X.: Objective assessment of upper-limb mobility for poststroke rehabilitation. IEEE Trans. Biomed. Eng. 63, 859–868 (2016)
Ong, Z.C., Seet, Y.C., Khoo, S.Y., Noroozi, S.: Development of an economic wireless human motion analysis device for quantitative assessment of human body joint. Measurement 115, 306–15 (2018)
Fletcher, S.R., Johnson, T.L., Thrower, J.: A study to trial the use of inertial non-optical motion capture for ergonomic analysis of manufacturing work. Proc. Inst. Mech. Eng. Part B: J. Eng. Manuf. 232(1), 90–98 (2018)
Brownlee, J., What is Deep Learning? Accessed 25 Oct 2020. https://machinelearningmastery.com/what-is-deep-learning/ (2020)
Plantard, P., Auvinet, E., Le Pierres, A.S., Multon, F.: Pose estimation with a kinect for ergonomic studies: evaluation of the accuracy using a virtual mannequin. Sensors (Switzerland) 15(1), 1785–1803 (2015)
Wang, X., Hu, Y.H., Lu, M.L., Radwin, R.G.: The accuracy of a 2D video-based lifting monitor. Ergonomics 62(8), 1043–1054 (2019)
Schechter, S.: What is markerless Augmented Reality? Retrieved October 26, 2020. https://www.marxentlabs.com/what-is-markerless-augmented-reality-dead-reckoning/. Accessed 20 Oct 2020
Bazarevsky, V.: BlazePose: On-device Real-time Body Pose tracking. Accessed 18 Jun 2021. https://arxiv.org/abs/2006.10204 (2021)
Alessandro, F., Norbert, S, Markus, M., Gabriele, B., Emanuele R., Didier S.: Survey of motion tracking methods based on inertial sensors: a focus on upper limb human motion. Sensors 17, 1257 (2017)
Mohamed, A.S.A., Chingeng, P.S., Mat Isa, N.A., Surip, S.S.: Body matching algorithm using normalize dynamic time warping (NDTW) skeleton tracking for traditional dance movement. In: Badioze Zaman, H. et al. (eds.) Advances in Visual Informatics. IVIC 2017. Lecture Notes in Computer Science, vol. 10645, pp. 669-680 Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70010-6_62
Warren, T.: A closer look at Microsoft’s new Kinect sensor. https://www.theverge.com/2019/2/25/18239860/microsoft-kinect-azure-dk-hands-on-mwc-2019. Accessed 28 Nov 2020
Kinetisense. What Is a Functional Movement Screening? (2020). https://www.kinetisense.com/a-functional-movement-screen/. Accessed 28 Nov 2020
VALD Performance (n.d.). News and Research. https://valdperformance.com/blog/. Accessed 28 Nov 2020
Mourkani, S.S.: IMU-based Suit for Strength Exercises: Design, Calibration and Track (Phd Thesis), Technische Universitat Kaiserslautern, Germany (2021)
Acknowledgement
The authors are grateful to the Ministry of Higher Education Malaysia for Fundamental Research Grant Scheme with Project Code: FRGS/1/2020/STG07/USM/02/12 for supporting this documented work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Conflicts of Interest
The authors declare that they have no conflicts of interest to report regarding the present study.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Pauzi, A.S.B. et al. (2021). Movement Estimation Using Mediapipe BlazePose. In: Badioze Zaman, H., et al. Advances in Visual Informatics. IVIC 2021. Lecture Notes in Computer Science(), vol 13051. Springer, Cham. https://doi.org/10.1007/978-3-030-90235-3_49
Download citation
DOI: https://doi.org/10.1007/978-3-030-90235-3_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-90234-6
Online ISBN: 978-3-030-90235-3
eBook Packages: Computer ScienceComputer Science (R0)