{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,3]],"date-time":"2024-09-03T05:55:31Z","timestamp":1725342931172},"reference-count":46,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2020,4,30]],"date-time":"2020-04-30T00:00:00Z","timestamp":1588204800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"Natural Science Foundation of China","doi-asserted-by":"publisher","award":["41731066"],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2018YFC1505100"],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"Several pedestrian navigation solutions have been proposed to date, and most of them are based on smartphones. Real-time recognition of pedestrian mode and smartphone posture is a key issue in navigation. Traditional ML (Machine Learning) classification methods have drawbacks, such as insufficient recognition accuracy and poor timing. This paper presents a real-time recognition scheme for comprehensive human activities, and this scheme combines deep learning algorithms and MEMS (Micro-Electro-Mechanical System) sensors\u2019 measurements. In this study, we performed four main experiments, namely pedestrian motion mode recognition, smartphone posture recognition, real-time comprehensive pedestrian activity recognition, and pedestrian navigation. In the procedure of recognition, we designed and trained deep learning models using LSTM (Long Short-Term Memory) and CNN (Convolutional Neural Network) networks based on Tensorflow framework. The accuracy of traditional ML classification methods was also used for comparison. Test results show that the accuracy of motion mode recognition was improved from 89.9 % , which was the highest accuracy and obtained by SVM (Support Vector Machine), to 90.74 % (LSTM) and 91.92 % (CNN); the accuracy of smartphone posture recognition was improved from 81.60 % , which is the highest accuracy and obtained by NN (Neural Network), to 93.69 % (LSTM) and 95.55 % (CNN). We give a model transformation procedure based on the trained CNN network model, and then obtain the converted . t f l i t e model, which can be run in Android devices for real-time recognition. Real-time recognition experiments were performed in multiple scenes, a recognition model trained by the CNN network was deployed in a Huawei Mate20 smartphone, and the five most used pedestrian activities were designed and verified. The overall accuracy was up to 89.39 % . Overall, the improvement of recognition capability based on deep learning algorithms was significant. Therefore, the solution was helpful to recognize comprehensive pedestrian activities during navigation. On the basis of the trained model, a navigation test was performed; mean bias was reduced by more than 1.1 m. Accordingly, the positioning accuracy was improved obviously, which is meaningful to apply DL in the area of pedestrian navigation to make improvements.<\/jats:p>","DOI":"10.3390\/s20092574","type":"journal-article","created":{"date-parts":[[2020,5,4]],"date-time":"2020-05-04T18:00:43Z","timestamp":1588615243000},"page":"2574","source":"Crossref","is-referenced-by-count":18,"title":["Deep Learning-Based Human Activity Real-Time Recognition for Pedestrian Navigation"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-5796-3612","authenticated-orcid":false,"given":"Junhua","family":"Ye","sequence":"first","affiliation":[{"name":"College of Geology Engineering and Geomantic, Chang\u2019an University, Xi\u2019an 710054, China"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-1255-9173","authenticated-orcid":false,"given":"Xin","family":"Li","sequence":"additional","affiliation":[{"name":"College of Geology Engineering and Geomantic, Chang\u2019an University, Xi\u2019an 710054, China"}]},{"given":"Xiangdong","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Geology Engineering and Geomantic, Chang\u2019an University, Xi\u2019an 710054, China"}]},{"given":"Qin","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Geology Engineering and Geomantic, Chang\u2019an University, Xi\u2019an 710054, China"}]},{"given":"Wu","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Land Surveying and Geo-Informatics, Hong Kong Polytechnic University, Hong Kong 999077, China"}]}],"member":"1968","published-online":{"date-parts":[[2020,4,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ye, J., Li, Y., Luo, H., Wang, J., Chen, W., and Zhang, Q. (2019). Hybrid Urban Canyon Pedestrian Navigation Scheme Combined PDR, GNSS and Beacon Based on Smartphone. Remote Sens., 11.","DOI":"10.3390\/rs11182174"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kakiuchi, N., and Kamijo, S. (2013, January 6\u20139). Pedestrian dead reckoning for mobile phones through walking and running mode recognition. Proceedings of the International IEEE Conference on Intelligent Transportation Systems, The Hague, The Netherlands.","DOI":"10.1109\/ITSC.2013.6728243"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Micucci, D., Mobilio, M., and Napoletano, P. (2017). UniMiB SHAR: A Dataset for Human Activity Recognition Using Acceleration Data from Smartphones. Appl. Sci., 7.","DOI":"10.20944\/preprints201706.0033.v2"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"503291","DOI":"10.1155\/2014\/503291","article-title":"Activity Recognition on Smartphones via Sensor-Fusion and KDA-Based SVMs","volume":"10","author":"Khan","year":"2014","journal-title":"Int. J. Distrib. Sens. Netw."},{"key":"ref_5","unstructured":"Kwapisz, J.R., Weiss, G.M., and Moore, S.A. (2010, January 25\u201328). Activity Recognition using Cell Phone Accelerometers. Proceedings of the Fourth International Workshop on Knowledge Discovery from Sensor Data (at KDD-10), Washington, DC, USA."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Yang, J., Cheng, K., Chen, J., Zhou, B., and Li, Q. (2018, January 22\u201323). Smartphones based Online Activity Recognition for Indoor Localization using Deep Convolutional Neural Network. Proceedings of the 2018 Ubiquitous Positioning, Indoor Navigation and Location-Based Services (UPINLBS), Wuhan, China.","DOI":"10.1109\/UPINLBS.2018.8559719"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Klein, I., Solaz, Y., and Ohayon, G. (2017). Smartphone Motion Mode Recognition. Proceedings, 2.","DOI":"10.3390\/ecsa-4-04929"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Li, F., Shirahama, K., Nisar, M.A., K\u00f6ping, L., and Grzegorzek, M. (2018). Comparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors. Sensors, 18.","DOI":"10.3390\/s18020679"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wang, B., Liu, X., Yu, B., Jia, R., and Gan, X. (2018). Pedestrian Dead Reckoning Based on Motion Mode Recognition Using a Smartphone. Sensors, 18.","DOI":"10.3390\/s18061811"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zhou, B., Yang, J., and Li, Q. (2019). Smartphone-Based Activity Recognition for Indoor Localization Using a Convolutional Neural Network. Sensors, 19.","DOI":"10.3390\/s19030621"},{"key":"ref_11","first-page":"93","article-title":"Human Activity Recognition Supported on Indoor Localization: A Systematic Review","volume":"249","author":"Ceron","year":"2018","journal-title":"Stud. Health Technol. Inform."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wu, J., Feng, Y., and Sun, P. (2018). Sensor Fusion for Recognition of Activities of Daily Living. Sensors, 18.","DOI":"10.3390\/s18114029"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Luo, H., Wang, Q., Zhao, F., Ning, B., Ke, Q., and Zhang, C. (2019). A Fast Indoor\/Outdoor Transition Detection Algorithm Based on Machine Learning. Sensors, 19.","DOI":"10.3390\/s19040786"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Niitsoo, A., Edelh\u00e4u\u00dfer, T., Eberlein, E., Hadaschik, N., and Mutschler, C. (2019). A Deep Learning Approach to Position Estimation from Channel Impulse Responses. Sensors, 19.","DOI":"10.3390\/s19051064"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Manos, A., Klein, I., and Hazan, T. (2019). Gravity-Based Methods for Heading Computation in Pedestrian Dead Reckoning. Sensors, 19.","DOI":"10.3390\/s19051170"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1109\/MC.2018.2381112","article-title":"Deep Learning for Human Activity Recognition in Mobile Computing","volume":"51","author":"Guan","year":"2018","journal-title":"Computer"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"21219","DOI":"10.3390\/s150921219","article-title":"Inferring Human Activity in Mobile Devices by Computing Multiple Contexts","volume":"15","author":"Chen","year":"2015","journal-title":"Sensors"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1016\/j.eswa.2018.03.056","article-title":"Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges","volume":"105","author":"Nweke","year":"2018","journal-title":"Expert Syst. Appl."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Fan, L., Wang, Z., and Wang, H. (2013, January 13\u201315). Human Activity Recognition Model Based on Decision Tree. Proceedings of the 2013 International Conference on Advanced Cloud and Big Data, Nanjing, China.","DOI":"10.1109\/CBD.2013.19"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"867","DOI":"10.1016\/j.aei.2015.03.001","article-title":"Construction equipment activity recognition for simulation input modeling using mobile sensors and machine learning classifiers","volume":"29","author":"Akhavian","year":"2015","journal-title":"Adv. Eng. Inform."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zeng, M., Nguyen, L.T., Yu, B., Mengshoel, O.J., Zhu, J., Wu, P., and Zhang, J. (2014, January 6\u20137). Convolutional Neural Networks for human activity recognition using mobile sensors. Proceedings of the 6th International Conference on Mobile Computing, Applications and Services, Austin, TX, USA.","DOI":"10.4108\/icst.mobicase.2014.257786"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1016\/j.procs.2014.07.009","article-title":"A Study on Human Activity Recognition Using Accelerometer Data from Smartphones","volume":"34","author":"Bayat","year":"2014","journal-title":"Procedia Comput. Sci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"025103","DOI":"10.1088\/0957-0233\/23\/2\/025103","article-title":"Pedestrian dead reckoning employing simultaneous activity recognition cues","volume":"23","author":"Altun","year":"2012","journal-title":"Meas. Sci. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1007\/s10916-016-0497-2","article-title":"Smartphone-Based Patients\u2019 Activity Recognition by Using a Self-Learning Scheme for Medical Monitoring","volume":"40","author":"Guo","year":"2016","journal-title":"J. Med. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"6067","DOI":"10.1016\/j.eswa.2014.04.037","article-title":"Unsupervised learning for human activity recognition using smartphone sensors","volume":"41","author":"Kwon","year":"2014","journal-title":"Expert Syst. Appl."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1016\/j.asoc.2017.09.027","article-title":"Real-time human activity recognition from accelerometer data using Convolutional Neural Networks","volume":"62","author":"Ignatov","year":"2018","journal-title":"Appl. Soft Comput."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Wang, K., Wang, X., Lin, L., Wang, M., and Zuo, W. (2014, January 18\u201319). 3D Human Activity Recognition with Reconfigurable Convolutional Neural Networks. Proceedings of the 22nd ACM International Conference on Multimedia, Mountain View, CA, USA.","DOI":"10.1145\/2647868.2654912"},{"key":"ref_28","unstructured":"Hammerla, N.Y., Halloran, S., and Pl\u00f6tz, T. (2016). Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearables. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Morales, F.J.O., and Roggen, D. (2016, January 12\u201316). Deep convolutional feature transfer across mobile activity recognition domains, sensor modalities and locations. Proceedings of the 2016 ACM International Symposium on Wearable Computers, Heidelberg, Germany.","DOI":"10.1145\/2971763.2971764"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1007\/s10916-018-0948-z","article-title":"Human Activity Recognition from Body Sensor Data using Deep Learning","volume":"42","author":"Hassan","year":"2018","journal-title":"J. Med. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1016\/j.future.2017.11.029","article-title":"A robust human activity recognition system using smartphone sensors and deep learning","volume":"81","author":"Hassan","year":"2018","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Jiang, W., and Yin, Z. (2015, January 26\u201330). Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks. Proceedings of the 23rd ACM International Conference on Multimedia, Brisbane, Australia.","DOI":"10.1145\/2733373.2806333"},{"key":"ref_33","unstructured":"Alsheikh, M.A., Seleim, A.A., Niyato, D., Doyle, L., Lin, S., and Tan, H.P. (2016, January 12\u201313). Deep Activity Recognition Models with Triaxial Accelerometers. Proceedings of the Workshops at the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, Arizona."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Wang, Q., Ye, L., Luo, H., Men, A., Zhao, F., and Huang, Y. (2019). Pedestrian Stride-Length Estimation Based on LSTM and Denoising Autoencoders. Sensors, 19.","DOI":"10.3390\/s19040840"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1662","DOI":"10.1109\/TITS.2016.2617200","article-title":"A Survey on Approaches of Motion Mode Recognition Using Sensors","volume":"18","author":"Elhoushi","year":"2017","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_36","unstructured":"(2019, August 10). Fast Fourier Transform. Available online: https:\/\/en.wikipedia.org\/wiki\/Fast_Fourier_transform."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Huang, H.Y., Hsieh, C.Y., Liu, K.C., Cheng, H.C., Hsu, S.J., and Chan, C.T. (2019). Multi-Sensor Fusion Approach for Improving Map-Based Indoor Pedestrian Localization. Sensors, 19.","DOI":"10.3390\/s19173786"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Guo, S., Xiong, H., Zheng, X., and Zhou, Y. (2017). Activity Recognition and Semantic Description for Indoor Mobile Localization. Sensors, 17.","DOI":"10.3390\/s17030649"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Deng, Z., Fu, X., and Wang, H. (2018). An IMU-Aided Body-Shadowing Error Compensation Method for Indoor Bluetooth Positioning. Sensors, 18.","DOI":"10.3390\/s18010304"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Niu, L., and Song, Y.Q. (2019). A Faster R-CNN Approach for Extracting Indoor Navigation Graph from Building Designs. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Copernicus GmbH.","DOI":"10.5194\/isprs-archives-XLII-2-W13-865-2019"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Wang, Q., Ye, L., Luo, H., Men, A., Zhao, F., and Ou, C. (2019). Pedestrian Walking Distance Estimation Based on Smartphone Mode Recognition. Remote Sens., 11.","DOI":"10.3390\/rs11091140"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1181","DOI":"10.1016\/j.procs.2015.01.031","article-title":"Smart Phone Based Data Mining for Human Activity Recognition","volume":"46","author":"Chetty","year":"2015","journal-title":"Procedia Comput. Sci."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.patrec.2018.02.010","article-title":"Deep learning for sensor-based activity recognition: A survey","volume":"119","author":"Wang","year":"2019","journal-title":"Pattern Recognit. Lett."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","article-title":"An introduction to ROC analysis","volume":"27","author":"Fawcett","year":"2006","journal-title":"Pattern Recognit. Lett."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1109\/TIM.2015.2477159","article-title":"Motion Mode Recognition for Indoor Pedestrian Navigation Using Portable Devices","volume":"65","author":"Elhoushi","year":"2016","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_46","unstructured":"Anguita, D., Ghio, A., Oneto, L., Parra, X., and Reyes-Ortiz, J.L. (2013, January 24\u201326). A Public Domain Dataset for Human Activity Recognition Using Smartphones. Proceedings of the 21th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, ESANN 2013, Bruges, Belgium."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/9\/2574\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,27]],"date-time":"2024-06-27T13:37:19Z","timestamp":1719495439000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/9\/2574"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,30]]},"references-count":46,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2020,5]]}},"alternative-id":["s20092574"],"URL":"https:\/\/doi.org\/10.3390\/s20092574","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,30]]}}}