Gait coordination feature modeling and multi-scale gait representation for gait recognition | International Journal of Machine Learning and Cybernetics Skip to main content
Springer Nature Link
Log in

Gait coordination feature modeling and multi-scale gait representation for gait recognition

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Gait recognition is an advanced biometric modality that identifies individuals based on their walking postures. Currently, the majority of gait recognition methods employ silhouette images as the primary representation of gait. However, the inclusion of silhouette information, which incorporates details about clothing and carried items, may interfere with the extraction of gait features. To this end, we present GaitSkeleton, a novel approach that utilizes skeleton sequences as the gait representation. GaitSkeleton models the motor coordination feature of gait and combines three-scale spatial features to obtain discriminative features for gait recognition. To capture the coordination relationship between joints, we design a Coordination Feature Learner (CFL). CFL is embedded into the ST-GCN block to enhance the model’s representational capacity. Additionally, we introduce a novel feature cascade module called MSCM, which further enhances the discriminative information by sampling spatial features at three scales. The entire network significantly enhances the discriminative nature of gait features. Experimental results on the CASIA-B dataset demonstrate that our method achieves state-of-the-art performance in model-based gait recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability Statement

The public gait dataset CASIA-B is available in http://www.cbsr.ia.ac.cn/english/Gait%20Databases.asp. The prepared dataset can be downloaded from https://github.com/tteepe/GaitGraph. The implementation code is available from the corresponding author on reasonable request.

Notes

  1. http://www.cbsr.ia.ac.cn/english/Gait%20Databases.asp.

References

  1. Deligianni F, Guo Y, Yang G-Z (2019) From emotions to mood disorders: a survey on gait analysis methodology. IEEE J Biomed Health Inform 23(6):2302–2316

    Article  Google Scholar 

  2. Singh JP, Jain S, Arora S, Singh UP (2018) Vision-based gait recognition: a survey. IEEE Access 6:70497–70527

    Article  Google Scholar 

  3. Etemad SA, Arya A (2014) Classification and translation of style and affect in human motion using RBF neural networks. Neurocomputing 129:585–595

    Article  Google Scholar 

  4. Wan C, Wang L, Phoha VV (2018) A survey on gait recognition. ACM Comput Surv (CSUR) 51(5):1–35

    Article  Google Scholar 

  5. Rida I, Almaadeed N, Almaadeed S (2019) Robust gait recognition: a comprehensive survey. IET Biom 8(1):14–28

    Article  Google Scholar 

  6. Nambiar A, Bernardino A, Nascimento JC (2019) Gait-based person re-identification: a survey. ACM Comput Surv (CSUR) 52(2):1–34

    Article  Google Scholar 

  7. Sepas-Moghaddam A, Etemad A (2022) Deep gait recognition: a survey. IEEE Trans Pattern Anal Mach Intell

  8. Chen C, Liang J, Zhao H, Hu H, Tian J (2009) Frame difference energy image for gait recognition with incomplete silhouettes. Pattern Recognit Lett 30(11):977–984

    Article  Google Scholar 

  9. Uddin MZ, Muramatsu D, Takemura N, Ahad MAR, Yagi Y (2019) Spatio-temporal silhouette sequence reconstruction for gait recognition against occlusion. IPSJ Trans Comput Vis Appl 11(1):1–18

    Google Scholar 

  10. Ibrahim A, Mohd-Isa W-N, Ho CC (2018) Background subtraction on gait videos containing illumination variates. In: AIP conference proceedings, vol 2016. AIP Publishing LLC, p 020057

  11. Sepas-Moghaddam A, Etemad A (2020) View-invariant gait recognition with attentive recurrent learning of partial representations. IEEE Trans Biom Behav Identity Sci 3(1):124–137

    Article  Google Scholar 

  12. Xing W, Li Y, Zhang S (2018) View-invariant gait recognition method by three-dimensional convolutional neural network. J Electron Imaging 27(1):013010

    Article  Google Scholar 

  13. Batchuluun G, Yoon HS, Kang JK, Park KR (2018) Gait-based human identification by combining shallow convolutional neural network-stacked long short-term memory and deep convolutional neural network. IEEE Access 6:63164–63186

    Article  Google Scholar 

  14. Feng Y, Li Y, Luo J (2016) Learning effective gait features using LSTM. In: 2016 23rd International conference on pattern recognition (ICPR). IEEE, pp 325–330

  15. Liu D, Ye M, Li X, Zhang F, Lin L (2016) Memory-based gait recognition. In: BMVC, pp 1–12

  16. Liao R, Cao C, Garcia EB, Yu S, Huang Y (2017) Pose-based temporal-spatial network (PTSN) for gait recognition with carrying and clothing variations. In: Chinese conference on biometric recognition. Springer, Berlin, pp 474–483

  17. Kanazawa A, Black MJ, Jacobs DW, Malik J (2018) End-to-end recovery of human shape and pose. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7122–7131

  18. Kocabas M, Athanasiou N, Black MJ (2020) Vibe: video inference for human body pose and shape estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5253–5263

  19. Sun Y, Bao Q, Liu W, Fu Y, Black MJ, Mei T (2021) Monocular, one-stage, regression of multiple 3d people. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 11179–11188

  20. Cao Z, Simon T, Wei S-E, Sheikh Y (2017) Realtime multi-person 2d pose estimation using part affinity fields. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7291–7299

  21. Fang H-S, Xie S, Tai Y-W, Lu C (2017) RMPE: regional multi-person pose estimation. In: Proceedings of the IEEE international conference on computer vision, pp 2334–2343

  22. Sun K, Xiao B, Liu D, Wang J (2019) Deep high-resolution representation learning for human pose estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5693–5703

  23. Yu S, Tan D, Tan T (2006) A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In: 18th International conference on pattern recognition (ICPR’06), vol 4. IEEE, pp 441–444

  24. Wang X, Yan WQ (2020) Cross-view gait recognition through ensemble learning. Neural Comput Appl 32(11):7275–7287

    Article  Google Scholar 

  25. Li X, Makihara Y, Xu C, Yagi Y, Ren M (2020) Gait recognition invariant to carried objects using alpha blending generative adversarial networks. Pattern Recognit 105:107376

    Article  Google Scholar 

  26. Wang X, Yan WQ (2020) Human gait recognition based on frame-by-frame gait energy images and convolutional long short-term memory. Int J Neural Syst 30(01):1950027

    Article  Google Scholar 

  27. Xu C, Makihara Y, Li X, Yagi Y, Lu J (2020) Cross-view gait recognition using pairwise spatial transformer networks. IEEE Trans Circuits Syst Video Technol 31(1):260–274

    Article  Google Scholar 

  28. Mehmood A, Khan MA, Sharif M, Khan SA, Shaheen M, Saba T, Riaz N, Ashraf I (2020) Prosperous human gait recognition: an end-to-end system based on pre-trained CNN features selection. Multimed Tools Appl 1–21

  29. Elharrouss O, Almaadeed N, Al-Maadeed S, Bouridane A (2021) Gait recognition for person re-identification. J Supercomput 77(4):3653–3672

    Article  Google Scholar 

  30. Tong S, Fu Y, Yue X, Ling H (2018) Multi-view gait recognition based on a spatial-temporal deep neural network. IEEE Access 6:57583–57596

    Article  Google Scholar 

  31. Li S, Liu W, Ma H (2019) Attentive spatial-temporal summary networks for feature learning in irregular gait recognition. IEEE Trans Multimed 21(9):2361–2375

    Article  Google Scholar 

  32. Zhang Y, Huang Y, Yu S, Wang L (2019) Cross-view gait recognition by discriminative feature learning. IEEE Trans Image Process 29:1001–1015

    Article  MathSciNet  Google Scholar 

  33. Zhao A, Li J, Ahmed M (2020) SpiderNet: a spiderweb graph neural network for multi-view gait recognition. Knowl Based Syst 206:106273

    Article  Google Scholar 

  34. Wang H, Fan Y, Fang B, Dai S (2018) Generalized linear discriminant analysis based on Euclidean norm for gait recognition. Int J Mach Learn Cybern 9(4):569–576

    Article  Google Scholar 

  35. Shiraga K, Makihara Y, Muramatsu D, Echigo T, Yagi Y (2016) GEINet: View-invariant gait recognition using a convolutional neural network. In: 2016 International conference on biometrics (ICB). IEEE, pp 1–8

  36. Thapar D, Nigam A, Aggarwal D, Agarwal P (2018) VGR-net: a view invariant gait recognition network. In: 2018 IEEE 4th International Conference on Identity, Security, and Behavior Analysis (ISBA). IEEE, pp. 1–8

  37. Chao H, He Y, Zhang J, Feng J (2019) Gaitset: Regarding gait as a set for cross-view gait recognition. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 8126–8133

  38. Zhang Z, Tran L, Yin X, Atoum Y, Liu X, Wan J, Wang N (2019) Gait recognition via disentangled representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4710–4719

  39. Li X, Makihara Y, Xu C, Yagi Y, Ren M (2020) Gait recognition via semi-supervised disentangled representation learning to identity and covariate features. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13309–13319

  40. Hou S, Cao C, Liu X, Huang Y (2020) Gait lateral network: learning discriminative and compact representations for gait recognition. In: European conference on computer vision. Springer, Berlin, pp 382–398

  41. Fan C, Peng Y, Cao C, Liu X, Hou S, Chi J, Huang Y, Li Q, He Z (2020) Gaitpart: temporal part-based model for gait recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 14225–14233

  42. Lin B, Zhang S, Bao F (2020) Gait recognition with multiple-temporal-scale 3d convolutional neural network. In: Proceedings of the 28th ACM international conference on multimedia, pp 3054–3062

  43. Lin B, Zhang S, Liu Y, Qin S (2021) Multi-scale temporal information extractor for gait recognition. In: 2021 IEEE international conference on image processing (ICIP). IEEE, pp 2998–3002

  44. Huang X, Zhu D, Wang H, Wang X, Yang B, He B, Liu W, Feng B (2021) Context-sensitive temporal feature learning for gait recognition. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 12909–12918

  45. Lin B, Zhang S, Yu X (2021) Gait recognition via effective global-local feature representation and local temporal aggregation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 14648–14656

  46. An W, Liao R, Yu S, Huang Y, Yuen PC (2018) Improving gait recognition with 3d pose estimation. In: Chinese conference on biometric recognition. Springer, Berlin, pp 137–147

  47. Jun K, Lee D-W, Lee K, Lee S, Kim MS (2020) Feature extraction using an RNN autoencoder for skeleton-based abnormal gait recognition. IEEE Access 8:19196–19207

    Article  Google Scholar 

  48. Li N, Zhao X, Ma C (2020) JointsGait: a model-based gait recognition method based on gait graph convolutional networks and joints relationship pyramid mapping. arXiv preprint. arXiv:2005.08625

  49. An W, Yu S, Makihara Y, Wu X, Xu C, Yu Y, Liao R, Yagi Y (2020) Performance evaluation of model-based gait on multi-view very large population database with pose sequences. IEEE Trans Biom Behav Identity Sci 2(4):421–430

    Article  Google Scholar 

  50. Liao R, Yu S, An W, Huang Y (2020) A model-based gait recognition method with body pose and human prior knowledge. Pattern Recognit 98:107069

    Article  Google Scholar 

  51. Teepe T, Khan A, Gilg J, Herzog F, Hörmann S, Rigoll G (2021) Gaitgraph: graph convolutional network for skeleton-based gait recognition. In: 2021 IEEE international conference on image processing (ICIP). IEEE, pp 2314–2318

  52. Song Y-F, Zhang Z, Shan C, Wang L (2020) Stronger, faster and more explainable: a graph convolutional baseline for skeleton-based action recognition. In: Proceedings of the 28th ACM international conference on multimedia, pp 1625–1633

  53. Khosla P, Teterwak P, Wang C, Sarna A, Tian Y, Isola P, Maschinot A, Liu C, Krishnan D (2020) Supervised contrastive learning. Adv Neural Inf Process Syst 33:18661–18673

    Google Scholar 

  54. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7794–7803

  55. Zhang H, Goodfellow I, Metaxas D, Odena A (2019) Self-attention generative adversarial networks. In: International conference on machine learning. PMLR, pp 7354–7363

  56. Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: common objects in context. In: European conference on computer vision. Springer, Berlin, pp 740–755

  57. Takemura N, Makihara Y, Muramatsu D, Echigo T, Yagi Y (2018) Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition. IPSJ Trans Comput Vis Appl 10(1):1–14

    Google Scholar 

  58. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint. arXiv:1412.6980

  59. Smith LN, Topin N (2019) Super-convergence: very fast training of neural networks using large learning rates. In: Artificial intelligence and machine learning for multi-domain operations applications, vol 11006. International Society for Optics and Photonics, p 1100612

Download references

Funding

This research was supported by the Science Foundation of China University of Petroleum, Beijing (no. 2462020YXZZ024), National Key R &D Program of China (2019YFA0708300) and National Natural Science Foundation of China (Grant no. 52074323).

Author information

Author notes

  1. Dandan Zhu, Laihu Ji and Liping Zhu contributed equally to this work.

Authors and Affiliations

  1. College of Information Science and Engineering, China University of Petroleum (Beijing), Fuxue Road, Beijing, 102200, China

    Dandan Zhu, Laihu Ji & Liping Zhu

  2. Beijing Key Laboratory of Petroleum Data Mining, China University of Petroleum (Beijing), Fuxue Road, Beijing, 102200, China

    Dandan Zhu, Laihu Ji & Liping Zhu

  3. School of Computer Science, Peking University, Yiheyuan Road, Beijing, 100871, China

    Chengyang Li

Authors
  1. Dandan Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laihu Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liping Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chengyang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Laihu Ji.

Ethics declarations

Conflict of interest

The authors declare that they have no competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, D., Ji, L., Zhu, L. et al. Gait coordination feature modeling and multi-scale gait representation for gait recognition. Int. J. Mach. Learn. & Cyber. 15, 3791–3802 (2024). https://doi.org/10.1007/s13042-024-02120-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-024-02120-8

Keywords

Search

Navigation