An improvement of multi-scale covariance descriptor for embedded system | Journal of Real-Time Image Processing Skip to main content

Advertisement

Springer Nature Link
Log in

An improvement of multi-scale covariance descriptor for embedded system

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Video surveillance has been a major area of focus for researchers and engineers. Actually, video surveillance includes several useful and complex tasks such as tracking, human detection, re-identification and recognition. Multi-scale covariance (MSCOV) descriptor has recently grown in interest due to its good performances for person detection, re-identification and matching. Unfortunately, its original version requires heavy computations, and it is difficult to be executed in real time on embedded systems. This paper presents two aspects of improvement to adapt the MSCOV descriptor for embedded systems. First, the local binary pattern (LBP) features are introduced and a trade-off between accuracy and processing cost is used to define the best features combination. Second, parallel implementation and embedded co-processor are exploited to accelerate processing time on multi-core CPU architectures. Both optimizations are implemented and evaluated for executing a complete application of person re-identification systems. The software implementation is performed using the VIPeR dataset. Using LBP, 21.57% processing speed-up and 50% less memory requirements for the descriptor computation are achieved without any accuracy performance degradation. We also prototype the proposed design using Zynq platform based on ARM Cortex-A9. The results demonstrate the effectiveness of the parallelization and conduct more than 11 times processing speed-up against the original algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Lienhart, R., Maydt, J.: An extended set of haar-like features for rapid object detection. In: 2002 Proceedings. 2002 International Conference on Image Processing, vol. 1, pp. 900–903. IEEE (2002)

  2. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  3. Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)

    Article  Google Scholar 

  4. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)

  5. Tuzel, O., Porikli, F., Meer, P.: Region covariance: a fast descriptor for detection and classification. In: European Conference on Computer Vision, pp. 589–600. Springer (2006)

  6. Said, Y., Atri, M.: Efficient and high-performance pedestrian detector implementation for intelligent vehicles. IET Intel. Transp. Syst. 10(6), 438–444 (2016)

    Article  Google Scholar 

  7. Ayedi, W., Snoussi, H., Abid, M.: A fast multi-scale covariance descriptor for object re-identification. Pattern Recognit. Lett. 33(14), 1902–1907 (2012)

    Article  Google Scholar 

  8. Ayedi, W., Snoussi, H., Smach, F., Abid, M.: The multi-scale covariance descriptor: performances analysis in human detection. In: 2012 IEEE Workshop on Biometric Measurements and Systems for Security and Medical Applications (BIOMS), pp. 1–5. IEEE (2012)

  9. Ayedi, W., Snoussi, H., Smach, F., Abid, M.: Tree based object matching using multi-scale covariance descriptor. In: Proceedings of the International Conference on Image Processing, Computer Vision, and Pattern Recognition (IPCV), The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp) (2012)

  10. Abid, N., Ouni, T., Loukil, K., Ammari, A.C., Abid, M.: Optimized parallel model of human detection based on the multi-scale covariance descriptor. In: International Conference on Parallel Processing and Applied Mathematics, pp. 423–433. Springer (2015)

  11. Hiromoto, M., Miyamoto, R.: Hardware architecture for high-accuracy real-time pedestrian detection with cohog features. In: 2009 IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops), pp. 894–899. IEEE (2009)

  12. Kadota, R., Sugano, H., Hiromoto, M., Ochi, H., Miyamoto, R., Nakamura, Y.: Hardware architecture for hog feature extraction. In: Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, 2009. IIH-MSP’09, pp. 1330–1333, IEEE (2009)

  13. Lee, S., Son, H., Choi, J.C., Min, K.: Hog feature extractor circuit for real-time human and vehicle detection. In: TENCON 2012-2012 IEEE Region 10 Conference, pp. 1–5. IEEE (2012)

  14. Li, Y., Gai, K., Qiu, M., Dai, W., Liu, M.: Adaptive human detection approach using FPGA-based parallel architecture in reconfigurable hardware. Concurr. Comput. Pract. Exp. 29(14), 1–14 (2017)

    Google Scholar 

  15. Rettkowski, J., Boutros, A., Göhringer, D.: HW/SW co-design of the HOG algorithm on a Xilinx Zynq SoC. J. Parallel Distrib. Comput. 109, 50–62 (2017)

    Article  Google Scholar 

  16. Amiri, M., Siddiqui, F.M., Kelly, C., Woods, R., Rafferty, K., Bardak, B.: FPGA-based soft-core processors for image processing applications. J. Signal Process. Syst. 87(1), 139–156 (2017)

    Article  Google Scholar 

  17. Hsiao, P.Y., Lin, S.Y., Chen, C.Y.: A real-time fpga based human detector. In: 2016 International Symposium on Computer, Consumer and Control (IS3C), pp. 1014–1017. IEEE (2016)

  18. de Holanda, J.A.M., Cardoso, J.M.P., Marques, E.: Towards a multi-softcore fpga approach for the hog algorithm. In: 2016 IEEE 14th International Conference on Industrial Informatics (INDIN), pp. 117–122. IEEE (2016)

  19. Martelli, S., Tosato, D., Cristani, M., Murino, V.: Fast fpga-based architecture for pedestrian detection based on covariance matrices. In: 2011 18th IEEE International Conference on Image Processing (ICIP), pp. 389–392. IEEE (2011)

  20. Dai, X., Khamis, S., Zhang, Y., Davis, L.S.: Parameterizing region covariance: an efficient way to apply sparse codes on second order statistics. Comput. Vis. Pattern Recognit. 1602, 1–10 (2016)

    Google Scholar 

  21. Cukur, H., Binol, H., Bal, A., Yavuz, F.: Covariance descriptor fusion for target detection. In: Signal Processing, Sensor/Information Fusion, and Target Recognition XXV. vol. 9842. International Society for Optics and Photonics (2016)

  22. Pang, Y., Yuan, Y., Li, X.: Gabor-based region covariance matrices for face recognition. IEEE Trans. Circ. Syst. Video Technol. 18(7), 989–993 (2008)

    Article  Google Scholar 

  23. Zhang, Y., Li, S.: Gabor-LBP based region covariance descriptor for person re-identification. In: 2011 Sixth International Conference on Image and Graphics (ICIG), pp. 368–371. IEEE (2011)

  24. Guo, S., Ruan, Q.: Facial expression recognition using local binary covariance matrices. In: 4th IET International Conference on Wireless, Mobile & Multimedia Networks (ICWMMN 2011), pp. 237–242. IET (2011)

  25. Romero, A., Gouiffès, M., Lacassagne, L.: Enhanced local binary covariance matrices (ELBCM) for texture analysis and object tracking. In: Proceedings of the 6th International Conference on Computer Vision/Computer Graphics Collaboration Techniques and Applications, vol. 10. ACM (2013)

  26. Karczmarek, P., Kiersztyn, A., Pedrycz, W., Dolecki, M.: An application of chain code-based local descriptor and its extension to face recognition. Pattern Recognit. 65, 26–34 (2017)

    Article  Google Scholar 

  27. Gao, F., Huang, Z., Wang, S., Ji, X.: Optimized parallel implementation of face detection based on embedded heterogeneous many-core architecture. Int. J. Pattern Recognit. Artif. Intell. 31(7), 1756011–1756034 (2017)

    Article  MathSciNet  Google Scholar 

  28. Jain, S., Durgesh, M., Ramesh, T.: Facial expression recognition using variants of LBP and classifier fusion. In: Proceedings of International Conference on ICT for Sustainable Development, pp. 725–732. Springer (2016)

  29. Zhang, M., Wang, B., Zhou, S., Pan, Z.: Dynamic gesture recognition based on edge feature enhancement using sobel operator. In: International Conference on Technologies for E-Learning and Digital Entertainment, pp. 152–163. Springer (2017)

  30. Yao, Z., En-zeng, D., Xiao, Y.: Design of real-time edge detection system based on the improved sobel operator and its FPGA implementation. J. Tianjin Univ. Technol. 1, 001 (2017)

    Google Scholar 

  31. Pietikäinen, M., Hadid, A., Zhao, G., Ahonen, T.: Local binary patterns for still images. In: Computer vision using local binary patterns, pp. 13–47. Springer (2011)

  32. Gray, D., Brennan, S., Tao, H.: Evaluating appearance models for recognition, reacquisition, and tracking. In: Proceedings on IEEE International Workshop on Performance Evaluation for Tracking and Surveillance (PETS). vol. 3, pp. 1–7. Citeseer (2007)

  33. Mohan, A., Papageorgiou, C., Poggio, T.: Example-based object detection in images by components. IEEE Trans. Pattern Anal. Mach. Intell. 23(4), 349–361 (2001)

    Article  Google Scholar 

  34. INRIA Person Dataset. [Online] http://pascal.inrialpes.fr/data/human/. Accessed 24 Aug 2016

  35. Ammari, A.C., Jemai, A.: Multiprocessor platform-based design for multimedia. IET Comput. Digit. Tech. 3(1), 52–61 (2009)

    Article  Google Scholar 

  36. EPLF Person Dataset. [Online] http://cvlab.epfl.ch/data/pom. Accessed 12 Jan 2017

  37. Xilinx.: Zynq-7000 extensible processing platform technical reference manual. Technical Report (2012)

  38. Maggiani, L., Bourrasset, C., Quinton, J.C., Berry, F., Sérot, J.: Bio-inspired heterogeneous architecture for real-time pedestrian detection applications. J. Real Time Image Process. 14, 535–548 (2016)

    Article  Google Scholar 

  39. Truong, M.T.N., Kim, S.: Parallel implementation of color-based particle filter for object tracking in embedded systems. Hum. Centric Comput. Inf. Sci. 7(1), 2 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratory of Computer and Embedded Systems, National School of Engineering of Sfax, Sfax University, Sfax, Tunisia

    Nesrine Abid, Kais Loukil, Tarek Ouni, Walid Ayedi & Mohamed Abid

  2. Digital Research Center of Sfax, Sfax University, Sfax, Tunisia

    Kais Loukil & Mohamed Abid

  3. Electrical and Computer Engineering Department, Faculty of Engineering, Sultan Qaboos University, Al-Khod, Muscat, Sultanate of Oman

    Ahmed Chiheb Ammari

  4. MMA Laboratory, National Institute of the Applied Sciences and Technology, Carthage University, Tunis, Tunisia

    Ahmed Chiheb Ammari

Authors
  1. Nesrine Abid
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Kais Loukil
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Tarek Ouni
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Walid Ayedi
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Ahmed Chiheb Ammari
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Mohamed Abid
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Nesrine Abid.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abid, N., Loukil, K., Ouni, T. et al. An improvement of multi-scale covariance descriptor for embedded system. J Real-Time Image Proc 17, 419–435 (2020). https://doi.org/10.1007/s11554-018-0759-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-018-0759-y

Keywords