The study of stereo matching optimization based on multi-baseline trinocular model | Multimedia Tools and Applications Skip to main content
Springer Nature Link
Log in

The study of stereo matching optimization based on multi-baseline trinocular model

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The huge computational complexity, occlusion and low texture region problems make stereo matching a big challenge. In this work, we use multi-baseline trinocular camera model to study how to accelerate the stereo matching algorithms and improve the accuracy of disparity estimation. A special scheme named the trinocular dynamic disparity range (T-DDR) was designed to accelerate the stereo matching algorithms. In this scheme, we optimize matching cost calculation, cost aggregation and disparity computation steps by narrowing disparity searching range. Meanwhile, we designed another novel scheme called the trinocular disparity confidence measure (T-DCM) to improve the accuracy of the disparity map. Based on those, we proposed the semi-global matching with T-DDR (T-DDR-SGM) and T-DCM (T-DCM-SGM) algorithms for trinocular stereo matching. According to the evaluation results, the T-DDR-SGM could not only significantly reduce the computational complexity but also slightly improving the accuracy, while the T-DCM-SGM could excellently handle the occlusion and low texture region problems. Both of them achieved a better result. Moreover, the optimization schemes we designed can be extended to the other stereo matching algorithms which possesses pixel-wise matching cost calculation and aggregation steps not only the SGM. We proved that the proposed optimization methods for the trinocular stereo matching are effective and the trinocular stereo matching is useful for either improving accuracy or reducing computational complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Akin A, Capoccia R, Narinx J, Baz I, Schmid A, Leblebici Y (2015) Trinocular adaptive window size disparity estimation algorithm and its real-time hardware. In VLSI Design, Automation and Test (VLSI-DAT), pp 1–4: IEEE

  2. Dosovitskiy A, Ros G, Codevilla F, Lopez A, Koltun V (2017) CARLA: An open urban driving simulator. arXiv preprint arXiv: 03938

  3. El Jaafari I, El Ansari M, Koutti L, Mazoul A, Ellahyani A (2016) Fast spatio-temporal stereo matching for advanced driver assistance systems. Neurocomputing 194:24–33

    Article  Google Scholar 

  4. Gehrig SK, Eberli F, Meyer T (2009) A real-time low-power stereo vision engine using semi-global matching. In International Conference on Computer Vision Systems, pp 134–143: Springer

  5. Geiger A, Lenz P, Stiller C, Urtasun R (2013) "Vision meets robotics: the KITTI dataset," (in English). Int J Robot Res 32(11):1231–1237

    Article  Google Scholar 

  6. Hirschmuller H (2005) Accurate and efficient stereo processing by semi-global matching and mutual information. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), vol 2 pp 807–814: IEEE

  7. Hirschmuller H (2007) Stereo processing by semiglobal matching and mutual information. IEEE Trans Pattern Anal 30(2):328–341

    Article  Google Scholar 

  8. Honegger D, Sattler T, Pollefeys M (2017) Embedded real-time multi-baseline stereo. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pp 5245–5250: IEEE

  9. Hu X, P. J. I. T. o. P. A. Mordohai, and M. Intelligence (2012) A quantitative evaluation of confidence measures for stereo vision. vol 34, no 11, pp 2121–2133

  10. Kendall A et al (2017) End-to-end learning of geometry and context for deep stereo regression. In Proceedings of the IEEE International Conference on Computer Vision, pp 66–75

  11. Lee KJ, Bong K, Kim C, Yoo H-J (2016) An intelligent ADAS processor with real-time semi-global matching and intention prediction for 720p stereo vision. In 2016 IEEE Hot Chips 28 Symposium (HCS), pp 1–1: IEEE

  12. Li M, Shi L, Chen X, Du S, Li Y (2019) Using temporal correlation to optimize stereo matching in video sequences. IEICE Trans Inf 102(6):1183–1196

    Article  Google Scholar 

  13. Luo W, Schwing AG, Urtasun R (2016) Efficient deep learning for stereo matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5695–5703

  14. Menze M, Geiger A (2015) Object scene flow for autonomous vehicles. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3061–3070

  15. Michael M, Salmen J, Stallkamp J, Schlipsing M (2013) Real-time stereo vision: Optimizing semi-global matching. In 2013 IEEE Intelligent Vehicles Symposium (IV), pp 1197–1202: IEEE

  16. Narinx J, Demirci T, Akin A, Leblebici Y (2017) A single-chip 2048× 1080 resolution 32fps 380mW trinocular disparity estimation processor in 28nm CMOS technology. In 2017 Symposium on VLSI Circuits, pp C228-C229: IEEE

  17. Park H, Lee KM (2016) Look wider to match image patches with convolutional neural networks. IEEE Signal Process Lett 24(12):1788–1792

    Article  Google Scholar 

  18. Pritchett P, Zisserman A (1998) Matching and reconstruction from widely separated views. In European workshop on 3D structure from multiple images of large-scale environments

  19. Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vis 47(1–3):7–42

    Article  Google Scholar 

  20. Strecha C, Pylvänäinen T, Fua P (2010) Dynamic and scalable large scale image reconstruction. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 406–413: IEEE

  21. Yang W, Zhang G, Bao H, Kim J, Lee HY (2012) Consistent depth maps recovery from a trinocular video sequence. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp 1466–1473: IEEE

  22. Zagoruyko S, Komodakis N (2015) Learning to compare image patches via convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4353–4361

  23. Zbontar J, LeCun Y (2015) Computing the stereo matching cost with a convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1592–1599

  24. Zbontar J, LeCun Y (2016) Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches. J Mach Learn Res 17(1–32):2

    MATH  Google Scholar 

  25. Zhang K, Lu J, Lafruit G (2009) Cross-based local stereo matching using orthogonal integral images. IEEE Trans Circ Syst Video Technol 19(7):1073–1079

    Article  Google Scholar 

  26. Zhou J, Wang L, Gu X, Xu K, Zhang Y (2015) A depth map estimation approach for trinocular stereo. In 2015 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, pp 1–6: IEEE

Download references

Author information

Author notes

  1. Jie Wang and Chenglei Peng contributed equally to this work.

Authors and Affiliations

  1. School of Electronic Science and Engineering, Nanjing University, Nanjing, 210023, Jiangsu, China

    Jie Wang, Chenglei Peng, Ming Li, Yang Li & Sidan Du

  2. Nanjing Institute of Advanced Artificial Intelligence, Nanjing, 210023, Jiangsu, China

    Jie Wang & Chenglei Peng

Authors
  1. Jie Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chenglei Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sidan Du
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yang Li or Sidan Du.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, J., Peng, C., Li, M. et al. The study of stereo matching optimization based on multi-baseline trinocular model. Multimed Tools Appl 81, 12961–12972 (2022). https://doi.org/10.1007/s11042-022-12579-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12579-8

Keywords

Search

Navigation