Abstract
3D depth computation from stereo data has been one of the most researched topics in computer vision. While state-of-art approaches have flourished over time, reconstruction of transparent materials is still considered an open problem. Based on 3D light field data we propose a method to obtain smooth and consistent double-layer estimates of scenes with transparent materials. Our novel approach robustly combines estimates from models with different layer hypotheses in a cost volume with subsequent minimization of a joint second order \(\mathrm {TGV}\) energy on two depth layers. Additionally we showcase the results of our approach on objects from common inspection use-cases in an industrial setting and compare our work to related methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aach, T., Mota, C., Stuke, I., Muhlich, M., Barth, E.: Analysis of superimposed oriented patterns. IEEE Trans. Image Process. 15(12), 3690–3700 (2006)
Adelson, E.H., Bergen, J.R.: The plenoptic function and the elements of early vision. In: Landy, M.S., Movshon, J.A. (eds.) Computational Models of Visual Processing, pp. 3–20. The MIT Press, Cambridge (1991)
Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3(3), 492–526 (2010). https://doi.org/10.1137/090769521
Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011). https://doi.org/10.1007/s10851-010-0251-1
Chen, G., Han, K., Wong, K.Y.K.: TOM-Net: learning transparent object matting from a single image. In: CVPR (2018)
Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F.: The lumigraph. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1996, pp. 43–54. Association for Computing Machinery, New York (1996). https://doi.org/10.1145/237170.237200
Ji, Y., Xia, Q., Zhang, Z.: Fusing depth and Silhouette for scanning transparent object with RGB-D sensor. Int. J. Opt. 2017, 1–11 (2017). https://doi.org/10.1155/2017/9796127
Johannsen, O., Sulc, A., Goldlücke, B.: Occlusion-aware depth estimation using sparse light field coding. In: German Conference on Pattern Recognition (Proceedings of the GCPR) (2016)
Johannsen, O., Sulc, A., Goldlücke, B.: What sparse light field coding reveals about scene structure. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Kim, J., Reshetouski, I., Ghosh, A.: Acquiring axially-symmetric transparent objects using single-view transmission imaging. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1484–1492 (2017)
Koppal, S.J.: Lambertian reflectance. In: Ikeuchi, K. (ed.) Computer Vision: A Reference Guide, pp. 441–443. Springer, Boston (2014). https://doi.org/10.1007/978-0-387-31439-6_534
Kuschk, G., Cremers, D.: Fast and accurate large-scale stereo reconstruction using variational methods. In: Proceedings of the 2013 IEEE International Conference on Computer Vision Workshops, pp. 700–707 (2013)
Li, Y., Osher, S.: A new median formula with applications to PDE based denoising. Commun. Math. Sci. 7(3), 741–753 (2009). https://projecteuclid.org:443/euclid.cms/1256562821
Li, Z., Yeh, Y.Y., Chandraker, M.: Through the looking glass: neural 3D reconstruction of transparent shapes. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Miyazaki, D., Ikeuchi, K.: Inverse polarization raytracing: estimating surface shapes of transparent objects. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 2, pp. 910–917 (2005)
Qian, Y., Gong, M., Yang, Y.: 3d reconstruction of transparent objects with position-normal consistency. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016), pp. 4369–4377 (2016)
Ranftl, R., Gehrig, S., Pock, T., Bischof, H.: Pushing the limits of stereo using variational stereo estimation. In: 2012 IEEE Intelligent Vehicles Symposium, pp. 401–407 (2012)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1), 259–268 (1992). https://doi.org/10.1016/0167-2789(92)90242-F. http://www.sciencedirect.com/science/article/pii/016727899290242F
Tanaka, K., Mukaigawa, Y., Kubo, H., Matsushita, Y., Yagi, Y.: Recovering transparent shape from time-of-flight distortion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016), pp. 4387–4395 (2016)
Trifonov, B., Bradley, D., Heidrich, W.: Tomographic reconstruction of transparent objects. In: ACM SIGGRAPH 2006 Sketches, SIGGRAPH 2006, p. 55-es. Association for Computing Machinery, New York (2006). https://doi.org/10.1145/1179849.1179918
Štolc, S., Huber-Mörk, R., Holländer, B., Soukup, D.: Depth and all-in-focus images obtained by multi-line-scan light-field approach. In: Electronic Imaging (2014)
Wanner, S., Goldlücke, B.: Globally consistent depth labeling of 4D lightfields. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)
Wanner, S., Goldlücke, B.: Reconstructing reflective and transparent surfaces from epipolar plane images. In: German Conference on Pattern Recognition (Proceedings of the GCPR, Oral Presentation) (2013)
Wanner, S., Meister, S., Goldlücke, B.: Datasets and benchmarks for densely sampled 4D light fields. In: Vision, Modelling and Visualization (VMV) (2013)
Wu, B., Zhou, Y., Qian, Y., Cong, M., Huang, H.: Full 3D reconstruction of transparent objects. ACM Trans. Graph. 37, 1–11 (2018). https://doi.org/10.1145/3197517.3201286
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Kopf, C., Pock, T., Blaschitz, B., Štolc, S. (2021). Inline Double Layer Depth Estimation with Transparent Materials. In: Akata, Z., Geiger, A., Sattler, T. (eds) Pattern Recognition. DAGM GCPR 2020. Lecture Notes in Computer Science(), vol 12544. Springer, Cham. https://doi.org/10.1007/978-3-030-71278-5_30
Download citation
DOI: https://doi.org/10.1007/978-3-030-71278-5_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-71277-8
Online ISBN: 978-3-030-71278-5
eBook Packages: Computer ScienceComputer Science (R0)