Abstract
Environmental perception obtained via object detectors have no predictable safety layer encoded into their model schema, which creates the question of trustworthiness about the system’s prediction. As can be seen from recent adversarial attacks, most of the current object detection networks are vulnerable to input tampering, which in the real world could compromise the safety of autonomous vehicles. The problem would be amplified even more when uncertainty errors could not propagate into the submodules, if these are not a part of the end-to-end system design. To address these concerns, a parallel module which verifies the predictions of the object proposals coming out of Deep Neural Networks are required. This work aims to verify 3D object proposals from MonoRUn model by proposing a plausibility framework that leverages cross sensor streams to reduce false positives. The verification metric being proposed uses prior knowledge in the form of four different energy functions, each utilizing a certain prior to output an energy value leading to a plausibility justification for the hypothesis under consideration. We also employ a novel two-step schema to improve the optimization of the composite energy function representing the energy model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aeberhard, M., Paul, S., Kaempchen, N., Bertram, T.: Object existence probability fusion using dempster-shafer theory in a high-level sensor data fusion architecture, pp. 770–775. IEEE (2011). https://doi.org/10.1109/IVS.2011.5940430
Blanco, J.L.: A tutorial on SE(3) transformation parameterizations and on-manifold optimization. Technical report, 012010, University of Malaga (2010). http://ingmec.ual.es/jlblanco/papers/jlblanco2010geometry3D_techrep.pdf
Caesar, H., et al.: nuScenes: a multimodal dataset for autonomous driving. In: CVPR (2020)
Chang, A.X., et al.: Shapenet: an information-rich 3D model repository. CoRR abs/1512.03012 (2015). http://arxiv.org/abs/1512.03012
Chen, H., Huang, Y., Tian, W., Gao, Z., Xiong, L.: Monorun: monocular 3D object detection by reconstruction and uncertainty propagation. CoRR abs/2103.12605 (2021). https://arxiv.org/abs/2103.12605
Cofer, D., Amundson, I., Sattigeri, R., Passi, A.: Run-Time Assurance for Learning-Enabled Systems Run-Time Assurance for Learning-Enabled Systems (2020). https://doi.org/10.1007/978-3-030-55754-6
Engelmann, F., Stückler, J., Leibe, B.: Joint object pose estimation and shape reconstruction in urban street scenes using 3D shape priors (2016). https://doi.org/10.1007/978-3-319-45886-1_18. https://github.com/VisualComputingInstitute/ShapePriors_GCPR16
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The kitti vision benchmark suite. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2012)
Geissler, F., Unnervik, A., Paulitsch, M.: A plausibility-based fault detection method for high-level fusion perception systems. https://doi.org/10.1109/OJITS.2020.3027146
Gustafsson, F.K., Danelljan, M., Bhat, G., Schön, T.B.: Energy-based models for deep probabilistic regression. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12365, pp. 325–343. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58565-5_20
Gustafsson, F.K., Danelljan, M., Schön, T.B.: Accurate 3D object detection using energy-based models. https://github.com/fregu856/ebms_3dod
He, K., Gkioxari, G., Dollár, P., Girshick, R.B.: Mask R-CNN. In: IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, 22–29 October 2017, pp. 2980–2988. IEEE Computer Society (2017). https://doi.org/10.1109/ICCV.2017.322
Hinton, G.E.: Products of experts, vol. 1, pp. 1–6 (1999)
Khesbak, M.S.: Depth camera and laser sensors plausibility evaluation for small size obstacle detection. In: 18th International Multi-Conference on Systems, Signals & Devices, SSD 2021, Monastir, Tunisia, 22–25 March 2021, pp. 625–631. IEEE (2021). https://doi.org/10.1109/SSD52085.2021.9429373
LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M., Huang, F.: A tutorial on energy-based learning. In: Bakir, G., Hofman, T., Schölkopf, B., Smola, A., Taskar, B. (eds.) Predicting Structured Data. MIT Press, Cambridge (2006)
Maag, K.: False negative reduction in video instance segmentation using uncertainty estimates. CoRR abs/2106.14474 (2021). https://arxiv.org/abs/2106.14474
Nocedal, J., Wright, S.: Numerical Optimization (2006). https://doi.org/10.1007/978-0-387-40065-5
Osadchy, M., Cun, Y.L., Miller, M.L.: Synergistic face detection and pose estimation with energy-based models (2006). https://doi.org/10.1007/11957959_10
Prisacariu, V.A., Segal, A.V., Reid, I.: Simultaneous monocular 2D segmentation, 3D pose recovery and 3D reconstruction. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012. LNCS, vol. 7724, pp. 593–606. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-37331-2_45
Rao, Q., Krüger, L., Dietmayer, K.: 3D shape reconstruction in traffic scenarios using monocular camera and lidar. In: Chen, C.-S., Lu, J., Ma, K.-K. (eds.) ACCV 2016. LNCS, vol. 10117, pp. 3–18. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54427-4_1
Rottmann, M., Maag, K., Chan, R., Hüger, F., Schlicht, P., Gottschalk, H.: Detection of false positive and false negative samples in semantic segmentation. CoRR abs/1912.03673 (2019). http://arxiv.org/abs/1912.03673
Shafer, G.: A Mathematical Theory of Evidence. Princeton University Press, Princeton (1976)
Von Rueden, L., et al.: Informed machine learning - a taxonomy and survey of integrating knowledge into learning systems. arXiv, pp. 1–20 (2019)
Wang, R., Yang, N., Stuckler, J., Cremers, D.: Directshape: direct photometric alignment of shape priors for visual vehicle pose and shape estimation (2020). https://doi.org/10.1109/icra40945.2020.9197095
Wörmann, J., et al.: Knowledge augmented machine learning with applications in autonomous driving: a survey (2022). https://doi.org/10.48550/ARXIV.2205.04712. https://arxiv.org/abs/2205.04712
Acknowledgements
The research leading to these results is funded by the German Federal Ministry for Economic Affairs and Climate Action within the project “KI Wissen”. The authors would like to thank the consortium for the successful cooperation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Vivekanandan, A., Maier, N., Zöllner, J.M. (2023). Plausibility Verification for 3D Object Detectors Using Energy-Based Optimization. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13801. Springer, Cham. https://doi.org/10.1007/978-3-031-25056-9_38
Download citation
DOI: https://doi.org/10.1007/978-3-031-25056-9_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25055-2
Online ISBN: 978-3-031-25056-9
eBook Packages: Computer ScienceComputer Science (R0)