Abstract
Object search – the problem of finding a target object in a cluttered scene – is essential to solve for many robotics applications in warehouse and household environments. However, cluttered environments entail that objects often occlude one another, making it difficult to segment objects and infer their shapes and properties. Instead of relying on the availability of CAD or other explicit models of scene objects, we augment a manipulation planner for cluttered environments with a state-of-the-art deep neural network for shape completion as well as a volumetric memory system, allowing the robot to reason about what may be contained in occluded areas. We test the system in a variety of tabletop manipulation scenes composed of household items, highlighting its applicability to realistic domains. Our results suggest that incorporating both components into a manipulation planning framework significantly reduces the number of actions needed to find a hidden object in dense clutter.
A. Price and L. Jin—Equal contribution.
This research was funded in part by Toyota Research Institute (TRI). This article solely reflects the opinions of its authors and not TRI or any other Toyota entity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Jonschkowski, R., Eppner, C., Höfer, S., Martín-Martín, R., Brock, O.: Probabilistic multiclass segmentation for the Amazon Picking Challenge. In: IROS, pp. 1–7. IEEE (2016)
Schwarz, M., et al.: NimbRo picking: versatile part handling for warehouse automation. In: ICRA, pp. 3032–3039. IEEE (2017)
Milan, A., et al.: Semantic segmentation from limited training data. In: ICRA, pp. 1908–1915. IEEE (2018)
Varley, J., DeChant, C., Richardson, A., Ruales, J., Allen, P.: Shape completion enabled robotic grasping. In: IROS, pp. 2442–2447. IEEE (2017)
Yang, B., Wen, H., Wang, S., Clark, R., Markham, A., Trigoni, N.: 3D object reconstruction from a single depth view with adversarial learning. In: ICCV Workshop (2017)
Stilman, M., Schamburek, J.U., Kuffner, J., Asfour, T.: Manipulation planning among movable obstacles. In: ICRA, pp. 3327–3332 (2007)
Dogar, M.R., Srinivasa, S.S.: A planning framework for non-prehensile manipulation under clutter and uncertainty. Auton. Robots 33(3), 217–236 (2012). https://doi.org/10.1007/s10514-012-9306-z
Wong, L.L., Kaelbling, L.P., Lozano-Perez, T.: Manipulation-based active search for occluded objects. In: ICRA, pp. 2814–2819 (2013)
Dogar, M.R., Koval, M.C., Tallavajhula, A., Srinivasa, S.S.: Object search by manipulation. Auton. Robots 36(1–2), 153–167 (2014). https://doi.org/10.1007/s10514-013-9372-x
Li, J.K., Hsu, D., Lee, W.S.: Act to see and see to act: POMDP planning for objects search in clutter. In: IROS, pp. 5701–5707, November 2016
Zeng, A., et al.: Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching. In: ICRA (2018)
Liu, M.Y., Tuzel, O., Veeraraghavan, A., Taguchi, Y., Marks, T.K., Chellappa, R.: Fast object localization and pose estimation in heavy clutter for robotic bin picking. IJRR 31(8), 951–973 (2012)
Choi, C., Christensen, H.I.: 3D pose estimation of daily objects using an RGB-D camera. In: IROS (2012)
Tulsiani, S., Gupta, S., Fouhey, D., Efros, A.A., Malik, J.: Factoring shape, pose, and layout from the 2D image of a 3D scene. In: CVPR (2018)
Xiang, Y., Schmidt, T., Narayanan, V., Fox, D.: PoseCNN: a convolutional neural network for 6D object pose estimation in cluttered scenes. In: Robotics: Science and Systems (RSS) (2018)
Wu, Z., et al.: 3D ShapeNets: a deep representation for volumetric shapes. In: CVPR, pp. 1912–1920 (2015)
Firman, M., Mac Aodha, O., Julier, S., Brostow, G.J.: Structured prediction of unobserved voxels from a single depth image. In: CVPR, pp. 5431–5440 (2016)
Song, S., Yu, F., Zeng, A., Chang, A.X., Savva, M., Funkhouser, T.: Semantic scene completion from a single depth image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1746–1754 (2017)
Fan, H., Su, H., Guibas, L.J.: A point set generation network for 3D object reconstruction from a single image. In: CVPR, vol. 2, p. 6 (2017)
Yang, B., Rosa, S., Markham, A., Trigoni, N., Wen, H.: Dense 3D object reconstruction from a single depth view. TPAMI 41(12), 2820–2834 (2018)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: ICCV, pp. 2980–2988. IEEE (2017)
Lin, G., Milan, A., Shen, C., Reid, I.: RefineNet: multi-path refinement networks for high-resolution semantic segmentation. In: CVPR, July 2017
Pham, T., Do, T.-T., Sünderhauf, N., Reid, I.: SceneCut: joint geometric and object segmentation for indoor scenes. In: ICRA (2018)
Arbelaez, P.: Boundary extraction in natural images using ultrametric contour maps. In: CVPR Workshop, pp. 182–182. IEEE (2006)
Maninis, K., Pont-Tuset, J., Arbeláez, P., Van Gool, L.: Convolutional oriented boundaries: from image segmentation to high-level tasks. TPAMI 40(4), 819–833 (2017)
Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33715-4_54
Hornung, A., Wurm, K.M., Bennewitz, M., Stachniss, C., Burgard, W.: OctoMap: an efficient probabilistic 3D mapping framework based on octrees. Auton. Robots 34(3), 189–206 (2013). https://doi.org/10.1007/s10514-012-9321-0
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
Calli, B., Singh, A., Walsman, A., Srinivasa, S., Abbeel, P., Dollar, A.M.: The YCB object and model set: towards common benchmarks for manipulation research. In: ICAR, pp. 510–517. IEEE (2015)
Kappler, D., Bohg, J., Schaal, S.: Leveraging big data for grasp planning. In: ICRA, pp. 4304–4311. IEEE (2015)
Chang, A.X., et al.: ShapeNet: an information-rich 3D model repository (2015). arXiv:1512.03012
Min, P.: Binvox (2004). http://www.patrickmin.com/binvox. Accessed 01 May 2018
Gupta, M., Müller, J., Sukhatme, G.S.: Using manipulation primitives for object sorting in cluttered environments. IEEE Trans. Autom. Sci. Eng. 12(2), 608–614 (2015)
Boularias, A., Bagnell, J.A., Stentz, A.: Learning to manipulate unknown objects in clutter by reinforcement. In: AAAI, January 2015
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Price, A., Jin, L., Berenson, D. (2022). Inferring Occluded Geometry Improves Performance When Retrieving an Object from Dense Clutter. In: Asfour, T., Yoshida, E., Park, J., Christensen, H., Khatib, O. (eds) Robotics Research. ISRR 2019. Springer Proceedings in Advanced Robotics, vol 20. Springer, Cham. https://doi.org/10.1007/978-3-030-95459-8_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-95459-8_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-95458-1
Online ISBN: 978-3-030-95459-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)