Abstract
While any grasp must satisfy the grasping stability criteria, good grasps depend on the specific manipulation scenario: the object, its properties and functionalities, as well as the task and grasp constraints. We propose a probabilistic logic approach for robot grasping, which improves grasping capabilities by leveraging semantic object parts. It provides the robot with semantic reasoning skills about the most likely object part to be grasped, given the task constraints and object properties, while also dealing with the uncertainty of visual perception and grasp planning. The probabilistic logic framework is task-dependent. It semantically reasons about pre-grasp configurations with respect to the intended task and employs object-task affordances and object/task ontologies to encode rules that generalize over similar object parts and object/task categories. The use of probabilistic logic for task-dependent grasping contrasts with current approaches that usually learn direct mappings from visual perceptions to task-dependent grasping points. The logic-based module receives data from a low-level module that extracts semantic objects parts, and sends information to the low-level grasp planner. These three modules define our probabilistic logic framework, which is able to perform robotic grasping in realistic kitchen-related scenarios.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
The work can be extended to consider several objects simultaneously.
If the sum is less than 1, there is a non-zero probability that nothing happens.
The notation p&p is abbreviation for pick and place.
Available at: www.theaffordances.net.
Available at: https://people.cs.kuleuven.be/~laura.antanas/.
The synthetic dataset and part of ORCA datasets are available for download at http://www.first-mm.eu/data.html.
References
Abdo, N., Kretzschmar, H., & Stachniss, C. (2012). From low-level trajectory demonstrations to symbolic actions for planning. In ICAPS workshop on combining task and motion planning for real-world applications (pp. 1–8).
Aleotti, J., & Caselli, S. (2011). Part-based robot grasp planning from human demonstration. In ICRA (pp. 4554–4560).
Antanas, L., Dries, A., Moreno, P., & De Raedt, L. (2018). Relational affordance learning for task-dependent robot grasping. In N. Lachiche & C. Vrain (Eds.), Inductive logic programming (pp. 1–15). Cham: Springer.
ASUS, Asus xtion pro. http://www.asus.com/Multimedia/Motion_Sensor/Xtion_PRO/. Accessed 15 July 2014.
Baltzakis, H. Orca simulator. http://www.ics.forth.gr/cvrl/_software/orca_setup.exe. Accessed 15 July 2014.
Barck-Holst, C., Ralph, M., Holmar, F., & Kragic, D. (2009). Learning grasping affordance using probabilistic and ontological approaches. In International conference on advanced robotics (pp. 1–6).
Besl, P. J., & McKay, N. D. (1992). A method for registration of 3-d shapes. TPAMI, 14(2), 239–256.
Bohg, J., Johnson-Roberson, M., León, B., Felip, J., Gratal, X., Bergström, N., et al. (2011). Mind the gap—robotic grasping under incomplete observation. In ICRA (pp. 686–693).
Bohg, J., & Kragic, D. (2010). Learning grasping points with shape context. RAS, 58(4), 362–377.
Bohg, J., Welke, K., Leon, B., Do, M., Song, D., Wohlkinger, W., et al. (2012). Task-based grasp adaptation on a humanoid robot. In IFAC symposium on robot control (pp. 779–786).
Cocora, A., Kersting, K., Plagemann, C., Burgard, W., & Raedt, L. D. (2006). Learning relational navigation policies. In IEEE/RSJ ICIRS (pp. 2792–2797).
Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20, 273–297.
Dang, H., & Allen, P. K. (2012). Semantic grasping: Planning robotic grasps functionally suitable for an object manipulation task. In IEEE/RSJ ICIRS (pp. 1311–1317).
Detry, R., Ek, C. H., Madry, M., & Kragic, D. (2012a). Compressing grasping experience into a dictionary of prototypical grasp-predicting parts. In International workshop on human-friendly robotics (pp. 1–1).
Detry, R., Ek, C. H., Madry, M., & Kragic, D. (2013). Learning a dictionary of prototypical grasp-predicting parts from grasping experience. In ICRA (pp. 601–608).
Detry, R., Ek, C. H., Madry, M., Piater, J. H., & Kragic, D. (2012b). Generalizing grasps across partly similar objects. In ICRA (pp. 3791–3797).
Fierens, D., Van den Broeck, G., Thon, I., Gutmann, B., & De Raedt, L. (2011). Inference in probabilistic logic programs using weighted CNF’s. In F. Gagliardi Cozman, A. Pfeffer (Eds.) UAI (pp. 211–220).
Figueiredo, R., Moreno, P., & Bernardino, A. (2017). Automatic object shape completion from 3d point clouds for object manipulation. In International joint conference on computer vision, imaging and computer graphics theory and applications (Vol. 4: VISAPP, pp. 565–570).
Fischinger, D., Vincze, M., & Jiang, Y. (2013). Learning grasps for unknown objects in cluttered scenes. In ICRA (pp. 609–616).
Flanagan, J. R., Bowman, M. C., & Johansson, R. S. (2006). Control strategies in object manipulation tasks. Current Opinion in Neurobiology, 16(6), 650–659.
Gibson, J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin.
Hanheide, M., Gretton, C., Dearden, R., Hawes, N., Wyatt, J. L., Pronobis, A., et al. (2011). Exploiting probabilistic knowledge under uncertain sensing for efficient robot behaviour. In IJCAI (pp. 2442–2449).
Hart, S., Grupen, R. A., & Jensen, D. (2005). A relational representation for procedural task knowledge. In AAAI (pp. 1280–1285).
Jerez, J., & Suero, A. Newton game dynamics. Open-source physics engine. http://www.newtondynamics.com. Accessed 15 July 2014.
Jiang, Y., Moseson, S., & Saxena, A. (2011). Efficient grasping from rgbd images: Learning using a new rectangle representation. In ICRA (pp. 3304–3311).
Kroemer, O., Amor, H. B., Ewerton, M., & Peters, J. (2012). Point cloud completion using extrusions. In Humanoids (pp. 680–685).
Kulick, J., Toussaint, M., Lang, T., & Lopes, M. (2013). Active learning for teaching a robot grounded relational symbols. In IJCAI (pp. 1451–1457). AAAI Press.
K. Robotics, Kuka lightweight robot (LWR). http://www.kuka-robotics.com/en/products/addons/lwr/. Accessed 15 July 2014.
Lang, T., & Toussaint, M. (2010). Planning with noisy probabilistic relational rules. JAIR, 39, 1–49.
Lenz, I., Lee, H., & Saxena, A. (2015). Deep learning for detecting robotic grasps. CoRR arXiv:1301.3592.
Limketkai, B., Liao, L., & Fox, D. (2005). Relational object maps for mobile robots. In IJCAI (pp. 1471–1476).
Madry, M., Song, D., Ek, C. H., & Kragic, D. (2012a). Robot bring me something to drink from: Object representation for transferring task specific grasps. In ICRA Workshop on semantic perception, mapping and exploration (pp. 1–6).
Madry, M., Song, D., & Kragic, D. (2012b). From object categories to grasp transfer using probabilistic reasoning. In ICRA (pp. 1716–1723).
Marton, Z. C., Rusu, R. B., Jain, D., Klank, U., & Beetz, M. (2009). Probabilistic categorization of kitchen objects in table settings with a composite sensor. In IEEE/RSJ ICIRS (pp. 4777–4784).
Meert, W., Struyf, J., & Blockeel, H. (2008). Learning ground CP-logic theories by leveraging bayesian network learning techniques. Fundamenta Informaticae, 89(1), 131–160.
Mitra, N. J., Guibas, L., & Pauly, M. (2006). Partial and approximate symmetry detection for 3D geometry. ACM Transactions on Graphics, 25(3), 560–568.
Moldovan, B., Antanas, L., & Hoffmann, M. (2013a). Opening doors: An initial SRL approach. Lecture Notes in Computer Science, 7842, 178–192.
Moldovan, B., Moreno, P., Nitti, D., Santos-Victor, J., & De Raedt, L. (2018). Relational affordances for multiple-object manipulation. Autonomous Robots, 42(1), 19–44.
Moldovan, B., Moreno, P., & van Otterlo, M. (2013b) On the use of probabilistic relational affordance models for sequential manipulation tasks in robotics. In ICRA (pp. 1290–1295).
Moldovan, B., Moreno, P., van Otterlo, M., Santos-Victor, J., & De Raedt, L. (2012). Learning relational affordance models for robots in multi-object manipulation tasks. In ICRA (pp. 4373–4378).
Montesano, L., & Lopes, M. (2012). Active learning of visual descriptors for grasping using non-parametric smoothed beta distributions. RAS, 60(3), 452–462.
Muja, M., & Ciocarlie, M. Table top segmentation package. http://www.ros.org/wiki/tabletop_object_detector. Accessed 15 July 2014.
Neumann, M., Garnett, R., Bauckhage, C., & Kersting, K. (2016). Propagation kernels: Efficient graph kernels from propagated information. Machine Learning, 102(2), 209–245.
Neumann, M., Moreno, P., Antanas, L., Garnett, R., & Kersting, K. (2013). Graph kernels for object category prediction in task—dependent robot grasping. In MLG (pp. 1–6).
Neumann, M., Patricia, N., Garnett, R., & Kersting, K. (2012). Efficient graph kernels by randomization. In ECML/PKDD (pp. 378–393).
Nyga, D., Balint-Benczedi, F., & Beetz, M. (2014). PR2 Looking at things: Ensemble learning for unstructured information processing with Markov logic networks. In IEEE international conference on robotics and automation (ICRA), Hong Kong, China.
Palinko, O., Sciutti, A., Rea, F., & Sandini, G. (2014). Weight-aware robot motion planning for lift-to-pass action. International conference on human-agent interaction (pp. 193–196). New York, NY, USA: ACM.
Platt, J. C. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In A. Smola, P. Bartlett, B. Schölkopf, & D. Schuurmans (Eds.), Advances in large margin classifiers (pp. 61–74). Cambridge: MIT Press.
Rusinkiewicz, S., & Levoy, M. (2001). Efficient variants of the ICP algorithm. In 3DIM (pp. 145–152).
Sánchez, G., & Latombe, J.-C. (2003). A single-query bi-directional probabilistic roadmap planner with lazy collision checking. Robotics research, Volume 6 of Springer tracts in advanced robotics (pp. 403–417). Berlin: Springer.
Saxena, A., Wong, L. L. S., & Ng, A. Y. (2008). Learning grasp strategies with partial shape information. In AAAI (pp. 1491–1494). AAAI Press.
Song, D., Huebner, K., Kyrki, V., & Kragic, D. (2010). Learning task constraints for robot grasping using graphical models. In IEEE/RSJ ICIRS (pp. 1579–1585).
Şucan, I. A., Moll, M., & Kavraki, L. E. (2012). The open motion planning library. Robotics & Automation Magazine, 19(4), 72–82.
Sweeney, J., & Grupen, R. A. (2007). A model of shared grasp affordances from demonstration. In Humanoids (pp. 27–35).
Tenorth, M., & Beetz, M. (2009). Knowrob knowledge processing for autonomous personal robots. In IEEE/RSJ ICIRS (pp. 4261–4266).
Tenorth, M., Profanter, S., Balint-Benczedi, F., & Beetz, M. (2013). Decomposing cad models of objects of daily use and reasoning about their functional parts. In 2013 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 5943–5949). IEEE.
Thrun, S., & Wegbreit, B. (2005). Shape from symmetry. In ICCV (pp. 1824–1831).
Toussaint, M., Plath, N., Lang, T., & Jetchev, N. (2010). Integrated motor control, planning, grasping and high-level reasoning in a blocks world using probabilistic inference. In ICRA (pp. 385–391).
Vennekens, J., Denecker, M., & Bruynooghe, M. (2009). CP-logic: A language of causal probabilistic events and its relation to logic programming. Theory and Practice of Logic Programming, 9(3), 245–308.
Winkler, J., Bartels, G., Msenlechner, L., & Beetz, M. (2012). Knowledge enabled high-level task abstraction and execution. Conference for Advances in Cognitive Systems, 2(1), 131–148.
W. Robotics. Universal gripper WSG 50. http://www.weiss-robotics.de/gripper-systems/gripper-modules/universal-gripper-wsg-50.html. Accessed 15 July 2014.
Yousef, H., Boukallel, M., & Althoefer, K. (2011). Tactile sensing for dexterous in-hand manipulation in roboticsa review. Sensors and Actuators A: Physical, 167(2), 171–187 (Solid-State Sensors. Actuators and Microsystems Workshop).
Zhu, X. (2013). Persistent homology: An introduction and a new text representation for natural language processing. In IJCAI (pp. 1953–1959). AAAI Press.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Antanas, L., Moreno, P., Neumann, M. et al. Semantic and geometric reasoning for robotic grasping: a probabilistic logic approach. Auton Robot 43, 1393–1418 (2019). https://doi.org/10.1007/s10514-018-9784-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10514-018-9784-8