Virtual Reality-Based Time-Delayed Haptic Teleoperation Using Point Cloud Data | Journal of Intelligent & Robotic Systems Skip to main content
Springer Nature Link
Log in

Virtual Reality-Based Time-Delayed Haptic Teleoperation Using Point Cloud Data

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

This paper presents a methodology for bilateral teleoperation of robots based on the use of point cloud data obtained with an RGB-D camera for providing haptic feedback, as well as a virtual robot workspace that allows the user to interact with the teleoperated robot and the environment, regardless of the time-delay between the master device and the slave robot. The robot movements are executed first in the Virtual Workspace and then in the real one. The Virtual Workspace is created using point cloud data, and the haptic feedback is calculated directly using this data. The proposed methodology was validated in 2D and 3D trajectory-following experiments with 20 volunteers each teleoperating a KUKA industrial robot. In addition, we studied two inverse kinematic teleoperation modes: “Proportional Workspace” and “Delta end-effector movements”, with the objective of determining which one is more suitable for the haptic teleoperation methodology proposed. The results obtained confirm that the proposed haptic teleoperation methodology enhances the precision of the operator by reducing the average error between the robot end-effector trajectory and the goal trajectory. Furthermore, the likelihood that the end-effector will collide with objects in the environment is reduced. The methodology also diminishes the time that it takes to complete the trajectory-following task in an unknown workspace. The inverse kinematic teleoperation mode with the better results is the “Proportional Workspace”, which is more intuitive for operators (users).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Hokayem, P.F., Spong, M.W.: Bilateral teleoperation: An historical survey. Automatica 42(12), 2035–2057 (2006)

    Article  MathSciNet  Google Scholar 

  2. Schiele, A., Krüger, T., Kimmer, S., Aiple, M., Rebelo, J., Smisek, J., den Exter, E., Mattheson, E., Hernandez, A., van der Hulst, F.: Haptics-2 — A system for bilateral control experiments from space to ground via geosynchronous satellites, IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 892–897 (2016)

  3. National Aeronautics and Space Administration Goddard Space Flight Center: Satellite servicing capabilities office. [Online]. Available: http://ssco.gsfc.nasa.gov/ (2015)

  4. Kraft, R., 13-046, D. Washington.: NASA press release NASA’s refueling demonstration proves viability of satellite-servicing technologies, Technical report NASA (2013)

  5. Joppin, C., Hastings, D.E.: On-orbit upgrade and repair: The Hubble Space Telescope example. J. Spacecr. Rocket. 43(3), 614–625 (2006)

    Article  Google Scholar 

  6. Lanzerotti, L.J.: Assessment of options for extending the life of the Hubble Space Telescope: Final Report, National Academy Press (2005)

  7. Stoll, E., Letschnik, J., Walter, U., Artigas, J., Kremer, P., Preusche, C., Hirzinger, G.: On-orbit servicing. IEEE Robot. Autom. Mag. 16(4), 29–33 (2009)

    Article  Google Scholar 

  8. Duff, E., Caris, C., Bonchis, A., Taylor, K., Gunn, C., Adcock, M.: The development of a telerobotic rock breaker. Springer Tracts in Advanced Robotics 62, 411–420 (2010)

    Article  Google Scholar 

  9. Boeing, A.: A remotely operated robotic rock breaker with collision avoidance for the mining industry. In: Proceedings of the 30th international association for automation and robotics in construction, pp 875–884 (2013)

  10. Uzunoğlu, E., Mehmet İsmet, CD: Extending model-mediation method to multi-degree-of-freedom teleoperation systems experiencing time delays in communication Robotica, pp 1–16. Cambridge University Press, Cambridge (2015)

    Google Scholar 

  11. Ju, Z., Yang, C., Li, Z., Cheng, L., Ma, H.: Teleoperation of humanoid baxter robot using haptic feedback, Intl. conference on multisensor fusion and information integration for intelligent systems, pp. 1-6 September (2014)

  12. Leeper, A., Hsiao, K., Ciocarlie, M., Sucan, I., Salisbury, K.: Methods for collision-free arm teleoperation in clutter using constraints from 3d sensor data, Intl. Conference on Humanoid Robots, pp. 520–527 (2013)

  13. Mitra, P., Niemeyer, G.: Model-mediated telemanipulation. Int. J. Robot. Res. 27(2), 253–262 (2008)

    Article  Google Scholar 

  14. Vozar, S., Leonard, S., Kazanzides, P., Whitcomb, L.: Experimental evaluation of force control for virtual-fixture-assisted teleoperation for on-orbit manipulation of satellite thermal blanket insulation, IEEE international conference on robotics and automation (ICRA), pp. 4424–4431 (2015)

  15. Xia, T., Leonard, S., Kandaswamy, I., Blank, A., Whitcomb, L., Kazanzides, P.: Model-based telerobotic control with virtual fixtures for satellite servicing tasks, IEEE international conference on robotics and automation (ICRA), pp. 1479,1484 (2013)

  16. Xia, T., Leonard, S., Deguet, A., Whitcomb, L., Kazanzides, P.: Augmented reality environment with virtual fixtures for robotic telemanipulation in space, IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 5059–5064 (2012)

  17. Zainan, J., Hong, L., Jie, W., Jianbin, H.: Virtual reality-based teleoperation with robustness against modeling errors. Chin. J. Aeronaut. 22(3), 325–333 (2009)

    Article  Google Scholar 

  18. Jiang, Z., Liu, Y., Liu, H., Zou, J: Flexible virtual fixture enhanced by vision and haptics for unstructured environment teleoperation, IEEE international conference on robotics and biomimetics (ROBIO), pp. 2643–2648 (2013)

  19. Mitra, P., Niemeyer, G.: Haptic simulation of manipulator collisions using dynamic proxies. Presence: Teleoperators and Virtual Environments 16(4), 367–384 (2007)

    Article  Google Scholar 

  20. El-Far, N., Georganas, N., El Saddik, A.: An algorithm for haptically rendering objects described by point clouds, Canadian conference on electrical and computer engineering, pp. 1443–1448 (2008)

  21. Rydén, F., Nia Kosari, S., Chizeck, H.: Proxy Method for Fast Haptic Rendering from Time Varying Point Clouds, Proc. IEEE/ RSJ Int’l Conf. intelligent robots and systems (IROS ‘12), pp. 2614–2619 (2011)

  22. Rydén, F., Chizeck, H.: A proxy method for real-time 3-DOF haptic rendering of streaming point cloud data. IEEE Trans. Haptic 6(3), 257–267 (2013)

    Article  Google Scholar 

  23. Rydén, F., Stewart, A., Chizeck, H.: Advanced telerobotic underwater manipulation using virtual fixtures and haptic rendering, Oceans - San Diego, 2013, pp. 1–8 (2013)

  24. Xu, X., Cizmeci, B., Al-Nuaimi, A., Steinbach, E.: Point cloud-based model-mediated teleoperation with dynamic and perception-based model updating. IEEE Trans. Instrum. Meas. 63(11), 2558–2569 (2014)

    Article  Google Scholar 

  25. Rusu, R., Cousins, S.: 3D is here: Point cloud library (PCL), IEEE intl. conference on robotics and automation (ICRA) (2011)

  26. Sucan, I.A., Chitta, S.: MoveIt!, [Online] Available: http://moveit.ros.org

  27. Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., Ng, A. Y.: ROS: an open-source robot operating system, Proc. open-source software workshop of the intl conference on robotics and automation (ICRA) (2009)

  28. Chavent, P.: Rapport de stage de deuxième année, September, 2003. Available: http://paul.chavent.free.fr/kuka/doc/rapport.pdf

  29. Chavent, P., Kuka Manuel du devellopeur, A.P.I.: Version 0.0.5, September, 2003. Available: http://paul.chavent.free.fr/kuka/doc/kuka_api_man_devl.pdf

  30. Huang, K., Jiang, L., Smith, J., Chizeck, H.: Sensor-aided teleoperated grasping of transparent objects, IEEE international conference on robotics and automation (ICRA), pp. 4953–4959 (2015)

Download references

Author information

Authors and Affiliations

  1. Electrical Engineering Department and the Advanced Mining Technology Center, Universidad de Chile, Santiago, Chile

    David Valenzuela-Urrutia, Rodrigo Muñoz-Riffo & Javier Ruiz-del-Solar

Authors
  1. David Valenzuela-Urrutia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rodrigo Muñoz-Riffo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Javier Ruiz-del-Solar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Valenzuela-Urrutia.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Valenzuela-Urrutia, D., Muñoz-Riffo, R. & Ruiz-del-Solar, J. Virtual Reality-Based Time-Delayed Haptic Teleoperation Using Point Cloud Data. J Intell Robot Syst 96, 387–400 (2019). https://doi.org/10.1007/s10846-019-00988-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-019-00988-1

Keywords

Search

Navigation