Abstract
Teleoperation of mobile machines can have benefits on numerous aspects, including improved safety, enhanced efficiency, and reduced environmental impact. However, effects like increased latency, lack of machine feeling and limited field of view result in challenges for operators. The use of VR-systems with operator head motion tracking could improve its application. This paper presents the investigation of a tele-remote control system, using a VR-visual feedback system including a stereo camera that mimics the operator’s head motion. An operator study was conducted to evaluate the influence of the tele-remote control system. It is shown that the investigated system is able to increase the achieved immersion and thus improve the operability of teleoperated mobile machines.
Zusammenfassung
Die Fernsteuerung mobiler Maschinen bietet zahlreiche Vorteile, wie verbesserte Sicherheit, Kosteneinsparungen, höhere Effizienz und geringere Umweltbelastung. Dennoch führen Effekte wie erhöhte Latenzzeiten, fehlendes Maschinengefühl und eine eingeschränkte Sicht zu Herausforderungen für Bediener. Der Einsatz von VR-Systemen könnte die Fernsteuerung in einem breiteren Spektrum von Anwendungen ermöglichen. Daher wird in diesem Beitrag die praktische Umsetzung eines Fernbedienungssystems unter Verwendung eines VR-Headsets und einer Stereokamera, die die Kopfbewegung des Bedieners nachahmt, vorgestellt. Um die Eignung solcher Systeme zu bewerten, wurde eine Bedienerstudie durchgeführt, deren Ergebnisse vorgestellt werden. Es wird gezeigt, dass das untersuchte System in der Lage ist, die erreichte Immersion zu erhöhen und damit die Bedienbarkeit von teleoperierten mobilen Maschinen zu verbessern.
About the authors
Simon Gerdes began his education in mechanical engineering while still in high school. He studied mechanical engineering at RWTH Aachen University, graduating with a bachelor's degree, followed by a master's degree in production engineering in 2023.
Johannes Sprink studied Mechanical Engineering at RWTH Aachen University. Since 2021 he has been a research associate at the Institute for Fluid Power Drives and Systems (ifas) at RWTH Aachen University. He is a team member of the Mobile Machines research group, which addresses fluid-mechatronic and information-technologic innovations at the machine level. Furthermore, he investigates greenhouse gas emissions in the fluid power industry and his research includes optimizing the operating concepts of hydraulic systems.
Christian Haas studied Mechanical Engineering at RWTH Aachen University. Since 2019 he has been a research associate at the Institute for Fluid Power Drives and Systems (ifas) at RWTH Aachen University. From 2023 onward he is group leader of the Mobile Machines research group, which addresses fluid-mechatronic and information-technologic innovations at the machine level. His research focus is in control engineering for fluid power drives and systems in stationary and mobile applications, to increase system performance and control accuracy.
Andreas Opgenoorth, M.Sc., studied Bachelor Mechanical Engineering with a specialization in design engineering from 2012 to 2016 and Master Development and Design at RWTH Aachen University from 2016 to 2018. Since 2018, Mr. Opgenoorth has been employed at the Institute for Fluid Power Drives and Systems (ifas) at RWTH Aachen University. Since 2018, he has worked as a research assistant in the Mobile and Stationary Systems research group. From December 2021 to May 2022, he headed the research group of Mobile Machinery. His main focus is on control and system design of hydraulic drive systems and integration of electric drives.
Prof. Katharina Schmitz graduated in mechanical engineering at RWTH Aachen University in 2010 with part of her studies taking place at Carnegie Mellon University in Pittsburgh (USA) and working in Le Havre (France). After graduation, she worked as a scientific staff member and Deputy Chief Engineer at ifas, the Institute for Fluid Power Drives and Controls of Prof. Murrenhoff. In 2015, she graduated as Dr.-Ing. and started working in the industrial sector for a family-owned company, which focuses on special purpose hydraulic solutions and large cylinders. There, she was promoted to Technical Director in 2016. Since March 2018 she has been a full professor at RWTH Aachen university and Director of ifas, the Institute or Fluid Power Drives and Systems.
-
Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
-
Research funding: The authors would like to thank the Hans Hermann Voss Foundation, Volvo Construction Equipment and Danfoss Power Solutions for funding and provision of hardware for the demonstrator.
-
Conflict of interest statement: The authors declare no conflicts of interest regarding this article.
References
[1] R. R. Murphy, “Human–robot interaction in rescue robotics,” IEEE Trans. Syst. Man Cybern. C, vol. 34, no. 2, pp. 138–153, 2004. https://doi.org/10.1109/TSMCC.2004.826267.Search in Google Scholar
[2] J. Y. C. Chen, E. C. Haas, and M. J. Barnes, “Human performance issues and user interface design for teleoperated robots,” IEEE Trans. Syst. Man Cybern. C, vol. 37, no. 6, pp. 1231–1245, 2007. https://doi.org/10.1109/TSMCC.2007.905819.Search in Google Scholar
[3] Hitachi, Verification Tests to Start Directed at the Development of Autonomous Ultra-large Hydraulic Excavators, Tokyo, Japan, 2020. Available at: https://www.hitachicm.com/global/en/news/press-releases/2020/20-07-16/, Updated on 2/14/2023, checked on 2/14/2023.Search in Google Scholar
[4] Komatsu, Two-way Radio Remote – Technology – United Kingdom | Komatsu Mining Corp, 2023. Available at: https://mining.komatsu/en-gb/technology/operator-assist/two-way-radio-remote, Updated on 2/13/2023, checked on 2/13/2023.Search in Google Scholar
[5] Komatsu, Komatsu to Teleoperate an Excavator in Arizona | Komatsu Australia – Komatsu Australia, 2021. Available at: https://www.komatsu.com.au/company/news-media/news/komatsu-to-teleoperate-an-excavator-in-arizona, Updated on 2/13/2023, checked on 2/13/2023.Search in Google Scholar
[6] Komatsu, Drill Tele-Remote – Operator assist Technology – United Kingdom | Komatsu Mining Corp, 2023. Available at: https://mining.komatsu/en-gb/technology/operator-assist/drill-tele-remote, Updated on 2/13/2023, checked on 2/13/2023.Search in Google Scholar
[7] Caterpillar, Cat® Remote Operator Station for Dozing Provides Safe, Comfortable Environment and Boosts Efficiency. Edited by MINING.COM, 2015. Available at: https://www.mining.com/web/cat-remote-operator-station-for-dozing-provides-safe-comfortable-environment-and-boosts-efficiency/, Updated on 2/14/2023, checked on 2/14/2023.Search in Google Scholar
[8] Caterpillar, Minestar | Command | Cat | Caterpillar, 2023. Available at: https://www.cat.com/en_US/by-industry/mining/surface-mining/surface-technology/command.html, Updated on 2/14/2023, checked on 2/14/2023.Search in Google Scholar
[9] Volvo, CE, Volvo CE Tests World’s First High-Lift Tele-Operation Over 5G, 2021, Updated on 11/18/2021, checked on 6/20/2023. Available at: https://www.volvoce.com/global/en/news-and-events/press-releases/2021/volvo-ce-tests-worlds-first-high-lift-tele-operation-over-5g/.Search in Google Scholar
[10] M. Hutter, T. Braungardt, F. Grigis, et al.., “IBEX — a tele-operation and training device for walking excavators,” in 2016 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR). 2016 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Lausanne, Switzerland, IEEE, 2016, pp. 48–53.10.1109/SSRR.2016.7784276Search in Google Scholar
[11] Bd.redaktion, Zeppelin-Lösungen unterstützen die Fahrerschulung – bd baumaschinendienst, 2022. Available at: https://www.baumaschinendienst.de/artikel/zeppelin-loesungen-unterstuetzen-die-fahrerschulung-19720/, Updated on 11/12/2022, checked on 2/14/2023.Search in Google Scholar
[12] Ximea, XIMEA – Case Study: Remotely Operated Walking Excavator, 2020. Available at: https://www.ximea.com/en/corporate-news/excavator-remotely-operated-tele, Updated on 2/14/2023, checked on 2/14/2023.Search in Google Scholar
[13] M. Slater and M. V. Sanchez-Vives, “Enhancing our lives with immersive virtual reality,” Front. Robot. AI, vol. 3, pp. 3–35, 2016. https://doi.org/10.3389/frobt.2016.00074.Search in Google Scholar
[14] A. S. Fernandes and S. K. Feiner, “Combating VR sickness through subtle dynamic field-of-view modification,” in 2016 IEEE Symposium on 3D User Interfaces (3DUI). 2016 IEEE Symposium on 3D User Interfaces (3DUI), Greenville, SC, USA, IEEE, 2016, pp. 201–210.10.1109/3DUI.2016.7460053Search in Google Scholar
[15] S. Ubik and J. Pospisilik, “Video camera latency analysis and measurement,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 1, pp. 140–147, 2021. https://doi.org/10.1109/TCSVT.2020.2978057.Search in Google Scholar
[16] C. Haas, A. Schneider, A. Opgenoorth, and K. Schmitz, “Practical evaluation of a control concept for a remote controlled 1.8T excavator using a 3D input device,” in BATH/ASME 2022 Symposium on Fluid Power and Motion Control. BATH/ASME 2022 Symposium on Fluid Power and Motion Control, Bath, UK, American Society of Mechanical Engineers, 2022.10.1115/FPMC2022-88959Search in Google Scholar
[17] P. J. Gianaros, E. R. Muth, J. T. Mordkoff, M. E. Levine, and R. M. Stern, “A questionnaire for the assessment of the multiple dimensions of motion sickness,” Aviat. Space Environ. Med., vol. 72, no. 2, pp. 115–119, 2001.Search in Google Scholar
[18] S. Sartisohn, “Auswirkungen von Schiffsbewegung auf physische und psychische Leistungsfähigkeit bei Besatzungsmitgliedern auf See.” Dissertation, 2019. Available at: https://macau.uni-kiel.de/servlets/MCRFileNodeServlet/dissertation_derivate_00008356/Simon_Sartisohn.pdf, Checked on 2/15/2023.Search in Google Scholar
[19] R. S. Kennedy, N. E. Lane, K. S. Berbaum, and M. G. Lilienthal, “Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness,” Int. J. Aviat. Psychol., vol. 3, no. 3, pp. 203–220, 1993. https://doi.org/10.1207/s15327108ijap0303_3.Search in Google Scholar
[20] T. M. W. Talsma, O. Hassanain, R. Happee, and K. N. de Winkel, “Validation of a moving base driving simulator for motion sickness research,” Appl. Ergon., vol. 106, p. 103897, 2023. https://doi.org/10.1016/j.apergo.2022.103897.Search in Google Scholar PubMed
© 2023 Walter de Gruyter GmbH, Berlin/Boston
