Human Interaction Smart Subsystem-Extending Speech-Based Human-Robot Interaction Systems with an Implementation of External Smart Sensors - PubMed Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Apr 22;20(8):2376.
doi: 10.3390/s20082376.

Human Interaction Smart Subsystem-Extending Speech-Based Human-Robot Interaction Systems with an Implementation of External Smart Sensors

Michal Podpora  1   2 , Arkadiusz Gardecki  1   2 , Ryszard Beniak  1   2 , Bartlomiej Klin  2 , Jose Lopez Vicario  3 , Aleksandra Kawala-Sterniuk  1
Affiliations

Human Interaction Smart Subsystem-Extending Speech-Based Human-Robot Interaction Systems with an Implementation of External Smart Sensors

Michal Podpora et al. Sensors (Basel). .

Abstract

This paper presents a more detailed concept of Human-Robot Interaction systems architecture. One of the main differences between the proposed architecture and other ones is the methodology of information acquisition regarding the robot's interlocutor. In order to obtain as much information as possible before the actual interaction took place, a custom Internet-of-Things-based sensor subsystems connected to Smart Infrastructure was designed and implemented, in order to support the interlocutor identification and acquisition of initial interaction parameters. The Artificial Intelligence interaction framework of the developed robotic system (including humanoid Pepper with its sensors and actuators, additional local, remote and cloud computing services) is being extended with the use of custom external subsystems for additional knowledge acquisition: device-based human identification, visual identification and audio-based interlocutor localization subsystems. These subsystems were deeply introduced and evaluated in this paper, presenting the benefits of integrating them into the robotic interaction system. In this paper a more detailed analysis of one of the external subsystems-Bluetooth Human Identification Smart Subsystem-was also included. The idea, use case, and a prototype, integration of elements of Smart Infrastructure systems and the prototype implementation were performed in a small front office of the Weegree company as a decent test-bed application area.

Keywords: COVID-19; cloud services; humanoid robots; pepper robot; sensor networks; smart infrastructure; thermal imaging.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Simplified overview diagram of the proposed system’s components.
Figure 2
Figure 2
Simplified scheme of the processing of video input streams performed by the VISS.
Figure 3
Figure 3
Sample image from the video datalog of the system, visualizing the data acquired by the thermal imaging sensor subsystem.
Figure 4
Figure 4
Visualization of the zones for different identification tasks and methods.
Figure 5
Figure 5
Comparison of mechanisms for obtaining information regarding interlocutor. Use case (A)—no RFID or Bluetooth signals available. Use case (B)—RFID UHF and Bluetooth subsystems are enabled. Red wide arrow means no information about incoming object/person, yellow wide arrow means partial information about object/person, green wide arrow means detailed information about object/person and blue narrow arrow means the possibility to start personalized interaction between robot (AI framework agent) and interlocutor.
Figure 6
Figure 6
Real-world implementation of the proposed concept. Prototype system with humanoid robot Pepper ready to work. ©2020 IEEE. Reprinted, with permission, from [20].
Figure 7
Figure 7
Bluetooth usability test—7 different test scenarios of a BT device carried towards the Bluetooth-based Human Identification Smart Sensor (BT HISS) node. X axis—distance in meters, Y axis—RSSI as a relative index (0 = best quality).
Figure 8
Figure 8
Relation of relative Google Speech-to-text service response times after sending the last frame of the audio speech data to Google.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Podpora M., Gardecki A., Kawala-Sterniuk A. Humanoid Receptionist Connected to IoT Subsystems and Smart Infrastructure is Smarter than Expected. IFAC-PapersOnLine. 2019;52:347–352. doi: 10.1016/j.ifacol.2019.12.685. - DOI
    1. Gardecki A., Podpora M., Kawala-Janik A. Innovative Internet of Things-reinforced Human Recognition for Human-Machine Interaction Purposes. IFAC-PapersOnLine. 2018;51:138–142. doi: 10.1016/j.ifacol.2018.07.143. - DOI
    1. Redondo M.E.L. Comfortability Detection for Adaptive Human-Robot Interactions; Proceedings of the 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW); Cambridge, UK. 3–6 Setember 2019; pp. 35–39.
    1. Belanche D., Casalo L.V., Flavian C. Marketing and Smart Technologies. Springer; Singapore: 2020. Customer’s Acceptance of Humanoid Robots in Services: The Moderating role of Risk Aversion; pp. 449–458.
    1. Cambridge Centre for Smart Infrastructure and Construction What Is Smart Infrastructure? [(accessed on 1 April 2020)]; Available online: https://iotevent.eu/what-is-smart-infrastructure/

LinkOut - more resources