Abstract
Sensor data taken during a human-robot interaction (HRI) have high potential for usage as new, objective measures of an interaction, either replacing or supplementing survey techniques that are currently most common in HRI research. Sensor data can be taken in large quantities quickly, naturally, and discreetly. They also have the potential to reflect a user’s biosignals—information about the user’s inner state (such as stress and attention) when interacting with the robot. We previously conducted three studies attempting to use sensor data as a measurement in HRI, with methodological differences in three different experimental environments. In this paper, we reanalyze and add new data to the previous findings under a consistent methodology, consolidate what correlations we find, and can conclude that sensor data is a useful metric in HRI across a wide range of experimental setups and subject pools. We fully describe the methodology we determined to be most effective, from selection of sensors to data analysis techniques to HRI experiment setup, as a basis for how this methodology can be used in other HRI studies. We describe necessary steps in the analysis of a large amount of sensor data (over 100,000 sets) and how sensor data can be compared with survey and behavioral data. Based on these correlations, we find that the most effective sensors are temperature sensors, tactile sensors, and face distance measurements. We also find that higher measurements across all of these sensors are more correlated with both survey and behavioral measurements reflecting positive thinking towards a robot (including non-technophobia, reciprocal behaviors, and positive ratings of the robot) during an interaction. Based on these results, we argue that robot sensor usage is an important and objective metric for HRI research.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Paulhus DL (1991) Measurement and control of response bias. In: Robinson JP, Shaver PR, Wrightman LS (eds) Measures of personality and social psychological attitudes. Academic Press, San Diego, pp 17–59
Gödert HW, Rill H-G, Vossel G (2001) Psychophysiological differentiation of deception: the effects of electrodermal lability and mode of responding on skin conductance and heart rate. Int J Psychophysiol 40(1):61–75
Crawford DG, Friesen DD, Tomlinson-Keasey C (1977) Effects of cognitively induced anxiety on hand temperature. Appl Psychophysiol Biofeedback 2(2):139–146
Popescu F, Fazli S, Badower Y, Blankertz B, Müller K-R (2007) Single trial classification of motor imagination using 6 dry EEG electrodes. PLoS ONE 2:7
Kennedy DO, Scholey AB (1999) Glucose administration, heart rate and cognitive performance: effects of increasing mental effort. Psychopharmacology 149(1):63–71
Storm H, Myre K, Rostrup M, Stokland O, Lien MD, Ræder JC (2002) Skin conductance correlates with perioperative stress. Acta Anaesthesiol Scand 46(7):887–895
Boudewyns PA (1976) A comparison of the effects of stress vs. relaxation instruction on the finger temperature response. Behav Ther 7(1):54–67
Strauss M, Reynolds C, Hughes S, Park K, McDarby G, Picard RW (2005) The HandWave Bluetooth skin conductance sensor. In: Affective computing and intelligent interaction, vol 3784, pp 699–706
Munekata N, Yoshida N, Sakurazawa S, Tsukahara Y, Matsubara H (2006) Design of positive biofeedback using a robot’s behaviors as motion media. In: Lecture notes in computer science, vol 4161/2006. Springer, Berlin, pp 340–349
Shipps EM, Freeman HR (2003) Handshake: Its relation to first impressions and measured personality traits. Psi Chi J Undergrad Res 8(4):144–148
Stiehl WD, Lieberman J, Breazeal C, Basel L, Lalla L, Wolf M (2005) Design of a therapeutic robotic companion for relational, affective touch. In: Proceedings of the 2005 IEEE international workshop on robots and human interactive communication, pp 408–415, Nashville, TN, USA, August, 2005
Morwitz VG, Johnson E, Schmittlein D (1993) Does measuring intent change behavior? J Consum Res 20(1):46–61
Bainbridge WA, Nozawa S, Ueda R, Kakiuchi Y, Nagahama K, Okada K, Inaba M (2011) Understanding expectations of a robot’s identity through multi-user interactions. In: Proceedings of the human-robot interaction 2011 workshop on expectations in intuitive human-robot interaction, Lausanne, Switzerland, March, 2011
Bainbridge WA, Nozawa S, Ueda R, Kakiuchi Y, Nagahama K, Okada K, Inaba M (2011) Using biofeedback to analyze human-robot interaction experiments. In: Proceedings of the JSME robotics and mechatronics conference 2011, Okayama, Japan, May, 2011 (Japanese language text)
Bainbridge WA, Nozawa S, Ueda R, Okada K, Inaba M (2011) Robot sensor data as a means to measure human reactions to an interaction. In: Proceedings of the IEEE-RAS 2011 international conference on humanoid robots, Bled, Slovenia, October, 2011
Matsui T, Inaba M (1990) Euslisp: an object-based implementation of Lisp. J Inf Process 13:3
Quigley M, Conley K, Gerkey BP, Faust J, Foote T, Leibs J, Wheeler R, Ng AY (2009) ROS: an open-source robot operating system. In: IEEE international conference on robotics and automation: workshop on open source software, Kobe, Japan, May, 2009
Aquest (2010). AquesTalk—text-to-speech synthesis middleware. Retrieved November 1, 2010, from: http://www.aquest.com/download/index.html
Taylor P, Black AW, Caley R (1998) The architecture of the Festival speech synthesis system. In: Proceedings of the 3rd ESCA workshop on speech synthesis, pp 141–147, Jenolan Caves, Australia
Sears DC, Rosen LD, Weil MM (1988). Technophobia measurement instruments information. Retrieved April 15, 2011 from: http://www.technostress.com/WRexam.htm
Silverman AF, Pressman ME, Bartel HW (1973) Self-esteem and tactile communication. J Humanist Psychol 13(2):73–77
Bainbridge WA, Hart J, Kim ES, Scassellati B (2009–2010) The benefits of iterations with physically present robots over video-displayed agents. Int J Soc Robot, 1–2
Walters ML, Dautenhahn K, Koay KL, Kaouri C, Boekhorst R, Nehaniv C, Werry I, Lee D (2005) Close encounters: spatial distances between people and a robot of mechanistic appearance. In: Proceedings of the 5th IEEE-RAS international conference on humanoid robots, Tsukuba, Japan, December, 2005
Selvin HC, Stuart A (1966) Data-Dredging procedures in survey analysis. Am Stat 20(3):20–23
Falk B, Burstein R, Rosenblum J, Shapiro Y, Zylber-Katz E, Bashan N (1990) Effects of caffeine ingestion on body fluid balance and thermoregulation during exercise. Can J Physiol Pharm 68(7):889–892
Acknowledgements
Wilma A. Bainbridge would like to thank Yale University, The Gordon Grand Fellowship, and the Fox International Fellowship for their support for her research at the University of Tokyo.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bainbridge, W.A., Nozawa, S., Ueda, R. et al. A Methodological Outline and Utility Assessment of Sensor-based Biosignal Measurement in Human-Robot Interaction. Int J of Soc Robotics 4, 303–316 (2012). https://doi.org/10.1007/s12369-012-0146-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12369-012-0146-y