Who, Me? How Virtual Agents Can Shape Conversational Footing in Virtual Reality | SpringerLink
Skip to main content
Springer Nature Link
Log in

Who, Me? How Virtual Agents Can Shape Conversational Footing in Virtual Reality

  • Conference paper
  • First Online:
Intelligent Virtual Agents (IVA 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10498))

Included in the following conference series:

Abstract

The nonverbal behaviors of conversational partners reflect their conversational footing, signaling who in the group are the speakers, addressees, bystanders, and overhearers. Many applications of virtual reality (VR) will involve multiparty conversations with virtual agents and avatars of others where appropriate signaling of footing will be critical. In this paper, we introduce computational models of gaze and spatial orientation that a virtual agent can use to signal specific footing configurations. An evaluation of these models through a user study found that participants conformed to conversational roles signaled by the agent and contributed to the conversation more as addressees than as bystanders. We observed these effects in immersive VR, but not on a 2D display, suggesting an increased sensitivity to virtual agents’ footing cues in VR-based interfaces.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Andrist, S., Mutlu, B., Gleicher, M.: Conversational gaze aversion for virtual agents. In: Aylett, R., Krenn, B., Pelachaud, C., Shimodaira, H. (eds.) IVA 2013. LNCS (LNAI), vol. 8108, pp. 249–262. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40415-3_22

    Chapter  Google Scholar 

  2. Andrist, S., Pejsa, T., Mutlu, B., Gleicher, M.: Designing effective gaze mechanisms for virtual agents. In: Proc. CHI 2012, pp. 705–714 (2012)

    Google Scholar 

  3. Argyle, M., Cook, M.: Gaze and mutual gaze. Cambridge University Press (1976)

    Google Scholar 

  4. Argyle, M., Dean, J.: Eye-contact, distance and affiliation. Sociometry 28, 289–304 (1965)

    Article  Google Scholar 

  5. Aron, A., Aron, E.N., Smollan, D.: Inclusion of other in the self scale and the structure of interpersonal closeness. Journal of Personality and Social Psychology 63(4), 596 (1992)

    Article  Google Scholar 

  6. Bailenson, J.N., Beall, A.C., Blascovich, J.: Gaze and task performance in shared virtual environments. The Journal of Visualization and Computer Animation 13(5), 313–320 (2002)

    Article  Google Scholar 

  7. Bailenson, J.N., Blascovich, J., Beall, A.C., Loomis, J.M.: Equilibrium theory revisited: Mutual gaze and personal space in virtual environments. Presence 10(6), 583–598 (2001)

    Article  Google Scholar 

  8. Bailenson, J.N., Blascovich, J., Beall, A.C., Loomis, J.M.: Interpersonal distance in immersive virtual environments. Personality and Social Psychology Bulletin 29(7), 819–833 (2003)

    Article  Google Scholar 

  9. Bohus, D., Horvitz, E.: Models for multiparty engagement in open-world dialog. In: Proc. SIGDIAL 2009, pp. 225–234 (2009)

    Google Scholar 

  10. Bohus, D., Horvitz, E.: Multiparty turn taking in situated dialog: study, lessons, and directions. In: Proc. SIGDIAL 2011, pp. 98–109 (2011)

    Google Scholar 

  11. Garau, M., Slater, M., Vinayagamoorthy, V., Brogni, A., Steed, A., Sasse, M.A.: The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. In: Proc. CHI 2003, pp. 529–536 (2003)

    Google Scholar 

  12. Geller, D.M., Goodstein, L., Silver, M., Sternberg, W.C.: On being ignored: The effects of the violation of implicit rules of social interaction. Sociometry, 541–556 (1974)

    Article  Google Scholar 

  13. Goffman, E.: Footing. Semiotica 25(1–2), 1–30 (1979)

    Article  Google Scholar 

  14. Heylen, D., Van Es, I., Van Dijk, E., Nijholt, A., Van Dijk, B.: Experimenting with the gaze of a conversational agent. In: Proc. CLASS 2005, pp. 93–100 (2002)

    Google Scholar 

  15. Heylen, D.K.J.: Head gestures, gaze and the principles of conversational structure. International Journal of Humanoid Robotics 3(3), 241–267 (2006)

    Article  Google Scholar 

  16. Hollands, M.A., Ziavra, N.V., Bronstein, A.M.: A new paradigm to investigate the roles of head and eye movements in the coordination of whole-body movements. Experimental Brain Research 154(2), 261–266 (2004)

    Article  Google Scholar 

  17. Kendon, A.: Conducting interaction: Patterns of behavior in focused encounters. Cambridge University Press (1990)

    Google Scholar 

  18. Kendon, A.: Some functions of gaze-direction in social interaction. Acta Psychologica 26, 22–63 (1967)

    Article  Google Scholar 

  19. Kuzuoka, H., Suzuki, Y., Yamashita, J., Yamazaki, K.: Reconfiguring spatial formation arrangement by robot body orientation. In: Proc. HRI 2010, pp. 285–292 (2010)

    Google Scholar 

  20. Lee, S.P., Badler, J.B., Badler, N.I.: Eyes alive. ACM ToG 21, 637–644 (2002)

    Google Scholar 

  21. McCluskey, M.K., Cullen, K.E.: Eye, head, and body coordination during large gaze shifts in rhesus monkeys: Movement kinematics and the influence of posture. Journal of Neurophysiology 97(4), 2976–2991 (2007)

    Article  Google Scholar 

  22. Mutlu, B., Kanda, T., Forlizzi, J., Hodgins, J., Ishiguro, H.: Conversational gaze mechanisms for humanlike robots. ACM TiiS 1(2), 12:1–12:33 (2012)

    Google Scholar 

  23. Pedica, C., Vilhjálmsson, H.H.: Spontaneous avatar behavior for human territoriality. Applied Artificial Intelligence 24(6), 575–593 (2010)

    Article  Google Scholar 

  24. Pedica, C., Högni Vilhjálmsson, H., Lárusdóttir, M.: Avatars in conversation: the importance of simulating territorial behavior. In: Allbeck, J., Badler, N., Bickmore, T., Pelachaud, C., Safonova, A. (eds.) IVA 2010. LNCS (LNAI), vol. 6356, pp. 336–342. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15892-6_35

    Chapter  Google Scholar 

  25. Pejsa, T., Andrist, S., Gleicher, M., Mutlu, B.: Gaze and attention management for embodied conversational agents. ACM TiiS 5(1), 3:1–3:34 (2015)

    Google Scholar 

  26. Schegloff, E.A.: Sequencing in conversational openings. American Anthropologist 70(6), 1075–1095 (1968)

    Article  Google Scholar 

  27. Steptoe, W., Wolff, R., Murgia, A., Guimaraes, E., Rae, J., Sharkey, P., Roberts, D., Steed, A.: Eye-tracking for avatar eye-gaze and interactional analysis in immersive collaborative virtual environments. In: Proc. CSCW 2008, pp. 197–200 (2008)

    Google Scholar 

  28. Uemura, T., Arai, Y., Shimazaki, C.: Eye-head coordination during lateral gaze in normal subjects. Acta Oto-Laryngologica 90(3–4), 191–198 (1980)

    Article  Google Scholar 

  29. Wieser, M.J., Pauli, P., Grosseibl, M., Molzow, I., Mühlberger, A.: Virtual social interactions in social anxiety-the impact of sex, gaze, and interpersonal distance. Cyberpsychology, Behavior, and Social Networking 13(5), 547–554 (2010)

    Article  Google Scholar 

  30. Williams, K.D., Cheung, C.K.T., Choi, W.: Cyberostracism: effects of being ignored over the internet. Journal of Personality and Social Psychology 79(5), 748 (2000)

    Article  Google Scholar 

  31. Yee, N., Bailenson, J.N., Urbanek, M., Chang, F., Merget, D.: The unbearable likeness of being digital: The persistence of nonverbal social norms in online virtual environments. CyberPsychology & Behavior 10(1), 115–121 (2007)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Wisconsin–Madison, Madison, USA

    Tomislav Pejsa, Michael Gleicher & Bilge Mutlu

Authors
  1. Tomislav Pejsa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Gleicher
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bilge Mutlu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomislav Pejsa .

Editor information

Editors and Affiliations

  1. KTH Royal Institute of Technology, Stockholm, Sweden

    Jonas Beskow

  2. KTH Royal Institute of Technology, Stockholm, Sweden

    Christopher Peters

  3. Uppsala University, Uppsala, Sweden

    Ginevra Castellano

  4. Trinity College, Dublin, Ireland

    Carol O'Sullivan

  5. KTH Royal Institute of Technology, Stockholm, Sweden

    Iolanda Leite

  6. University of Bielefeld, Bielefeld, Germany

    Stefan Kopp

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Pejsa, T., Gleicher, M., Mutlu, B. (2017). Who, Me? How Virtual Agents Can Shape Conversational Footing in Virtual Reality. In: Beskow, J., Peters, C., Castellano, G., O'Sullivan, C., Leite, I., Kopp, S. (eds) Intelligent Virtual Agents. IVA 2017. Lecture Notes in Computer Science(), vol 10498. Springer, Cham. https://doi.org/10.1007/978-3-319-67401-8_45

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-67401-8_45

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-67400-1

  • Online ISBN: 978-3-319-67401-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation