Abstract
Virtual reality enables the creation of personalized user experience that brings together people of different cultures and ethnicity. We consider a novel concept of virtual reality innovation in museums, which is cognitively grounded and supported by data and their semantics, to enable users sharing their experiences, as well as to take the perspective of other users, with the ultimate goal of increasing social cohesion. The implementation of this scenario requires an autonomous artificial system that detects emotions and values from a dialogue involving museum visitors who express their personal point of view, listen to those from other visitors, and possibly take the perspective of others.
An important feature of this system is the ability of detecting similarity and dissimilarity between user perspectives expressed in speech, when exposed to artworks. This ability helps defining an effective strategy for sharing diverse user perspectives for increasing social cohesion. Moreover, it enables an unbiased quantification of the success of the interaction in terms of change in the user perspective. Based on results from previous work, we employ the Ekman’s emotion model and Haidt’s moral value model to extract emotional and moral value profiles from user descriptions of artworks. We propose a novel method for measuring the similarity between user perspectives by comparing emotional and moral value profiles. Our results show that the employment of unsupervised text classification models is a promising research direction for this task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
SPICE: Social Cohesion, Participation and Inclusion through Cultural Engagement – EC Grant Agreement number 870811.
- 2.
The Atlas of Emotions developed from a revised version of Ekman’s Basic Emotions theory is available here: https://atlasofemotions.org/.
- 3.
All methods take as input text documents, which can be generated by a speech-to-text tool.
References
Acheampong, F.A., Nunoo-Mensah, H., Chen, W.: Transformer models for text-based emotion detection: a review of BERT-based approaches. Artif. Intell. Rev. 54(8), 5789–5829 (2021)
Ahn, S.J., Le, A.M.T., Bailenson, J.: The effect of embodied experiences on self-other merging, attitude, and helping behavior. Media Psychol. 16(1), 7–38 (2013)
Alm, E.C.O.: Affect in* text and speech. University of Illinois at Urbana-Champaign (2008)
del Arco, F.M.P., Strapparava, C., Lopez, L.A.U., Martín-Valdivia, M.T.: Emoevent: a multilingual emotion corpus based on different events. In: Proceedings of the 12th Language Resources and Evaluation Conference, pp. 1492–1498 (2020)
Asprino, L., Bulla, L., De Giorgis, S., Gangemi, A., Marinucci, L., Mongiovì, M.: Uncovering values: detecting latent moral content from natural language with explainable and non-trained methods. In: Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pp. 33–41 (2022)
Blascovich, J., Loomis, J., Beall, A.C., Swinth, K.R., Hoyt, C.L., Bailenson, J.N.: Immersive virtual environment technology as a methodological tool for social psychology. Psychol. Inq. 13(2), 103–124 (2002)
Casula, E.P., et al.: Feeling of ownership over an embodied avatar’s hand brings about fast changes of fronto-parietal cortical dynamics. J. Neurosci. 42(4), 692–701 (2022)
Daga, E., et al.: Integrating citizen experiences in cultural heritage archives: requirements, state of the art, and challenges. ACM J. Comput. Cult. Herit. (JOCCH) 15(1), 1–35 (2022)
Dan-Glauser, E.S., Scherer, K.R.: The difficulties in emotion regulation scale (DERS). Swiss J. Psychol. (2012)
Demszky, D., Movshovitz-Attias, D., Ko, J., Cowen, A., Nemade, G., Ravi, S.: Goemotions: a dataset of fine-grained emotions. arXiv preprint arXiv:2005.00547 (2020)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Ekman, P.: Basic emotions. In: Handbook of Cognition and Emotion, vol. 98, no. 45–60, p. 16 (1999)
Ekman, P.: Atlas of emotion (2018)
Fusaro, M., Tieri, G., Aglioti, S.M.: Seeing pain and pleasure on self and others: behavioral and psychophysiological reactivity in immersive virtual reality. J. Neurophysiol. 116(6), 2656–2662 (2016)
Graham, J., et al.: Moral foundations theory: the pragmatic validity of moral pluralism. In: Advances in Experimental Social Psychology, vol. 47, pp. 55–130. Elsevier (2013)
Graham, J., Nosek, B.A., Haidt, J.: The moral stereotypes of liberals and conservatives: exaggeration of differences across the political spectrum. PLoS ONE 7(12), e50092 (2012)
Haidt, J.: The emotional dog and its rational tail: a social intuitionist approach to moral judgment. Psychol. Rev. 108(4), 814 (2001)
Haidt, J.: The righteous mind: why good people are divided by politics and religion. Vintage (2012)
Hamilton-Giachritsis, C., Banakou, D., Garcia Quiroga, M., Giachritsis, C., Slater, M.: Reducing risk and improving maternal perspective-taking and empathy using virtual embodiment. Sci. Rep. 8(1), 1–10 (2018)
Hartmann, J.: Emotion english distilroberta-base (2022). https://huggingface.co/j-hartmann/emotion-english-distilroberta-base/
Holt-Lunstad, J., Smith, T.B., Baker, M., Harris, T., Stephenson, D.: Loneliness and social isolation as risk factors for mortality: a meta-analytic review. Perspect. Psychol. Sci. 10(2), 227–237 (2015)
Holt-Lunstad, J., Smith, T.B., Layton, J.B.: Social relationships and mortality risk: a meta-analytic review. PLoS Med. 7(7), e1000316 (2010)
Kishore, S., Spanlang, B., Iruretagoyena, G., Halan, S., Szostak, D., Slater, M.: A virtual reality embodiment technique to enhance helping behavior of police toward a victim of police racial aggression. PRESENCE: Virtual Augmented Reality 28, 5–27 (2022)
Li, Y., Su, H., Shen, X., Li, W., Cao, Z., Niu, S.: Dailydialog: a manually labelled multi-turn dialogue dataset. arXiv preprint arXiv:1710.03957 (2017)
Lieto, A., Pozzato, G.L., Striani, M., Zoia, S., Damiano, R.: Degari 2.0: a diversity-seeking, explainable, and affective art recommender for social inclusion. Cogn. Syst. Res. 77, 1–17 (2023)
Lucifora, C., et al.: Cyber-therapy: the use of artificial intelligence in psychological practice. In: Russo, D., Ahram, T., Karwowski, W., Di Bucchianico, G., Taiar, R. (eds.) IHSI 2021. AISC, vol. 1322, pp. 127–132. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-68017-6_19
Mohammad, S., Bravo-Marquez, F., Salameh, M., Kiritchenko, S.: Semeval-2018 task 1: affect in tweets. In: Proceedings of the 12th International Workshop on Semantic Evaluation, pp. 1–17 (2018)
Nandwani, P., Verma, R.: A review on sentiment analysis and emotion detection from text. Soc. Netw. Anal. Min. 11(1), 1–19 (2021)
Pacheco, M.L., Goldwasser, D.: Modeling content and context with deep relational learning. Trans. Assoc. Comput. Linguist. 9, 100–119 (2021)
Pérez, J.M., Giudici, J.C., Luque, F.: pysentimiento: a python toolkit for sentiment analysis and socialnlp tasks. arXiv preprint arXiv:2106.09462 (2021)
Poria, S., Hazarika, D., Majumder, N., Naik, G., Cambria, E., Mihalcea, R.: Meld: a multimodal multi-party dataset for emotion recognition in conversations. arXiv preprint arXiv:1810.02508 (2018)
Priniski, J.H., et al.: Mapping moral valence of tweets following the killing of George Floyd. arXiv preprint arXiv:2104.09578 (2021)
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (2019). https://arxiv.org/abs/1908.10084
Riva, P., Eck, J.: The many faces of social exclusion. In: Social Exclusion: Psychological Approaches to Understanding and Reducing Its Impact, pp. ix–xv (2016)
Riva, P., Wirth, J.H., Williams, K.D.: The consequences of pain: the social and physical pain overlap on psychological responses. Eur. J. Soc. Psychol. 41(6), 681–687 (2011)
Saravia, E., Liu, H.C.T., Huang, Y.H., Wu, J., Chen, Y.S.: Carer: contextualized affect representations for emotion recognition. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3687–3697 (2018)
Scherer, K.R., Wallbott, H.G.: Evidence for universality and cultural variation of differential emotion response patterning. J. Pers. Soc. Psychol. 66(2), 310 (1994)
Slater, M., et al.: Bystander responses to a violent incident in an immersive virtual environment. PLoS ONE 8(1), e52766 (2013)
Slater, M., Spanlang, B., Sanchez-Vives, M.V., Blanke, O.: First person experience of body transfer in virtual reality. PLoS ONE 5(5), e10564 (2010)
Steuer, J.: Defining virtual reality: dimensions determining telepresence. J. Commun. 42(4), 73–93 (1992)
Wesselmann, E.D., Grzybowski, M.R., Steakley-Freeman, D.M., DeSouza, E.R., Nezlek, J.B., Williams, K.D.: Social exclusion in everyday life. In: Social Exclusion: Psychological Approaches to Understanding and Reducing Its Impact, pp. 3–23 (2016)
Wright, D., Augenstein, I.: Transformer based multi-source domain adaptation. arXiv preprint arXiv:2009.07806 (2020)
Yee, N., Bailenson, J.N.: Walk a mile in digital shoes: the impact of embodied perspective-taking on the reduction of negative stereotyping in immersive virtual environments. Proc. PRESENCE 24, 26 (2006)
Yin, W., Hay, J., Roth, D.: Benchmarking zero-shot text classification: datasets, evaluation and entailment approach (2019)
Acknowledgements
This work is supported by the H2020 projects TAILOR: Foundations of Trustworthy AI - Integrating Reasoning, Learning and Optimization – EC Grant Agreement number 952215 – and SPICE: Social Cohesion, Participation and Inclusion through Cultural Engagement – EC Grant Agreement number 870811, as well as by the Italian PNRR MUR project PE0000013-FAIR.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bulla, L., De Giorgis, S., Gangemi, A., Lucifora, C., Mongiovì, M. (2023). Comparing User Perspectives in a Virtual Reality Cultural Heritage Environment. In: Indulska, M., Reinhartz-Berger, I., Cetina, C., Pastor, O. (eds) Advanced Information Systems Engineering. CAiSE 2023. Lecture Notes in Computer Science, vol 13901. Springer, Cham. https://doi.org/10.1007/978-3-031-34560-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-34560-9_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-34559-3
Online ISBN: 978-3-031-34560-9
eBook Packages: Computer ScienceComputer Science (R0)