Identifying Real and Posed Smiles from Observers’ Galvanic Skin Response and Blood Volume Pulse | SpringerLink
Skip to main content
Springer Nature Link
Log in

Identifying Real and Posed Smiles from Observers’ Galvanic Skin Response and Blood Volume Pulse

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12532))

Included in the following conference series:

Abstract

This study addresses the question whether galvanic skin response (GSR) and blood volume pulse (BVP) of untrained and unaided observers can be used to identify real and posed smiles from different sets of smile videos or smile images. Observers were shown smile face videos/images, either singly or paired, with the intention to recognise each viewed as real or posed smiles. We created four experimental situations, namely single images (SI), single videos (SV), paired images (PI), and paired videos (PV). The GSR and BVP signals were recorded and processed. Our machine learning classifiers reached the highest accuracy of 93.3%, 87.6%, 92.0%, 91.7% for PV, PI, SV, and SI respectively. Finally, PV and SI were found to be the easiest and hardest way to identify real and posed smiles respectively. Overall, we demonstrated that observers’ subconscious physiological signals (GSR and BVP) are able to identify real and posed smiles at a good accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11439
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14299
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Ayata, D., Yaslan, Y., Kamaşak, M.: Emotion recognition via galvanic skin response: comparison of machine learning algorithms and feature extraction methods. Istanbul Univ.-J. Electr. Electron. Eng. 17(1), 3147–3156 (2017)

    Google Scholar 

  2. Bailenson, J.N., et al.: Real-time classification of evoked emotions using facial feature tracking and physiological responses. Int. J. Hum. Comput. Stud. 66(5), 303–317 (2008)

    Article  Google Scholar 

  3. Barger, P.B., Grandey, A.A.: Service with a smile and encounter satisfaction: emotional contagion and appraisal mechanisms. Acad. Manag. J. 49(6), 1229–1238 (2006)

    Article  Google Scholar 

  4. Braithwaite, J.J., Watson, D.G., Jones, R., Rowe, M.: A guide for analysing electrodermal activity (EDA) & skin conductance responses (SCRs) for psychological experiments. Psychophysiology 49(1), 1017–1034 (2013)

    Google Scholar 

  5. Bugental, D.B.: Unmasking the “polite smile” situational and personal determinants of managed affect in adult-child interaction. Pers. Soc. Psychol. Bull. 12(1), 7–16 (1986)

    Article  Google Scholar 

  6. Deutsch, F.M., LeBaron, D., Fryer, M.M.: What is in a smile? Psychol. Women Q. 11(3), 341–352 (1987)

    Article  Google Scholar 

  7. Dibeklioglu, H., Valenti, R., Salah, A.A., Gevers, T.: Eyes do not lie: spontaneous versus posed smiles. In: Proceedings of the 18th ACM International Conference on Multimedia, pp. 703–706 (2010)

    Google Scholar 

  8. Dzedzickis, A., Kaklauskas, A., Bucinskas, V.: Human emotion recognition: review of sensors and methods. Sensors 20(3), 592 (2020)

    Article  Google Scholar 

  9. Ekman, P., Davidson, R.J.: Voluntary smiling changes regional brain activity. Psychol. Sci. 4(5), 342–345 (1993)

    Article  Google Scholar 

  10. Ekman, P., Friesen, W.V.: Felt, false, and miserable smiles. J. Nonverbal Behav. 6(4), 238–252 (1982)

    Article  Google Scholar 

  11. El Ayadi, M., Kamel, M.S., Karray, F.: Survey on speech emotion recognition: features, classification schemes, and databases. Pattern Recogn. 44(3), 572–587 (2011)

    Article  Google Scholar 

  12. A medical-grade wearable device that offers real-time physiological data acquisition (2020). https://www.empatica.com/en-int/research/e4/

  13. Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recogn. 36(1), 259–275 (2003)

    Article  Google Scholar 

  14. Gouizi, K., Bereksi Reguig, F., Maaoui, C.: Emotion recognition from physiological signals. J. Med. Eng. Technol. 35(6–7), 300–307 (2011)

    Article  Google Scholar 

  15. Dibeklioglu, H., Salah, A.A., Gevers, T.: Recognition of genuine smiles. IEEE Trans. Multimedia 17, 279–294 (2015)

    Article  Google Scholar 

  16. Hossain, M.Z., Gedeon, T.: Classifying posed and real smiles from observers’ peripheral physiology. In: Proceedings of the 11th EAI International Conference on Pervasive Computing Technologies for Healthcare, pp. 460–463 (2017)

    Google Scholar 

  17. Hossain, M.Z., Gedeon, T.: Discriminating real and posed smiles: human and avatar smiles. Technical report, Brisbane, QLD, Australia, November 2017

    Google Scholar 

  18. Hossain, M.Z., Kabir, M.M., Shahjahan, M.: A robust feature selection system with Colin’s CCA network. Neurocomputing 173, 855–863 (2016)

    Article  Google Scholar 

  19. Islam, A., Ma, J., Gedeon, T., Hossain, M.Z., Liu, Y.H.: Measuring user responses to driving simulators: a galvanic skin response based study. In: 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), pp. 33–337. IEEE (2019)

    Google Scholar 

  20. Jerritta, S., Murugappan, M., Nagarajan, R., Wan, K.: Physiological signals based human emotion recognition: a review. In: 2011 IEEE 7th International Colloquium on Signal Processing and its Applications, pp. 410–415. IEEE (2011)

    Google Scholar 

  21. Koelstra, S., et al.: Deap: a database for emotion analysis; using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2011)

    Article  Google Scholar 

  22. Mehta, D., Siddiqui, M.F.H., Javaid, A.Y.: Facial emotion recognition: a survey and real-world user experiences in mixed reality. Sensors 18(2), 416 (2018)

    Article  Google Scholar 

  23. Mueser, K.T., Grau, B.W., Sussman, S., Rosen, A.J.: You’re only as pretty as you feel: facial expression as a determinant of physical attractiveness. J. Pers. Soc. Psychol. 46(2), 469 (1984)

    Article  Google Scholar 

  24. Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23(10), 1175–1191 (2001)

    Article  Google Scholar 

  25. Pugh, S.D.: Service with a smile: emotional contagion in the service encounter. Acad. Manag. J. 44(5), 1018–1027 (2001)

    Google Scholar 

  26. Samal, A., Iyengar, P.A.: Automatic recognition and analysis of human faces and facial expressions: a survey. Pattern Recogn. 25(1), 65–77 (1992)

    Article  Google Scholar 

  27. Sebe, N., Cohen, I., Gevers, T., Huang, T.S.: Multimodal approaches for emotion recognition: a survey. In: Internet Imaging VI, vol. 5670, pp. 56–67. International Society for Optics and Photonics (2005)

    Google Scholar 

  28. Shu, L., et al.: A review of emotion recognition using physiological signals. Sensors 18(7), 2074 (2018)

    Article  Google Scholar 

  29. Song, T., Lu, G., Yan, J.: Emotion recognition based on physiological signals using convolution neural networks. In: Proceedings of the 2020 12th International Conference on Machine Learning and Computing, pp. 161–165 (2020)

    Google Scholar 

  30. Teichmann, D., Klopp, J., Hallmann, A., Schuett, K., Wolfart, S., Teichmann, M.: Detection of acute periodontal pain from physiological signals. Physiol. Meas. 39(9), 095007 (2018)

    Article  Google Scholar 

  31. Wang, Z., Mao, H., Li, Y.J., Liu, F.: Smile big or not? Effects of smile intensity on perceptions of warmth and competence. J. Consum. Res. 43(5), 787–805 (2017)

    Google Scholar 

  32. Whitehill, J., Littlewort, G., Fasel, I., Bartlett, M., Movellan, J.: Toward practical smile detection. IEEE Trans. Pattern Anal. Mach. Intell. 31(11), 2106–2111 (2009)

    Article  Google Scholar 

  33. Wu, C.H., Lin, J.C., Wei, W.L.: Survey on audiovisual emotion recognition: databases, features, and data fusion strategies. APSIPA Trans. Signal Inf. Process. 3 (2014)

    Google Scholar 

  34. Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 39–58 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The Australian National University (ANU), Canberra, Australia

    Renshang Gao, Atiqul Islam, Tom Gedeon & Md Zakir Hossain

Authors
  1. Renshang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Atiqul Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tom Gedeon
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Md Zakir Hossain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Md Zakir Hossain .

Editor information

Editors and Affiliations

  1. Department of AI, Ping An Life, Shenzhen, China

    Haiqin Yang

  2. Faculty of Information Technology, King Mongkut's Institute of Technology Ladkrabang, Bangkok, Thailand

    Kitsuchart Pasupa

  3. City University of Hong Kong, Kowloon, Hong Kong

    Andrew Chi-Sing Leung

  4. Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong, Hong Kong

    James T. Kwok

  5. School of Information Technology, King Mongkut’s University of Technology Thonburi, Bangkok, Thailand

    Jonathan H. Chan

  6. The Chinese University of Hong Kong, New Territories, Hong Kong

    Irwin King

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gao, R., Islam, A., Gedeon, T., Hossain, M.Z. (2020). Identifying Real and Posed Smiles from Observers’ Galvanic Skin Response and Blood Volume Pulse. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12532. Springer, Cham. https://doi.org/10.1007/978-3-030-63830-6_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63830-6_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63829-0

  • Online ISBN: 978-3-030-63830-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation