Predicting online lecture ratings based on gesturing and vocal behavior | Journal on Multimodal User Interfaces Skip to main content
Springer Nature Link
Log in

Predicting online lecture ratings based on gesturing and vocal behavior

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

Nonverbal behavior plays an important role in any human–human interaction. Teaching—an inherently social activity—is not an exception. So far, the effect of nonverbal behavioral cues accompanying lecture delivery was investigated in the case of traditional ex-cathedra lectures, where students and teachers are co-located. However, it is becoming increasingly more frequent to watch lectures online and, in this new type of setting, it is still unclear what the effect of nonverbal communication is. This article tries to address the problem and proposes experiments performed over the lectures of a popular web repository (“Videolectures”). The results show that automatically extracted nonverbal behavioral cues (prosody, voice quality and gesturing activity) predict the ratings that “Videolectures” users assign to the presentations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. http://www.videolectures.net.

  2. http://www.khanacademy.org.

  3. http://online.stanford.edu/courses.

  4. http://www.youtube.com/yt/press/statistics.html.

  5. As of September 2011.

  6. The list of videos used in the experiment is available at https://pavisdata.iit.it/data/salvagnini/RatingPrediction_VL/RP_VL90v_INFOCOM2013.pdf.

References

  1. Ambady N, Rosenthal R (1992) Thin slices of expressive behavior as predictors of interpersonal consequences: a meta-analysis. Psychol Bull 111(2):256–274

    Article  Google Scholar 

  2. Ambady N, Rosenthal R (1993) Half a minute: predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness. J Pers Soc Psychol 64(3):431–431

    Article  Google Scholar 

  3. Babad E, Bernieri F, Rosenthal R (1991) Students as judges of teachers’ verbal and nonverbal behavior. Am Educ Res J 28(1):211–234

    Article  Google Scholar 

  4. Bazzani L, Cristani M, Tosato D, Farenzena M, Paggetti G, Menegaz G, Murino V (2013) Social interactions by visual focus of attention in a three-dimensional environment. Expert Syst 30(2):115–127

    Article  Google Scholar 

  5. Cristani M, Murino V, Vinciarelli A (2010) Socially intelligent surveillance and monitoring: analysing social dimensions of physical space. In: IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW 2010), pp 51–58

  6. Cristani M, Paggetti G, Vinciarelli A, Bazzani L, Menegaz G, Murino V (2011a) Towards computational proxemics: inferring social relations from interpersonal distances. In: Proceeedings of IEEE international conference on social computing, pp 290–297

  7. Cristani M, Pesarin A, Drioli C, Tavano A, Perina A, Murino V (2011b) Generative modeling and classification of dialogs by a low-level turn-taking feature. Pattern Recogn 44(8):1785–1800

    Article  Google Scholar 

  8. Cristani M, Pesarin A, Vinciarelli A, Crocco M, Murino V (2011c) Look at who’s talking: voice activity detection by automated gesture analysis. In: Am I workshops, pp 72–80

  9. Curhan J, Pentland A (2007) Thin slices of negotiation: predicting outcomes from conversational dynamics within the first 5 minutes. J Appl Psychol 92(3):802–811

    Article  Google Scholar 

  10. Delaherche E, Chetouani M, Mahdhaoui A, Saint-Georges C, Viaux S, Cohen D (2012) Interpersonal synchrony: a survey of evaluation methods across disciplines. IEEE Trans Affect Comput 3(3):349–365

    Article  Google Scholar 

  11. Fini A (2009) The technological dimension of a massive open online course: the case of the cck08 course tools. Int Rev Res Open Dist Learn 10(5):1–20

    Google Scholar 

  12. Harris M, Rosenthal R (2005) No more teachers’ dirty looks: effects of teacher nonverbal behavior on student outcomes. In: Riggio R, Feldman R (eds) Applications of nonverbal communication. Lawrence Erlbaum, London, pp 157–192

    Google Scholar 

  13. Jayagopi D, Hung H, Yeo C, Gatica-Perez D (2009) Modeling dominance in group conversations from non-verbal activity cues. IEEE Trans Audio Speech Lang Process 17(3):501–513

    Article  Google Scholar 

  14. Johnson WL, Rickel J (1997) Steve: an animated pedagogical agent for procedural training in virtual environments. ACM SIGART Bull 8(1–4):16–21

    Article  Google Scholar 

  15. Knapp M, Hall J (1972) Nonverbal communication in human interaction. Harcourt Brace College Publishers, New York

    Google Scholar 

  16. Kop R (2011) The challenges to connectivist learning on open online networks: learning experiences during a Massive Open Online Course. Int Rev Res Open Dist Learn Special Issue-Connect Des Deliv Soc Netw Learn 12(3):19–38

    Google Scholar 

  17. Kop R, Fournier H, Mak J (2011) A pedagogy of abundance or a pedagogy to support human beings? Participant support on massive open online courses. Int Rev Res Open Dist Learn 12(7):74–93

    Google Scholar 

  18. Krämer NC, Bente G (2010) Personalizing e-learning. the social effects of pedagogical agents. Educ Psychol Rev 22(1):71–87

    Article  Google Scholar 

  19. Kuncheva L (2002) A theoretical study on six classifier fusion strategies. IEEE Trans Pattern Anal Mach Intell 24(2):281–286

    Article  Google Scholar 

  20. Liu C (2009) Beyond pixels: exploring new representations and applications for motion analysis. PhD Thesis, Massachusetts Institute of Technology

  21. Mairesse F, Walker MA, Mehl MR, Moore RK (2007) Using linguistic cues for the automatic recognition of personality in conversation and text. J Artif Intell Res 30:457–500

    MATH  Google Scholar 

  22. Martin F (2012) Will Massive Open Online Courses change the way we teach? Commun ACM 55(8):28–30

    Article  Google Scholar 

  23. Mohammadi G, Vinciarelli A (2012) Automatic personality perception: prediction of trait attribution based on prosodic features. IEEE Trans Affect Comput 3(3):273–284

    Article  Google Scholar 

  24. Moran M, Seaman J, Tinti-Kane H (2011) Teaching, learning, and sharing: how today’s higher education faculty use social media. Technical Report, Pearson Education

  25. Morency LP (2010) Modeling human communication dynamics. IEEE Signal Process Mag 27(5):112–116

    Article  Google Scholar 

  26. Nass C, Brave S (2005) Wired for speech: how voice activates and advances the human–computer relationship. MIT Press, Cambridge

    Google Scholar 

  27. Otsuka K, Takemae Y, Yamato J (2005) A probabilistic inference of multiparty-conversation structure based on Markov-switching models of gaze patterns, head directions, and utterances. In: Proceedings of ACM international conference on multimodal interfaces, pp 191–198

  28. Pesarin A, Cristani M, Murino V, Vinciarelli A (2012) Conversation analysis at work: detection of conflict in competitive discussions through semi-automatic turn-organization analysis. Cogn Process 13(2):533–540

    Article  Google Scholar 

  29. Petrillo M, Cutugno F (2003) A syllable segmentation algorithm for English and Italian. In: Proceedings of Eurospeech, pp 2913–2916

  30. Pianesi F, Zancanaro M, Not E, Leonardi C, Falcon V, Lepri B (2008) Multimodal support to group dynamics. Pers Ubiquit Comput 12(3):181–195

    Article  Google Scholar 

  31. Picard R (2000) Affective computing. MIT Press, Cambridge

    Google Scholar 

  32. Reeves B, Nass C (1996) The media equation: how people treat computers, television, and new media like real people and places. Cambridge University Press, Cambridge

    Google Scholar 

  33. Richmond V, McCroskey J (1995) Nonverbal behaviors in interpersonal relations. Allyn and Bacon, Newton

  34. Rienks R, Heylen D (2006) Dominance detection in meetings using easily obtainable features. In: Lecture notes in computer science, Springer, vol 3869, pp 76–86

  35. Rienks R, Zhang D, Gatica-Perez D (2006) Detection and application of influence rankings in small group meetings. In: Proceedings of the international conference on multimodal interfaces, pp 257–264

  36. Rodriguez T, Plax TG, Kearney P (1996) Clarifying the relationship between teacher nonverbal immediacy and student cognitive learning: affective learning as the central causal mediator. Commun Educ 45(4):293–305

    Google Scholar 

  37. Saint-Georges C, Cassel R, Cohen D, Chetouani M, Laznik MC, Maestro S, Muratori F (2010) What studies of family home movies can teach us about autistic infants: a literature review. Res Autism Spect Disord 4(3):355–366

    Article  Google Scholar 

  38. Salamin H, Vinciarelli A (2012) Automatic role recognition in multiparty conversations: an approach based on turn organization, prosody, and conditional random fields. IEEE Trans Multimedia 14(2):338–345

    Article  Google Scholar 

  39. Salvagnini P, Salamin H, Cristani M, Vinciarelli A, Murino V (2012) Learning how to teach from “videolectures”: automatic prediction of lecture ratings based on teacher’s nonverbal behavior. In: Proceedings of IEEE international conference on cognitive infocommunications, pp 415–419

  40. Shaw E, Johnson WL, Ganeshan R (1999) Pedagogical agents on the web. In: Proceedings of the third annual conference on Autonomous Agents, ACM, pp 283–290

  41. Smith H (1979) Nonverbal communication in teaching. Rev Educ Res 49(4):631–672

    Article  Google Scholar 

  42. Tallent-Runnels M, Thomas J, Lan W, Cooper S, Ahern T, Shaw S, Liu X (2006) Teaching courses online: a review of the research. Rev Educ Res 76(1):93–135

    Article  Google Scholar 

  43. Thalmann NM, Thalmann D (2012) Virtual humans: back to the future. In: Proceedings of graphics interface, pp 1–8

  44. Vinciarelli A, Pantic M, Heylen D, Pelachaud C, Poggi I, D’Errico F, Schroeder M (2012) Bridging the gap between social animal and unsocial machine: a survey of social signal processing. IEEE Trans Affect Comput 3(1):69–87

    Article  Google Scholar 

  45. Wharton T (2009) The pragmatics of non-verbal communication. Cambridge University Press, Cambridge

    Book  Google Scholar 

  46. Wiggins J (ed) (1996) The five-factor model of personality. Guilford, New York

    Google Scholar 

  47. Woolfolk AE, Brooks D (1983) Nonverbal communication in teaching. Rev Res Educ 10:103–149

    Google Scholar 

  48. Yang Y, Ramanan D (2011) Articulated pose estimation with flexible mixtures-of-parts. In: IEEE conference on computer vision and pattern recognition, pp 1385–1392

Download references

Acknowledgments

The research that has led to this work has been supported in part by the European Community Seventh Framework Programme (FP7/2007–2013), under Grant Agreement No. 231287 (SSPNet). Dr. Cheng was supported by the Hankuk University of Foreign Studies Research Fund of 2013.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Hankuk University of Foreign Studies, Seoul, South Korea

    Dong Seon Cheng

  2. School of Computing Science, University of Glasgow, Glasgow, UK

    Hugues Salamin & Alessandro Vinciarelli

  3. Idiap Research Institute, Martigny, Switzerland

    Alessandro Vinciarelli

  4. Pattern Analysis and Computer Vision (PAVIS), Istituto Italiano di Tecnologia, Via Morego 30, 16163 , Genova, Italy

    Pietro Salvagnini & Vittorio Murino

  5. Department of Computer Science, University of Verona, Strada Le Grazie 15, 37134 , Verona, Italy

    Marco Cristani

Authors
  1. Dong Seon Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hugues Salamin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pietro Salvagnini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marco Cristani
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alessandro Vinciarelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Vittorio Murino
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Dong Seon Cheng or Marco Cristani.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cheng, D.S., Salamin, H., Salvagnini, P. et al. Predicting online lecture ratings based on gesturing and vocal behavior. J Multimodal User Interfaces 8, 151–160 (2014). https://doi.org/10.1007/s12193-013-0142-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-013-0142-z

Keywords

Search

Navigation