Challenges in Evaluating Players’ Interaction with Digital Games | SpringerLink
Skip to main content
Springer Nature Link
Log in

Challenges in Evaluating Players’ Interaction with Digital Games

  • Conference paper
  • First Online:
Grand Research Challenges in Games and Entertainment Computing in Brazil - GranDGamesBR 2020–2030 (GranDGamesBR 2020, GranDGamesBR 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1702))

Abstract

Evaluating the quality of the player’s interaction with a digital game considering human factors is not a trivial task. In addition to considering technical issues, it is necessary to explore the player’s perspective carefully, focusing on their satisfaction, motivation, and expectations regarding the game. Thus, human factors evaluation in games raises new challenges for academia and industry, such as defining the factors to be explored, the appropriate choice of methods to be applied, the use, translation, and validation of attitude scales, and the availability of resources, among others. The pursuit of the maturation and systematization of studies and practices related to evaluating the player’s interaction and experience is an international challenge, which has moved the efforts of research and practice communities but is still commonly neglected in the Brazilian context. Thus, it is necessary to encourage the development of national studies and initiatives in this area to develop better-grounded practices suited to our cultural context and the needs of the Brazilian market.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 8579
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 10724
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://sigchi.org/conferences/conference-history/chi/.

  2. 2.

    Search updated in April 2022.

  3. 3.

    https://sigchi.org/conferences/conference-history/chiplay/.

References

  1. Joselli, M.: A ética e os games: “morais, imorais ou amorais?”. Um estudo sobre a ética em games segundo Aristóteles, Santo Agostinho e Kant. Proceedings of XIII Brazilian Symposium on Games and Digital Enterteinment (SBGAMES‘14), 448–456 (2014)

    Google Scholar 

  2. Abeele, V.V., Spiel, K., Nacke, L., Johnson, D., Gerling, K.: Development and validation of the player experience inventory: a scale to measure player experiences at the level of functional and psychosocial consequences. Int. J. Hum. Comput. Stud. 135, 102370 (2020)

    Article  Google Scholar 

  3. Adams, A., Cox, A.L.: Questionnaires, In-Depth Interviews and Focus Groups. Cambridge University Press, Cambridge (2008)

    Book  Google Scholar 

  4. Amelio, C.: A indústria e o mercado de jogos digitais no brasil: evolução, características, obstáculos e análise comparativa. Trabalho de Conclusão de Curso. Universidade Federal de Minas Gerais (2018)

    Google Scholar 

  5. Aranha, R.V., Nunes, F.L.: Player experience with Brazilian accent: development and validation of PX-BR, a summarized instrument in Portuguese. In: XVIII Brazilian Symposium on Information Systems, pp. 1–8 (2022)

    Google Scholar 

  6. IE Association: Definition and domains of human factors and ergonomics (2020). https://iea.cc/what-is-ergonomics/. Accessed 17 Apr 2022

  7. Barbosa, S.D.J., da Silva, B., Silveira, M.S., Gasparini, I., Darin, T., Barbosa, G.D.J.: Interação humano-computador e experiência do usuario. Auto publicação (2021)

    Google Scholar 

  8. Bernhaupt, R.: Game User Experience Evaluation. Springer, Heidelberg (2015)

    Book  Google Scholar 

  9. Bernhaupt, R., Mueller, F.F.: Game user experience evaluation. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, pp. 940–943 (2016)

    Google Scholar 

  10. Biduski, D., Bellei, E.A., Rodriguez, J.P.M., Zaina, L.A.M., De Marchi, A.C.B.: Assessing long-term user experience on a mobile health application through an in-app embedded conversation-based questionnaire. Comput. Hum. Behav. 104, 106169 (2020)

    Article  Google Scholar 

  11. Blizzard, A.: Social connection in a virtual world (2021). https://www.activisionblizzard.com/newsroom/2021/10/social-connection-in-a-virtual-world. Accessed 20 Dec 2021

  12. Borges, J.B., Juy, C.L., de Andrade Matos, I.S., Silveira, P.V.A., Darin, T.D.G.R.: Player experience evaluation: a brief panorama of instruments and research opportunities. J. Interact. Syst. 11(1), 74–91 (2020)

    Google Scholar 

  13. Bourgonjon, J.: The meaning and relevance of video game literacy. CLCWeb: Comp. Lit. Cult. 16(5), 8 (2014)

    Google Scholar 

  14. Brockmyer, J.H., Fox, C.M., Curtiss, K.A., McBroom, E., Burkhart, K.M., Pidruzny, J.N.: The development of the game engagement questionnaire: a measure of engagement in video game-playing. J. Exp. Soc. Psychol. 45(4), 624–634 (2009)

    Article  Google Scholar 

  15. Brühlmann, F., Schmid, G.M.: How to measure the game experience? Analysis of the factor structure of two questionnaires. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, pp. 1181–1186 (2015)

    Google Scholar 

  16. Carneiro, N., Darin, T., Viana, W.: What are we talking about when we talk about location-based games evaluation? A systematic mapping study. In: Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems, pp. 1–13 (2019)

    Google Scholar 

  17. Carneiro, N., Melo, B., Cavalcante, L., Castro, R., Andrade, R.M., Darin, T.: Mobili: development and use of a usability checklist for mobile games and applications. In: 19th Brazilian Symposium on Software Quality, pp. 1–9 (2020)

    Google Scholar 

  18. Carneiro, N., Miranda, D., Pereira, G., Mendonça, G., Darin, T.: A systematic mapping on player’s profiles: motivations, behavior, and personality characteristics. J. Interact. Syst. 13(1), 257–273 (2022). https://doi.org/10.5753/jis.2022.2572. https://sol.sbc.org.br/journals/index.php/jis/article/view/2572

  19. Carneiro, N., Viana, W., Darin, T.: VALERIE: a guide to qualitative evaluation of player experience in location-based games using interviews. In: Proceedings of the XX Brazilian Symposium on Human Factors in Computing Systems, pp. 1–7 (2021)

    Google Scholar 

  20. Charlton, J.P., Danforth, I.D.: Validating the distinction between computer addiction and engagement: online game playing and personality. Behav. Inf. Technol. 29(6), 601–613 (2010)

    Article  Google Scholar 

  21. Cho, M., Lee, S., Lee, K.P.: Once a kind friend is now a thing: understanding how conversational agents at home are forgotten. In: Proceedings of the 2019 on Designing Interactive Systems Conference, pp. 1557–1569 (2019)

    Google Scholar 

  22. Coelho, B., Andrade, R.M.C., Darin, T.: Not the same everywhere: comparing the scope and definition of user experience between the Brazilian and international communities. Int. J. Hum.-Comput. Interact. 38(7), 595–613 (2022). https://doi.org/10.1080/10447318.2021.1960727

    Article  Google Scholar 

  23. Creswell, J.W., Creswell, J.D.: Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage Publications (2018)

    Google Scholar 

  24. Darin, T., Andrade, R., Sánchez, J.: Principles for evaluating usability in multimodal games for people who are blind. In: Antona, M., Stephanidis, C. (eds.) HCII 2019. LNCS, vol. 11573, pp. 209–223. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23563-5_18

    Chapter  Google Scholar 

  25. Darin, T., Carneiro, N.: Avaliacao de fatores humanos na interacao com jogos digitais. In: 2020 19th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames). SBC-Proceedings of SBGames, pp. 2179–2259 (2020)

    Google Scholar 

  26. Darin, T., Coelho, B., Borges, B.: Which instrument should I use? Supporting decision-making about the evaluation of user experience. In: Marcus, A., Wang, W. (eds.) HCII 2019. LNCS, vol. 11586, pp. 49–67. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23535-2_4

    Chapter  Google Scholar 

  27. Darin, T., Andrade, R., Sánchez, J.: Usability evaluation of multimodal interactive virtual environments for learners who are blind: An empirical investigation. Int. J. Hum Comput Stud. 158, 102732 (2022)

    Google Scholar 

  28. Deci, E.L., Ryan, R.M.: Hedonia, eudaimonia, and well-being: an introduction. J. Happiness Stud. 9(1), 1–11 (2008)

    Article  Google Scholar 

  29. Deci, E.L., Ryan, R.M.: Self-determination theory (2012)

    Google Scholar 

  30. Drachen, A., Mirza-Babaei, P., Nacke, L.E.: Games User Research. Oxford University Press, Oxford (2018)

    Book  Google Scholar 

  31. Facebook IQ, M: Strategic considerations for mobile gaming engagement campaigns (2019). https://www.facebook.com/business/news/insights/strategic-considerations-for-mobile-gaming-engagement-campaigns. Accessed 20 Dec 2021

  32. Fuentes, C., Herskovic, V., Rodríguez, I., Gerea, C., Marques, M., Rossel, P.O.: A systematic literature review about technologies for self-reporting emotional information. J. Ambient. Intell. Humaniz. Comput. 8(4), 593–606 (2017)

    Article  Google Scholar 

  33. Garg, R.: An analysis of (non-)use practices and decisions of internet of things. In: Lamas, D., Loizides, F., Nacke, L., Petrie, H., Winckler, M., Zaphiris, P. (eds.) INTERACT 2019. LNCS, vol. 11749, pp. 3–24. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-29390-1_1

    Chapter  Google Scholar 

  34. Gerling, K.M., Klauser, M., Niesenhaus, J.: Measuring the impact of game controllers on player experience in FPS games. In: Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, pp. 83–86 (2011)

    Google Scholar 

  35. Gerritsen, M., et al.: English in product advertisements in non-English-speaking countries in Western Europe: product image and comprehension of the text. J. Glob. Mark. 23(4), 349–365 (2010)

    Article  Google Scholar 

  36. Google: Change the game: Driving inclusivity and belonging in gaming (2019). https://static.googleusercontent.com/media/play.google.com/pt-BR//about/changethegame/assets/static/pdf/white-paper-2019.pdf. Accessed 20 Dec 2021

  37. Harrison, D., Marshall, P., Berthouze, N., Bird, J.: Tracking physical activity: problems related to running longitudinal studies with commercial devices. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, pp. 699–702 (2014)

    Google Scholar 

  38. Huizinga, J.: Homo Ludens. Reprint (1944)

    Google Scholar 

  39. Irizar-Arrieta, A., Casado-Mansilla, D., Garaizar, P., López-de Ipiña, D., Retegi, A.: User perspectives in the design of interactive everyday objects for sustainable behaviour. Int. J. Hum. Comput. Stud. 137, 102393 (2020)

    Article  Google Scholar 

  40. Karapanos, E., Martens, J.B., Hassenzahl, M.: Reconstructing experiences with iScale. Int. J. Hum. Comput. Stud. 70(11), 849–865 (2012)

    Article  Google Scholar 

  41. Karapanos, E., Zimmerman, J., Forlizzi, J., Martens, J.B.: User experience over time: an initial framework. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 729–738 (2009)

    Google Scholar 

  42. Kjærup, M., Skov, M.B., Nielsen, P.A., Kjeldskov, J., Gerken, J., Reiterer, H.: Longitudinal studies in HCI research: a review of CHI publications from 1982–2019. In: Karapanos, E., Gerken, J., Kjeldskov, J., Skov, M.B. (eds.) Advances in Longitudinal HCI Research. HIS, pp. 11–39. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67322-2_2

    Chapter  Google Scholar 

  43. Klarkowski, M.W.: The psychophysiological evaluation of the player experience. Ph.D. thesis, Queensland University of Technology (2017)

    Google Scholar 

  44. Koch, M., von Luck, K., Schwarzer, J., Draheim, S.: The novelty effect in large display deployments-experiences and lessons-learned for evaluating prototypes. In: Proceedings of 16th European Conference on Computer-Supported Cooperative Work-Exploratory Papers. European Society for Socially Embedded Technologies (EUSSET) (2018)

    Google Scholar 

  45. Kou, Y., Li, Y., Gui, X., Suzuki-Gill, E.: Playing with streakiness in online games: how players perceive and react to winning and losing streaks in league of legends. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2018)

    Google Scholar 

  46. Kuittinen, J., Kultima, A., Niemelä, J., Paavilainen, J.: Casual games discussion. In: Proceedings of the 2007 Conference on Future Play, pp. 105–112 (2007)

    Google Scholar 

  47. Kujala, S., Miron-Shatz, T., Jokinen, J.J.: The cross-sequential approach: a short-term method for studying long-term user experience. J. Usab. Stud. 14(2) (2019)

    Google Scholar 

  48. Lankes, M., Bernhaupt, R., Tscheligi, M.: Evaluating user experience factors using experiments: expressive artificial faces embedded in contexts. In: Bernhaupt, R. (ed.) Game User Experience Evaluation. HIS, pp. 113–131. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-15985-0_6

    Chapter  Google Scholar 

  49. Laurel, B.: Computers as Theatre. Addison-Wesley, Boston (2013)

    Google Scholar 

  50. Lazar, J., Feng, J.H., Hochheiser, H.: Research Methods in Human-Computer Interaction. Morgan Kaufmann, Burlington (2017)

    Google Scholar 

  51. Lazzaro, N.: Why we play: affect and the fun of games. Hum.-Comput. Interact. Designing Diverse Users Domains 155, 679–700 (2009)

    Google Scholar 

  52. Lehmann, J., Lalmas, M., Yom-Tov, E., Dupret, G.: Models of user engagement. In: Masthoff, J., Mobasher, B., Desmarais, M.C., Nkambou, R. (eds.) UMAP 2012. LNCS, vol. 7379, pp. 164–175. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31454-4_14

    Chapter  Google Scholar 

  53. MacKenzie, I.S.: Human-computer interaction: an empirical research perspective (2012)

    Google Scholar 

  54. Maia, C.L.B., Furtado, E.S.: A study about psychophysiological measures in user experience monitoring and evaluation. In: Proceedings of the 15th Brazilian Symposium on Human Factors in Computing Systems, pp. 1–9 (2016)

    Google Scholar 

  55. Mekler, E.D., Bopp, J.A., Tuch, A.N., Opwis, K.: A systematic review of quantitative studies on the enjoyment of digital entertainment games. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 927–936 (2014)

    Google Scholar 

  56. Michael, D.R., Chen, S.L.: Serious Games: Games That Educate, Train, and Inform. Muska & Lipman/Premier-Trade (2005)

    Google Scholar 

  57. Miranda, D., Li, C., Darin, T.: UES-BR: translation and cross-cultural adaptation of the user engagement scale for Brazilian Portuguese. Proc. ACM Hum.-Comput. Interact. 5(CHI PLAY), 1–22 (2021)

    Google Scholar 

  58. Miranda, D.M., Pontes, R.M., de Gois Ribeiro Darin, T.: It’s dark but just a game: towards an ethical and healthy game design practice. In: Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems, IHC 2022. Association for Computing Machinery, New York (2022). https://doi.org/10.1145/3554364.3559144

  59. Montero, H., Krawczyk, P., Topolewski, M., Pallot, M., Huotari, J., Lehtosaari, H.: Repeated cross-sectional study of a mobile app user-experience. In: 2020 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC), pp. 1–8. IEEE (2020)

    Google Scholar 

  60. Nacke, L., Drachen, A.: Towards a framework of player experience research. In: Proceedings of the Second International Workshop on Evaluating Player Experience in Games at FDG, vol. 11 (2011)

    Google Scholar 

  61. Nacke, L., et al.: Playability and player experience research. In: Proceedings of DIGRA 2009: Breaking New Ground: Innovation in Games, Play, Practice and Theory. DiGRA (2009)

    Google Scholar 

  62. Nacke, L.E., Bateman, C., Mandryk, R.L.: BrainHex: a neurobiological gamer typology survey. Entertainment Comput. 5(1), 55–62 (2014)

    Article  Google Scholar 

  63. Oliver, M.B., Raney, A.A.: Entertainment as pleasurable and meaningful: identifying hedonic and eudaimonic motivations for entertainment consumption. J. Commun. 61(5), 984–1004 (2011)

    Article  Google Scholar 

  64. World Health Organization: Addictive behaviours: Gaming disorder (2020). https://www.who.int/news-room/questions-and-answers/item/addictive-behaviours-gaming-disorder. Accessed 20 Dec 2021

  65. Peters, D., Calvo, R.A., Ryan, R.M.: Designing for motivation, engagement and wellbeing in digital experience. Front. Psychol. 9, 797 (2018)

    Article  Google Scholar 

  66. Phan, M.H., Keebler, J.R., Chaparro, B.S.: The development and validation of the game user experience satisfaction scale (guess). Hum. Factors 58(8), 1217–1247 (2016)

    Article  Google Scholar 

  67. Poels, K., de Kort, Y.A., IJsselsteijn, W.A.: D3.3: game experience questionnaire: development of a self-report measure to assess the psychological impact of digital games (2007)

    Google Scholar 

  68. Regani, S.D., Xu, Q., Wang, B., Wu, M., Liu, K.R.: Driver authentication for smart car using wireless sensing. IEEE Internet Things J. 7(3), 2235–2246 (2019)

    Article  Google Scholar 

  69. Roto, V., Law, E.C., Vermeeren, A.P., Hoonhout, J.: User experience white paper: bringing clarity to the concept of user experience (2011)

    Google Scholar 

  70. Ryan, R.M.: The Oxford Handbook of Human Motivation. OUP, USA (2012)

    Google Scholar 

  71. Ryan, R.M., Deci, E.L.: Self-determination Theory: Basic Psychological Needs in Motivation, Development, and Wellness. Guilford Publications (2017)

    Google Scholar 

  72. Sakuda, L.O., Fortim, I.: Ii censo da indústria brasileira de jogos digitais. Ministério da Cultura, Brasília (2018)

    Google Scholar 

  73. Salvendy, G.: Handbook of Human Factors and Ergonomics. Wiley, Hoboken (2012)

    Book  Google Scholar 

  74. Sánchez, J.L.G., Vela, F.L.G., Simarro, F.M., Padilla-Zea, N.: Playability: analysing user experience in video games. Behav. Inf. Technol. 31(10), 1033–1054 (2012)

    Article  Google Scholar 

  75. Sauro, J., Lewis, J.R.: Quantifying the User Experience: Practical Statistics for User Research. Morgan Kaufmann (2016)

    Google Scholar 

  76. Susi, T., Johannesson, M., Backlund, P.: Serious games: an overview (2007)

    Google Scholar 

  77. Tyack, A., Mekler, E.D.: Self-determination theory in HCI games research: current uses and open questions. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–22 (2020)

    Google Scholar 

  78. Van Laerhoven, K., et al.: Experiences from a wearable-mobile acquisition system for ambulatory assessment of diet and activity. In: Proceedings of the 4th International Workshop on Sensor-Based Activity Recognition and Interaction, pp. 1–8 (2017)

    Google Scholar 

  79. von Wangenheim, C.G., Petri, G., Borgatto, A.F.: MEEGA+ KIDS: a model for the evaluation of games for computing education in secondary school. RENOTE 18(1) (2020)

    Google Scholar 

  80. Wasserman, T., Wasserman, L.: Motivation, Effort, and the Neural Network Model. Springer, Heidelberg (2020)

    Book  Google Scholar 

  81. Wiemeyer, J., Nacke, L., Moser, C., et al.: Player experience. In: Dörner, R., Göbel, S., Effelsberg, W., Wiemeyer, J. (eds.) Serious Games, pp. 243–271. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-40612-1_9

    Chapter  Google Scholar 

  82. Yang, S., et al.: IoT structured long-term wearable social sensing for mental wellbeing. IEEE Internet Things J. 6(2), 3652–3662 (2018)

    Article  Google Scholar 

  83. Yang, W., Rifqi, M., Marsala, C., Pinna, A.: Towards better understanding of player’s game experience. In: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, pp. 442–449 (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Federal University of Ceara, Fortaleza, CE, Brazil

    Ticianne Darin, Nayana Carneiro, David Miranda & Bianca Coelho

Authors
  1. Ticianne Darin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nayana Carneiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Miranda
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bianca Coelho
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ticianne Darin .

Editor information

Editors and Affiliations

  1. UNIRIO, Rio de Janeiro, Brazil

    Rodrigo Pereira dos Santos

  2. UDESC, Joinville, Brazil

    Marcelo da Silva Hounsell

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Darin, T., Carneiro, N., Miranda, D., Coelho, B. (2023). Challenges in Evaluating Players’ Interaction with Digital Games. In: Santos, R.P.d., Hounsell, M.d.S. (eds) Grand Research Challenges in Games and Entertainment Computing in Brazil - GranDGamesBR 2020–2030. GranDGamesBR GranDGamesBR 2020 2021. Communications in Computer and Information Science, vol 1702. Springer, Cham. https://doi.org/10.1007/978-3-031-27639-2_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-27639-2_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-27638-5

  • Online ISBN: 978-3-031-27639-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation