Abstract
We present the results of an exploratory player experience study on the game Quantum Moves, a citizen science game where players move quantum particles to help create a quantum computer. Eight-hundred-and-seventeen players responded to a 13-question exploratory survey constructed to understand how players relate to the game, what are their motivations, and how could the game be improved. We analyzed the data using descriptive statistics and thematic analysis. Specifically, the thematic analysis helped identifying two cross-cutting themes amongst the players: (a) learning and (b) the opportunity to contribute to science. Results indicate that the opportunity to help science, along with game design, game elements, involvement of players with the scientific community, and players’ strategies influence experience. Implications of the particular findings for the research on player experience on citizen science games and development of evaluation methods are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
We searched the databases Scopus and Web of Science. Both databases are well-established, multi-disciplinary research platforms, including a wide variety of peer-reviewed journals, and they are being updated regularly. We chose these two databases to ensure relevant papers are included, since it is possible that one database omits relevant research. The Scopus database retrieved 11 articles when searching for TITLE-ABS-KEY (“citizen science” AND “gam*” AND “player experience”). The oldest article dates from 2014. No articles were found when searching (TITLE-ABS-KEY (“citizen science” AND “gam*”)) AND (“quality of experience”). The Web of Science returned one article from 2015 with a similar search string: TOPIC: (“citizen science”) AND TOPIC: (gam*) AND TOPIC: (“player experience*”) and no results when searching TOPIC: (“citizen science”) AND TOPIC: (gam*) AND TOPIC: (“player experience”). Search conducted in 4th September, 2019.
References
Bonney R, Shirk J, Phillips T et al (2014) Next steps for citizen science. Science 343:1436–1437. https://doi.org/10.1126/science.1251554
Tinati R, Luczak-Roesch M, Simperl E, Hall W (2017) An investigation of player motivations in Eyewire, a gamified citizen science project. Comput Hum Behav 73:527–540. https://doi.org/10.1016/j.chb.2016.12.074
Newman G, Wiggins A, Crall A et al (2012) The future of citizen science: emerging technologies and shifting paradigms. Front Ecol Environ 10:298–304. https://doi.org/10.1890/110294
Sørensen JJ, Pedersen MK, Munch M et al (2016) Exploring the quantum speed limit with computer games. Nature 532:210–218
Magnussen R (2017) Involving lay people in research and professional development through gaming: a systematic mapping review. In: Proceedings of the 11th European conference on games based learning, ECGBL 2017, pp 394–401
Weitenberg C, Kuhr S, Mølmer K, Sherson J (2011) A quantum computation architecture using optical tweezers. Phys Rev A. https://doi.org/10.1103/PhysRevA.84.032322
Wiemeyer J, Nacke L, Moser C, Floyd’ Mueller F (2016) Player experience. In: Dörner R, Göbel S, Effelsberg W, Wiemeyer J (eds) Serious games. Foundations, concepts and practice. Springer, Cham
Möller S, Raake A (2014) Quality and quality of experience. Advanced concepts, applications and methods. In: Möller S, Raake A (eds) quality of experience. Springer, Berlin
Canossa A, Drachen A (2009) Patterns of play: play-personas in user-centred game development. In: Proceedings of DiGRA 2009. DiGRA
Klimmt C (2003) Dimensions and determinants of the enjoyment of playing digital games: a three-level model. In: Proceedings of level up - 1st international digital games research conference 2003, Netherlands
Csikszentmihalyi M (1990) Flow. The psychology of optimal experience. Harper & Row, Manhattan
Sweetser P, Wyeth P (2005) GameFlow: a model for evaluating player enjoyment in games. Comput Entertain CIE Theor Pract Comput Appl Entertain 3:3. https://doi.org/10.1145/1077246.1077253
Brown E, Cairns P (2004) A grounded investigation of game immersion. In: CHI’04 extended abstracts on human factors in computing systems. ACM, Austria, pp 24–29
Klimmt C, Hartmann T, Frey A (2007) Effectance and control as determinants of video game enjoyment. Cyberpsychol Behav 10:845–848. https://doi.org/10.1089/cpb.2007.9942
Poels K, de Kort Y, Ijsselsteijn W (2012) Identification and categorization of digital game experiences: a qualitative study integrating theoretical insights and player perspectives. Westminst Pap 9:107–129. https://doi.org/10.16997/wpcc.153
Le Callet P, Möller S, Perkis A (series ed) (2013) Qualinet white paper on definitions of quality of experience (QoE) and related concepts [white paper]. Retrieved from http://www.qualinet.eu/
Nacke L, Drachen A, Göbel S (2010) Methods for evaluating gameplay experience in a serious gaming context. Int J Comput Sci Sport 9:40–51
Law E, von Ahn L (2011) Human computation. Morgan & Claypool, San Rafael
Deterding S, Sicart M, Nacke L, et al (2011) Gamification: using game design elements in non-gaming contexts. In: CHI’11 extended abstracts on human factors in computing systems. ACM Press, Toronto, pp 2425–2428
Ponti M, Hillman T, Kullenberg C, Kasperowski D (2018) Getting it right or being top rank: games in citizen science. Citiz Sci Theory Pract 3:1–12. https://doi.org/10.5334/cstp.101
Bowser A, Hansen D, He Y, et al (2013) Using gamification to inspire new citizen science volunteers. In: Proceedings of the companion publication of the 17th ACM conference on computer supported cooperative work and social computing. ACM Press, Toronto, pp 18–25
Curtis V (2016) Motivation to participate in an online citizen science game: a study of Foldit. Sci Commun 37:723–746. https://doi.org/10.1177/1075547015609322
Greenhill A, Holmes K, Woodcock J et al (2016) Playing with science: exploring how game activity motivates users participation on an online citizen science platform. Aslib J Inf Manag 68:306–325. https://doi.org/10.1108/AJIM-11-2015-0182
Iacovides I, Jennet C, Cornish-Trestrail C, Cox A (2013) Do games attract or sustain engagement in citizen science? A study of volunteer motivations. In: CHI’13 extended abstracts on human factors in computing systems. ACM Press, Paris, pp 1101–1106
Cox A, Cairns P, Shah P, Carroll M (2012) Not doing but thinking: the role of challenge in the gaming experience. In: Proceedings of CHI’12 SIGCHI conference on human factors in computing systems. ACM, Texas, pp 79–88
Prestopnik N, Tang J (2015) Points, stories, worlds, and diegesis: comparing player experiences in two citizen science games. Comput Hum Behav 52:492–506. https://doi.org/10.1016/j.chb.2015.05.051
Hess M, Wiemeyer J, Hamacher K, Goesele M (2014) Serious games for solving protein sequence alignments—combining citizen science and gaming. In: Göbel S, Wiemeyer J (eds) Games for training, education, health and sports. Springer, Berlin
Pedersen MK, Rasmussen N, Sherson J, Basaiawmoit R (2017) Leaderboard effects on player performance in a citizen science game. In: Proceedings of the 11th European conference on games based learning
Lieberoth A, Pedersen MK, Marin A et al (2014) Getting humans to do quantum optimization: user acquisition, engagement and early results from the citizen cyberscience game quantum moves. Hum Comput 1:219–244. https://doi.org/10.15346/hc.v1i2.11
Directorate-General for Research Science, Economy and Society (2010) European textbook on ethics in research. Publication office of the European Union, Luxembourg
González Mariño JC, Cantú Gallegos ML, Camacho-Cruz HE (2018) Redesigning the Bartle test of gamer psychology for its application in gamification processes of learning. In: Proceedings of the 12th international multi-conference on society, cybernetics and informatics. IIIS, Florida, pp 35–40
Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3:77–101. https://doi.org/10.1191/1478088706qp063oa
Campbell J, Quincy C, Osserman J, Pedersen O (2013) Coding in-depth semistructured interviews: problems of unitization and intercoder reliability and agreement. Sociol Methods Res 42:294–320. https://doi.org/10.1177/0049124113500475smr.sagepub.com
Eccleston P, Werneke U, Armon K et al (2001) Accounting for overlap? An application of Mezzich’s kappa statistic to test interrater reliability of interview data on parental accident and emergency attendance. J Adv Nurs 33:784–790. https://doi.org/10.1046/j.1365-2648.2001.01718.x
Stokke A (2015) What to do about Canada’s declining math scores. C. D. Howe Institute Commentary. https://doi.org/10.2139/ssrn.2613146
Dean D, Kuhn D (2006) Direct instruction vs. discovery: the long view. Science Education 91(3):384–397. https://doi.org/10.1002/sce.20194
Tinati R, van Kleek M, Simperl E, et al (2015) Designing for citizen data analysis: a cross-sectional case study of a multi-domain citizen science platform. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems. ACM, Seoul, pp 4069–4078
Habgood J, Overmars M (2006) The game maker’s apprentice. Game development for beginners. Apress, Berkeley
Seaborn K, Fels D (2014) Gamification in theory and action: a survey. Int J Hum-Comput Stud 74:14–31. https://doi.org/10.1016/j.ijhcs.2014.09.006
Franzoni C, Sauermann H (2014) Crowd science: the organization of scientific research in open collaborative projects. Res Policy 43:1–20. https://doi.org/10.1016/j.respol.2013.07.005
Elias G, Garfield R, Gutschera R et al (2012) Characteristics of games. MIT Press, Cambridge
Azwihangwisi M (2015) Piaget’s theory of human development and education. Int Encycl Soc Behav Sci 18:125–132
Horstman T, Chen M (2012) Gamers as scientists? The relationship between participating in foldit play and doing science. In: Presented at the American educational research association (AERA), Canada. Retrieved from https://www.researchgate.net/publication/258294401_Gamers_as_Scientists_The_Relationship_Between_Participating_in_Foldit_Play_and_Doing_Science
Jennett C, Kloetzer L, Schneider D et al (2016) Motivations, learning and creativity in online citizen science. J Sci Commun 15:1–23
Lafourcade M, Joubert A, Le Brun N (2015) Games with a purpose (GWAPs). Wiley, New York
Prestopnik N, Crowston K, Wang J (2017) Gamers, citizen scientists, and data: exploring participant contributions in two games with a purpose. Comput Hum Behav 68:254–268. https://doi.org/10.1016/j.chb.2016.11.035
Skarlatidou A, Ponti M, Sprinks J et al (2019) User experience of digital technologies in citizen science. J Sci Commun. https://doi.org/10.22323/2.18010501
Brockmyer J, Fox C, Curtiss K et al (2009) The development of the game engagement questionnaire: a measure of engagement in video game-playing. J Exp Soc Psychol 45:624–634. https://doi.org/10.1016/j.jesp.2009.02.016
Johnson D, Gardner J, Perry R (2018) Validation of two game experience scales: the player experience of need satisfaction (PENS) and game experience questionnaire (GEQ). Int J Hum-Comput Stud 118:38–46. https://doi.org/10.1016/j.ijhcs.2018.05.003
Preece J (2016) Citizen science: new research challenges for human–computer interaction. Int J Hum-Comput Interact 32:585–612. https://doi.org/10.1080/10447318.2016.1194153
Kim S, Robson C, Zimmerman T, et al (2011) Creek watch: pairing usefulness and usability for successful citizen science. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI’11). ACM Press, New York, pp 2125–2134
Acknowledgements
This work was supported through funding from Marianne and Marcus Wallenberg Grant No. 2013.0020, Lundbeck Foundation, Grant No. R139-2012-12633, Carlsberg Foundation Grant No. CF18-0019, and John Templeton Foundation Grant No. 60969. J.F.S. Acknowledges funding from the ERC, H2020 Grant No. 639560 (MECTRL) and the Synakos foundation.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1
Distribution of players according to the player types as assessed by the Bartle Test (Q20). NA stands for the players who could not take the test. The total sum of players per category is more than the players who answer the question; this is because several players reported that they belong to more than one category.
Appendix 2
Coding schema of player responses to question Q19
Topics and subtopics (indented) | Definition |
|---|---|
Not defined—no response | Players did not answer the question, or their answers were irrelevant |
Game type | Players use a name that defines the game as a genre and explain why they like it or not |
Casual game | Players state the game is a casual game or they play it casually |
Zen game | Players state the game is a Zen game or having calming properties |
Puzzle | Players state the game is a puzzle |
Single player game | Players state the game is a single-player game |
Distinct game | Players state the game is different from other games on the market and/or other citizen science games |
Educational effects | Players state the game has educational effects, or it is good for learning/teaching science. |
Interest in learning | Players state the game taught them something about science |
Generating curiosity | Players state the game generated curiosity for them to learn more |
Engagement | Assigned to indicate the way players engaged with the game |
Enjoyment | Players state they have enjoyed the game |
Monotony | Players state the game is monotonous or repetitive |
Boredom | Players state the game is boring |
Frustration | Players state the game is frustrating, either for its mechanics or for problems with the game build |
Challenge | Players state the game is challenging, either in a positive or a negative way |
Confidence | Players state they are confident when solving the problems proposed by the game |
Modes of learning | Applied when players mention how the game engaged them in learning, or they suggest a way in which they can learn about the game or from the game |
Vicarious learning | Players would like to see other players’ solutions to learn better ways to solve problems |
Interest in quantum mechanics | Players state they have learned more about the specific field of science of the game |
Game design suggestions | Applied when players suggest design ideas for improving the game |
Graphic design suggestions | Players suggest changes to the graphic aspects of the game |
Opportunities for player customization | Players state they would like to be able to change certain features of the game ad libitum (e.g., re-skin the game) |
Send reminders | Players state they would like to receive periodic reminders to play the game |
Sound design suggestions | Players suggest sounds designs for the game, such as auditory cues for success or background music |
New mechanics suggestions | Players suggest new or improved mechanics for the game |
Enlarge game | Players suggest developing the game further, for example by adding levels |
Improving tutorials | Players state tutorials should be improved, or they evaluated the current tutorial as insufficient |
Influence of leader board on motivation | Players evaluate how the leaderboard influenced their motivation when playing the game |
Interest in contributing to citizen science | Players state their interest in the game for their contribution to citizen science |
Intuitive (easy to understand) | Players state the game and game mechanics were easy to understand, or they did not have to make mental effort to understand it and play it |
Intuitive solution | Players state the game was easy to play and not much knowledge was required to solve the problems |
Knowledge representation | Players state the way the game represents information on quantum physics makes it easy to understand this topic |
Lack of game feedback | Players state not having enough feedback from the game |
Lack of information regarding quantum physics (scientific topic) | Players state the game lacked of more and deeper information about quantum physics |
Lack of interaction with the scientific team and the game developers | Players state the need for more interaction and conversations with the team of scientist and developers behind the game to better understand how the data gathered is being used |
Lack of understanding | Applied when players state not being able to understand certain aspects of the game |
Lack of understanding of the game | Players state they did not understand the game |
Lack of understanding of the game mechanics | Players state they did not understand the game mechanics |
Lack of understanding of the procedural understanding | Players state they did not understand the procedures required to solve the problems |
Lack of understanding of the UI interface | Players state they did not understand the user interface or they were confused by it |
Lack of understanding of the purpose of the game | Players state they did not understand the purpose of the game apart from the partial goals in each level |
Learning curve | Players make a statement on the learning curve of the game. Statements about game progression, level progression, and blockages in the gameplay fall into this category |
Parallel thinking | Participant state having to think in non-conventional ways to solve the problems proposed in the game |
Player types | Players state they engage or not engage with the game due to a personal trait |
Competitive | Players state they are competitive or like competition |
Non-competitive | Players state they are not competitive or dislike competition |
Not a gamer | Players state they are regular players of video games |
Puzzler | Players state they like solving puzzles |
Psychophysical constraints | Players state having problems performing tasks requiring fine motor skills |
Relationship between game and the scientific topic (QP) | Players state that it would be a gain for the game if it highlighted more its scientific relevance |
Relevance of controllers | Players state controllers were relevant for the implementation of the strategy, usually in a negative way |
Relevance of gaming experience | Player state that their experience playing games helped them solve the challenges proposed by the game |
Relevance of science knowledge | Player state their knowledge of science helped them solve the challenges proposed by the game |
Rewards other than the score system | Players state they find rewarding aspects of the game that are different from the score system, e.g., learning science or helping the development of a scientific tool |
Science dissemination | Players state the game is good for science dissemination |
Social aspects | Players state interest in the inclusion of social aspects in the game |
Strategy making | Applied when players talked about the strategies they used to solve the challenges proposed by the game |
Lack of strategy | Players state not having a strategy |
Scientific reasoning/experimentation | Players state experimenting and finding patterns which allow them to overcome the game challenges |
Serendipity | Players solved the problem by chance or luck |
Trial and error | Players solved the game challenges using trial and error |
Technical issues | Applied when players state having technical issues with the game |
Problems with responsiveness and accuracy | Players state having problems with the responsiveness or the accuracy of the game depending on the device they used |
Game bugs | Players reported bugs in the game |
Relevance of OS | Players reported incompatibility of the game with some operative systems |
Relevance of console | Players state the device (e.g., PC or mobile device) they used for playing influenced their performance |
Problems with data transmission | Players reported to stop playing because data transmission was very costly |
Too busy to play | Players state not playing much because they had other activities to do |
Rights and permissions
About this article
Cite this article
Díaz, C., Ponti, M., Haikka, P. et al. More than data gatherers: exploring player experience in a citizen science game. Qual User Exp 5, 1 (2020). https://doi.org/10.1007/s41233-019-0030-8
Received:
Published:
DOI: https://doi.org/10.1007/s41233-019-0030-8