Abstract
The ubiquity of Information and Communication Technologies (ICT) presents challenges in designing experiences and interfaces. Hence, it becomes crucial to have solutions that facilitate the comparison of various products, discerning their relative strengths and weaknesses. Therefore, this paper presents a proposal of an Evaluation Framework for Benchmarking the user eXperience and Usability of Online Social Networks. Its goal is to provide a holistic perspective of user eXperience and satisfaction, checked in conformance with a detailed heuristic analysis. This Framework is divided into three parts, comprising (i) the selection of experts, and products to benchmark; (ii) the evaluation of UX Satisfaction of each product with Emocards, and (iii) test the User eXperience within a Usability Heuristic Evaluation for each product. By applying the specific study case of seven senior online social networks and resorting to 14 experts in the field of ICT, it was possible to build a holistic Benchmarking Evaluation Framework. In particular, this Framework introduces new perspectives on (i) the priority sequence for UX and satisfaction, followed by heuristic detail; (ii) the seamless incorporation of feedback obtained during the think-aloud process into the analysis and data representation; (iii) the creation of a normalized score for each heuristic of every product, thus allowing for a direct comparison between products; and (iv) the creation of integrated representations of the two dimensions under study – i.e., UX satisfaction and Usability Heuristic Evaluation. Overall, we hope that this proposed Framework can serve as a simple yet effective approach to represent and compare evaluation results between products.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Available at: https://www.buzz50.com/ (Access date: 12–12-2023).
- 2.
Available at: https://www.early-retirement.org (Access date: 12–12-2023).
- 3.
Available at: https://www.gransnet.com/ (Access date: 12–12-2023).
- 4.
Available at: https://mione.altice.pt (Access date: 12–12-2023).
- 5.
Available at: https://www.olderiswiser.com/ (Access date: 12–12-2023).
- 6.
Available at: https://www.silversurfers.com/community/ (Access date: 12–12-2023).
- 7.
Available at: https://www.stitch.net/ (Access date: 12–12-2023).
- 8.
Available at: http://forms.ua.pt/ (Accessed date: 12–12-2023). A service offered by the University of Aveiro using the LimeSurvey software – https://www.limesurvey.org/pt/ (Accessed date: 12–12-2023).
References
Castells, M.: The internet galaxy: reflexions on the internet, business, and society. Oxford University Press (2002). https://doi.org/10.1093/ACPROF:OSO/9780199255771.001.0001
PORDATA: Indivíduos com 16 e mais anos que utilizam computador e Internet em % do total de indivíduos: por grupo etário. https://www.pordata.pt/Portugal/Indiv%c3%adduos+com+16+e+mais+anos+que+utilizam+computador+e+Internet+em+percentagem+do+total+de+indiv%c3%adduos+por+grupo+et%c3%a1rio-1139, (Accessed 29 May 2022)
Faverio, M.: Share of those 65 and older who are tech users has grown in the past decade - Pew Research Center (2022)
Pew Research Center: Use of online platforms, apps varies – sometimes widely – by demographic group, https://www.pewresearch.org/internet/2021/04/07/social-media-use-in-2021/pi_2021-04-07_social-media_0-03/, (Accessed 01 June 2022)
Pew Research Center: Social Media Fact Sheet. https://www.pewresearch.org/internet/fact-sheet/social-media/, (Accessed 01 June 2022)
Villani, D., Serino, S., Triberti, S., Riva, G.: Ageing positively with digital games. LNICST, pp. 148–155. Springer Verlag (2017). https://doi.org/10.1007/978-3-319-49655-9_20
Lee, H.R., Riek, L.D.: Reframing assistive robots to promote successful aging. ACM Trans. Human-Robot Interact. (THRI) 7 (2018). https://doi.org/10.1145/3203303
Langley, J., Wheeler, G., Partridge, R., Bec, R., Wolstenholme, D., Sproson, L.: Designing with and for older people. Intell. Syst. Reference Library. 167, 3–19 (2020). https://doi.org/10.1007/978-3-030-26292-1_1/COVER
Vandekerckhove, P., De Mul, Ma., Bramer, W.M., De Bont, A.A.: Generative participatory design methodology to develop electronic health interventions: systematic literature review. J Med Internet Res. 22 (2020). https://doi.org/10.2196/13780
Sumner, J., Chong, L.S., Bundele, A., Wei Lim, Y.: Co-designing technology for aging in place: a systematic review. Gerontologist 61, E395–E409 (2021). https://doi.org/10.1093/GERONT/GNAA064
Cole, A.C., Adapa, K., Khasawneh, A., Richardson, D.R., Mazur, L.: Codesign approaches involving older adults in the development of electronic healthcare tools: a systematic review. BMJ Open 12 (2022). https://doi.org/10.1136/BMJOPEN-2021-058390
Schrepp, M., Hinderks, A., Thomaschewski, J.: Applying the user experience questionnaire (UEQ) in different evaluation scenarios. In: Marcus, A. (eds.) Design, User Experience, and Usability. Theories, Methods, and Tools for Designing the User Experience. DUXU 2014. LNCS, vol. 8517. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07668-3_37
Chammas, A., Quaresma, M., Mont’Alvão, C.: A closer look on the user centred design. Proc. Manuf. 3, 5397–5404 (2015). https://doi.org/10.1016/j.promfg.2015.07.656
ISO: ISO 9241–210:2019 - Ergonomics of human-system interaction — Part 210: Human-centred design for interactive systems, https://www.iso.org/standard/77520.html, (Accessed 07 Feb 2024)
Boy, G.: The Handbook of Human-Machine Interaction. CRC Press (2017). https://doi.org/10.1201/9781315557380/HANDBOOK-HUMAN-MACHINE-INTERACTION-GUY-BOY
Nielsen, J.: Usability 101: Introduction to Usability, https://www.nngroup.com/articles/usability-101-introduction-to-usability/, (Accessed 19 Jan 2022)
Saavedra, M.-J., Rusu, C., Quiñones, D., Roncagliolo, S.: A set of usability and user experience heuristics for social networks. Presented at the (2019). https://doi.org/10.1007/978-3-030-21902-4_10
Saleh, A.M., Abuaddous, H.Y., Enaizan, O., Ghabban, F.: User experience assessment of a COVID-19 tracking mobile application (AMAN) in Jordan. Indonesian J. Elect. Eng. Comput. Sci. 23, 1120 (2021). https://doi.org/10.11591/ijeecs.v23.i2.pp1120-1127
Larbi, D., Denecke, K., Gabarron, E.: Usability testing of a social media chatbot for increasing physical activity behavior. J Pers Med. 12, 828 (2022). https://doi.org/10.3390/jpm12050828
Faradina, H.R., Wahyuningrum, T., Prasetyo, N.A.AI.K.: User Experience analysis on e-wallet using a combination of heuristic evaluation and UMUX. In: 2022 IEEE International Conference on Cybernetics and Computational Intelligence (CyberneticsCom), pp. 46–51. IEEE (2022). https://doi.org/10.1109/CyberneticsCom55287.2022.9865427
Halim, I., Saptari, A., A.Perumal, P., Abdullah, Z., Abdullah, S., Muhammad, M.N.: A review on usability and user experience of assistive social robots for older persons. Intern. J. Integrated Eng. 14 (2022). https://doi.org/10.30880/ijie.2022.14.06.010
Schrepp, M., Hinderks, A., Thomaschewski, J.: Construction of a benchmark for the user experience questionnaire (UEQ). Inter. J. Interact. Multimedia Artifi. Intell. 4, 40 (2017). https://doi.org/10.9781/ijimai.2017.445
Desmet, P., Overbeeke, K., Tax, S.: Designing products with added emotional value: development and appllcation of an approach for research through design. Des. J. 4, 32–47 (2001). https://doi.org/10.2752/146069201789378496
Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Empowering People - CHI 1990, pp. 249–256. ACM Press, New York (1990). https://doi.org/10.1145/97243.97281
Nielsen, J., Mack, R.L.: Usability inspection methods. Wiley (1994)
Desurvire, H., Wixon, D.: Game principles: change, choice & creation: making better games. In: CHI 2013 Extended Abstracts on Human Factors in Computing Systems on - CHI EA 2013, p. 1065. ACM Press, New York (2013). https://doi.org/10.1145/2468356.2468547
Nielsen, J.: 10 Usability Heuristics for User Interface Design. https://www.nngroup.com/articles/ten-usability-heuristics/, (Accessed 13 Jan 2022)
Nimrod, G.: The fun culture in seniors’ online communities. Gerontologist 51, 226–237 (2011). https://doi.org/10.1093/geront/gnq084
Wolf, M., Sims, J., Yang, H.: Social media? what social media? In: UK Academy for Information Systems Conference Proceedings 2018 (2018)
Acknowledgments
The authors wish to thank not only the participants of this study but also DigiMedia, the Department of Communication and Art at the University of Aveiro. The study reported in this publication was supported by F.C.T. – Foundation for Science and Technology (Fundação para a Ciência e Tecnologia), I.P. nr. 2021.06465. B.D, DigiMedia Research Center, under the project UIDB/05460/2020, and the IC Senior X project funded by DigiMedia (GIP3_2022).
Disclosure of Interests.
The authors declare no conflict of interest. Informed consent was obtained from all participants involved in the study.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Regalado, F., Mealha, Ó., Santos, C., Veloso, A.I. (2024). Benchmarking the User eXperience and Usability of Online Social Networks: Proposal of an Evaluation Framework. In: Gao, Q., Zhou, J. (eds) Human Aspects of IT for the Aged Population. HCII 2024. Lecture Notes in Computer Science, vol 14725. Springer, Cham. https://doi.org/10.1007/978-3-031-61543-6_29
Download citation
DOI: https://doi.org/10.1007/978-3-031-61543-6_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-61542-9
Online ISBN: 978-3-031-61543-6
eBook Packages: Computer ScienceComputer Science (R0)