Abstract
Since immersive VR devices have become commodities, immersive environments appear as a new tool in the development of high-fidelity prototypes of systems in which the user interaction relies on expensive or unusual hardware, e.g., industrial systems. However, there is not enough evidence that the interface of a complex system and its VR counterpart have equal usability and user experience qualities. Our main objective is to assess the feasibility of carrying out studies on user-based evaluation in industrial interactive systems through immersive VR simulation. To achieve this, we compared user assessment with a conventional prototype of an industrial system with its immersive VR simulation. We performed within-subjects user testing in both the physical and the VR setups, and collected (i) experimenters’ observations on usability issues and (ii) subjective and objective measures of 16 participants. Subjective measures were taken using standardized questionnaires and objective measures by logging the elapsed time to fulfill task scenarios. Our results indicate that the perceived quality of the immersive VR system is indistinguishable from the physical counterpart regarding User Experience, usability, and cybersickness. On the other hand, the users’ performance on VR simulation was significantly slower in immersive VR. Finally, the same usability issues could be detected with either of the conditions.
This study was financed in part by the Coordenação de Aperfeiçamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001, and by Petrobras under the project INF/LAMECC/PETROBRAS 0050.0098154.15.9.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Barbosa, S., da Silva, B.: Interação Humano-Computador. Elsevier, Rio de Janeiro (2010)
Bolder, A., Grünvogel, S.M., Angelescu, E.: Comparison of the usability of a car infotainment system in a mixed reality environment and in a real car. In: Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology, pp. 1–10 (2018)
Brooke, J.: SUS: a ‘quick and dirty’ usability scale. In: Usability Evaluation in Industry, pp. 189–194. Taylor Francis, London (1996)
Davis, S., Nesbitt, K., Nalivaiko, E.: A systematic review of cybersickness. In: Proceedings of the 2014 Conference on Interactive Entertainment, pp. 1–9 (2014)
Higuera-Trujillo, J.L., Maldonado, J.L.T., Millán, C.L.: Psychological and physiological human responses to simulated and real environments: a comparison between photographs, 360\(^\circ \) panoramas, and virtual reality. Appl. Ergon. 65, 398–409 (2017)
Hinkle, D.E., Wiersma, W., Jurs, S.G.: Applied Statistics for the Behavioral Sciences, 5th edn. Houghton Mifflin, Boston (2003)
Insko, B.E.: Passive haptics significantly enhances virtual environments. Ph.D. thesis, The University of North Carolina, Chapel Hill, NC, USA (2001)
ISO 9241–210:2019 - ergonomics of human-system interaction - part 210: Human-centred design for interactive systems. Standard, International Organization for Standardization, Geneva, Switzerland, July 2019
Kamide, H., Yasumoto, M., Mae, Y., Takubo, T., Ohara, K., Arai, T.: Comparative evaluation of virtual and real humanoid with robot-oriented psychology scale. In: 2011 IEEE International Conference on Robotics and Automation, pp. 599–604. IEEE (2011)
Kennedy, R.S., Lane, N.E., Berbaum, K.S., Lilienthal, M.G.: Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int. J. Aviat. Psychol. 3(3), 203–220 (1993)
Lakatos, E.M., Marconi, M.: Fundamentos de Metodologia Científica, Atlas, São Paulo, SP, Brazil (2017)
Laugwitz, B., Held, T., Schrepp, M.: Construction and evaluation of a user experience questionnaire. In: Holzinger, A. (ed.) USAB 2008. LNCS, vol. 5298, pp. 63–76. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-89350-9_6
LaViola Jr., J.J.: A discussion of cybersickness in virtual environments. SIGCHI Bull. 32(1), 47–56 (2000). https://doi.org/10.1145/333329.333344
Rubin, J., Chisnell, D.: Handbook of Usability Testing: How to Plan, Design and Conduct Effective Tests, 2nd edn. Wiley, Indianapolis (2008)
Satter, K., Butler, A.: Competitive usability analysis of immersive virtual environments in engineering design review. J. Comput. Inf. Sci. Eng. 15(3) (2015). https://doi.org/10.1115/1.4029750
Sauro, J., Lewis, J.: Quantifying the User Experience: Practical Statistics for User Research, 2nd edn. Elsevier, Cambridge (2016)
Sharp, H., Rogers, Y., Preece, J.: Interaction Design: Beyond Human-computer Interaction, 2nd edn. John Wiley and Sons Inc., West Sussex (2007)
Tullis, T., Albert, B.: Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Elsevier, Waltham (2013)
Vince, J.: Virtual Reality Systems. Pearson Education, Essex (1995)
Werrlich, S., Daniel, A., Ginger, A., Nguyen, P.A., Notni, G.: Comparing HMD-based and paper-based training. In: 2018 IEEE International Symposium on Mixed and Augmented Reality, pp. 134–142. IEEE (2018)
Wolfartsberger, J.: Analyzing the potential of virtual reality for engineering design review. Autom. Constr. 104, 27–37 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Cheiran, J.F.P. et al. (2020). Comparing Physical and Immersive VR Prototypes for Evaluation of an Industrial System User Interface. In: Magnenat-Thalmann, N., et al. Advances in Computer Graphics. CGI 2020. Lecture Notes in Computer Science(), vol 12221. Springer, Cham. https://doi.org/10.1007/978-3-030-61864-3_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-61864-3_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-61863-6
Online ISBN: 978-3-030-61864-3
eBook Packages: Computer ScienceComputer Science (R0)