Abstract
The virtual-hand interaction technique is a common input technique in virtual environments (VEs). The current application of virtual-hand interaction in VEs lacks specialized objective evaluation methods, making it difficult to establish a systematic functional goal orientation. To achieve a quantitative evaluation of the virtual-hand interaction system in VEs, we developed the modified evaluation method of goals, operators, methods, and selection rules (H-GOMS). The evaluation model contains five modules: analysis, decomposition, configuration, acquisition, and evaluation. To build the H-GOMS model, the relevant temporal parameters of operators in VEs were measured, and interactive rules were formulated for the configuration module. In addition to establishing the H-GOMS evaluation model, this paper demonstrates the development of a performance evaluation software (HI2ET) based on the Unity engine. We realized automatic retrieval and identification of the interactive behavior information from the software and applied the H-GOMS model algorithm for real-time visualization of the interactive process. The proposed method, with modeling of interactive tasks based on expert users, enables feasible and generally quantifiable performance evaluation for virtual-hand interaction systems in VEs.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Ahir K, Govani K, Gajera R, Shah M (2020) Application on virtual reality for enhanced education learning, military training and sports. Augment Hum Res 5(1):1–9. https://doi.org/10.1007/s41133-019-0025-2
Al-Shamayleh AS, Ahmad R, Abushariah M, Alam KA, Jomhari N (2018) A systematic literature review on vision-based gesture recognition techniques. Multimed Tools Appl 77(21):28121–28184. https://doi.org/10.1007/s11042-018-5971-z
Anderson JR, Bothell D, Lebiere C, Matessa M (1998) An integrated theory of list memory. J Mem Lang 38(4):341–380. https://doi.org/10.1006/jmla.1997.2553
Baber C (2005) Evaluation in human-computer interaction. In: Evaluation of human work, pp 357–388
Bachmann D, Weichert F, Rinkenauer G (2018) Review of three-dimensional human-computer interaction with focus on the leap motion controller. Sensors 18(7):2194. https://doi.org/10.3390/s18072194
Benko H, Holz C, Sinclair M, Ofek E (2016) Normaltouch and texturetouch: High-fidelity 3d haptic shape rendering on handheld virtual reality controllers. In: Proceedings of the 29th annual symposium on user interface software and technique, pp 717–728. https://doi.org/10.1145/2984511.2984526
Bevan N, Carter J, Earthy J, Geis T, Harker S (2016) New ISO standards for usability, usability reports and usability measures. In: International conference on human-computer interaction, pp 268–278. https://doi.org/10.1007/978-3-319-39510-4_25
Card SK, Moran TP, Newell A (1983) The psychology of human–computer interaction. Am J Psychol 25(4):66. https://doi.org/10.2307/1422176
Chakraborty BK, Sarma D, Bhuyan MK, MacDorman KF (2017) Review of constraints on vision-based gesture recognition for human–computer interaction. IET Comput Vision 12(1):3–15. https://doi.org/10.1049/iet-cvi.2017.0052
Chang YH, Devine KL (2019) Modeling cognitive activities in a virtual reality-assisted industrial robot programming environment. In: Proceedings of ASEE EDGD 73rd midyear conference, pp 99–104
Chen R, Hua Q, Ji X et al (2017) An interactive task analysis framework and interactive system research for computer aided diagnosis. IEEE Access 5:23413–23424. https://doi.org/10.1109/ACCESS.2017.2761235
Chrastil ER, Warren WH (2012) Active and passive contributions to spatial learning. Psychon Bull Rev 19(1):1–23. https://doi.org/10.3758/s13423-011-0182-x
Cunha D, Duarte R P, Cunha C A (2021) KLM-GOMS detection of interaction patterns through the execution of unplanned tasks. In: International conference on computational science and its applications, pp 203–219. https://doi.org/10.1007/978-3-030-86960-1_15
Diaper D, Stanton N (2003) The handbook of task analysis for human-computer interaction. https://doi.org/10.1201/b12470
Dong Z, Le K, Chuan L (2013) A virtual reality-based maintenance time measurement methodology for complex products. Assem Autom. https://doi.org/10.1108/aa-12-2013-048
El-Shawa S, Kraemer N, Sheikholeslami S, Mead R, Croft EA (2017) “Is this the real life? Is this just fantasy?”: human proxemic preferences for recognizing robot gestures in physical reality and virtual reality. In: 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 341–348. https://doi.org/10.1109/IROS.2017.8202178
Ependi U, Kurniawan TB, Panjaitan F (2019) System usability scale vs heuristic evaluation: a review. Simetris Jurnal Teknik Mesin Elektro dan Ilmu Komputer 10(1):65–74. https://doi.org/10.24176/simet.v10i1.2725
Estes S (2019) Cogulator: a cognitive calculator. http://cogulator.io
Fang YM, Lin C (2019) The usability testing of VR interface for tourism apps. Appl Sci 9(16):3215. https://doi.org/10.3390/app9163215
Fittkau F, Krause A, Hasselbring W (2015) Exploring software cities in virtual reality. In: 2015 IEEE 3rd working conference on software visualization (vissoft), pp 130–134. https://doi.org/10.1109/VISSOFT.2015.7332423
Fitts PM (1954) The information capacity of the human motor system in controlling the amplitude of movement. J Exp Psychol 47(6):381. https://doi.org/10.1037//0096-3445.121.3.262
Gatti E, Calzolari E, Maggioni E, Obrist M (2018) Emotional ratings and skin conductance response to visual, auditory and haptic stimuli. Sci Data 5(1):1–12. https://doi.org/10.1038/sdata.2018.120
Habibi P, Chattopadhyay D (2021) The impact of handedness on user performance in touchless input. Int J Hum Comput Stud 149:102600. https://doi.org/10.1016/j.ijhcs.2021.102600
Hald K, Rehm M, Moeslund TB (2018) Testing augmented reality systems for spotting sub-surface impurities. In: IFIP working conference on human work interaction design, pp 103–112. https://doi.org/10.1007/978-3-030-05297-3_7
Hansberger JT, Peng C, Mathis SL, Shanthakumar VA, Meacham SC, Cao L, Blakely VR (2017) Dispelling the gorilla arm syndrome: the viability of prolonged gesture interactions. In: International conference on virtual, augmented and mixed reality, pp 505–520. https://doi.org/10.1007/978-3-319-57987-0_41
Harris D, Wilson M, Vine S (2020) Development and validation of a simulation workload measure: the simulation task load index (SIM-TLX). Virtual Reality 24(4):557–566. https://doi.org/10.1007/s10055-019-00422-9
Höll M, Oberweger M, Arth C, Lepetit V (2018) Efficient physics-based implementation for realistic hand-object interaction in virtual reality. In: 2018 IEEE conference on virtual reality and 3D user interfaces (VR), pp 175–182. https://doi.org/10.1109/VR.2018.8448284
ISO (1998) 9241-11: 1998 Ergonomic requirements for office work with visual display terminals (VDTs)–part 11: Guidance on usability. ISO, Geneva. https://doi.org/10.1201/9781315572543
Jenkins DP, Stanton NA, Salmon PM, Walker GH (2017) Cognitive work analysis: coping with complexity. CRC Press. https://doi.org/10.1201/9781315572543
John BE, Prevas K, Salvucci DD, Koedinger K (2004) Predictive human performance modeling made easy. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp 455–462. https://doi.org/10.1145/985692.985750
Joshi S, Hamilton M, Warren R, Faucett D, Tian W, Wang Y, Ma J (2021) Implementing virtual reality technology for safety training in the precast/prestressed concrete industry. Appl Ergon 90:103286. https://doi.org/10.1016/j.apergo.2020.103286
Karafotias G, Teranishi A, Korres G, Eyssel F, Copti S, Eid M (2017) Intensifying emotional reactions via tactile gestures in immersive films. ACM Trans Multimed Comput Commun Appl 13(3):1–17. https://doi.org/10.1145/3092840
Karray F, Alemzadeh M, Abou Saleh J, Arab MN (2017) Human–computer interaction: overview on state of the art. Int J Smart Sens Intell Syst. https://doi.org/10.21307/ijssis-2017-283
Katona J (2021) A review of human–computer interaction and virtual reality research fields in cognitive Info communications. Appl Sci 11(6):2646. https://doi.org/10.3390/app11062646
Katzakis N, Chen L, Teather RJ, Ariza O, Steinicke F (2019) Evaluation of 3D pointing accuracy in the fovea and periphery in immersive head-mounted display environments. IEEE Trans Visual Comput Graphics. https://doi.org/10.1109/TVCG.2019.2947504
Kleinermann F, De Troyer O, Mansouri H, Romero R, Pellens B, Bille W (2005) Designing semantic virtual reality applications. In: Proceedings of the 2nd INTUITION international workshop, Senlis, France. https://api.semanticscholar.org/CorpusID:8576233
Koutsabasis P, Vogiatzidakis P (2019) Empirical research in mid-air interaction: a systematic review. Int J Hum–Comput Interact 35(18):1747–1768. https://doi.org/10.1080/10447318.2019.1572352
Kumar N, Kumar J (2016) Measurement of cognitive load in HCI systems using EEG power spectrum: an experimental study. Procedia Comput Sci 84:70–78. https://doi.org/10.1016/j.procs.2016.04.068
Lewis JR (2018) The system usability scale: past, present, and future. Int J Hum–Comput Interact 34(7):577–590. https://doi.org/10.1080/10447318.2018.1455307
Li J, Zhou X, Jia L, Xiao W, Jin Y, Xue C (2021) A modeling method to evaluate the finger-click interactive task in the virtual environment. In: International conference on intelligent human systems integration. Springer, Cham, pp 187–193. https://doi.org/10.1007/978-3-030-68017-6_28
Liapis A, Katsanos C, Sotiropoulos D, Xenos M, Karousos N (2015) Recognizing emotions in human–computer interaction: studying stress using skin conductance. In: IFIP conference on human–computer interaction, pp 255–262. https://doi.org/10.1007/978-3-319-22701-6_18
Lin W, Du L, Harris-Adamson C, Barr A, Rempel D (2017) Design of hand gestures for manipulating objects in virtual reality. In: International conference on human-computer interaction, pp 584–592. https://doi.org/10.1007/978-3-319-58071-5_44
Longo L (2017) Subjective usability, mental workload assessments and their impact on objective human performance. In: IFIP conference on human–computer interaction, pp 202–223. https://doi.org/10.1007/978-3-319-67684-5_13
Lubos P, Bruder G, Ariza O, Steinicke F (2016) Touching the sphere: leveraging joint-centered kinespheres for spatial user interaction. In: Proceedings of the 2016 symposium on spatial user interaction, pp 13–22. https://doi.org/10.1145/2983310.2985753
Lv X, Xue C, Xiao W (2021) Influence of size and depth perception on ray-casting interaction in virtual reality. In: International conference on intelligent human systems integration, pp 152–158. https://doi.org/10.1007/978-3-030-68017-6_23
Lytvynova S (2018) Cognitive tasks design by applying computer modeling system for forming competences in mathematics. In: ICTERI workshops, pp 278–293
Madathil KC, Greenstein JS (2017) An investigation of the efficacy of collaborative virtual reality systems for moderated remote usability testing. Appl Ergon 65:501–514. https://doi.org/10.1016/j.apergo.2017.02.011
Marchiori E, Niforatos E, Preto L (2018) Analysis of users’ heart rate data and self-reported perceptions to understand effective virtual reality characteristics. Inf Technol Tourism 18(1):133–155. https://doi.org/10.1007/s40558-018-0104-0
Murata A, Iwase H (2001) Extending Fitts’ law to a three-dimensional pointing task. Hum Mov Sci 20(6):791–805. https://doi.org/10.1016/S0167-9457(01)00058-6
Niehorster DC, Li L, Lappe M (2017) The accuracy and precision of position and orientation tracking in the HTC vive virtual reality system for scientific research. i-Perception 8(3): 2041669517708205. https://doi.org/10.1177/2041669517708205
Oyedotun OK, Khashman A (2017) Deep learning in vision-based static hand gesture recognition. Neural Comput Appl 28(12):3941–3951. https://doi.org/10.1007/s00521-016-2294-8
Park KB, Lee JY (2019) New design and comparative analysis of smartwatch metaphor-based hand gestures for 3D navigation in mobile virtual reality. Multimed Tools Appl 78(5):6211–6231. https://doi.org/10.1007/s11042-018-6403-9
Perng SS, Hu NT, Tsai PS, Wu TF, Chen JY (2020) Wearable Devices to Control Objects in Virtual Reality. Sens Mater 32(6):2007–2015. https://doi.org/10.18494/SAM.2020.2789
Pfeuffer K, Mayer B, Mardanbegi D, Gellersen H (2017) Gaze+ pinch interaction in virtual reality. In: Proceedings of the 5th symposium on spatial user interaction, pp 99–108. https://doi.org/10.1145/3131277.3132180
Pike M, Ch'ng E (2016) Evaluating virtual reality experience and performance: a brain-based approach. In: Proceedings of the 15th ACM SIGGRAPH conference on virtual-reality continuum and its applications in industry, vol 1, pp 469–474. https://doi.org/10.1145/3013971.3014012
Radianti J, Majchrzak TA, Fromm J, Wohlgenannt I (2020) A systematic review of immersive virtual reality applications for higher education: design elements, lessons learned, and research agenda. Comput Educ 147:103778. https://doi.org/10.1016/j.compedu.2019.103778
Ramkumar A, Stappers PJ, Niessen WJ, Adebahr S, Schimek-Jasch T, Nestle U, Song Y (2017) Using GOMS and NASA-TLX to evaluate human–computer interaction process in interactive segmentation. Int J Hum–Comput Interact 33(2):123–134. https://doi.org/10.1080/10447318.2016.1220729
Ramos MA, Thieme CA, Utne IB, Mosleh A (2020) Human-system concurrent task analysis for maritime autonomous surface ship operation and safety. Reliab Eng Syst Saf 195:106697. https://doi.org/10.1016/j.ress.2019.106697
Rautaray SS, Agrawal A (2015) Vision based hand gesture recognition for human computer interaction: a survey. Artif Intell Rev 43(1):1–54. https://doi.org/10.1007/s10462-012-9356-9
Reinschluessel AV, Teuber J, Herrlich M et al (2017) Virtual reality for user-centered design and evaluation of touch-free interaction techniques for navigating medical images in the operating room. In: Proceedings of the 2017 CHI conference extended abstracts on human factors in computing systems, pp 2001–2009. https://doi.org/10.1145/3027063.3053173
Roose KM, Veinott ES, Mueller ST (2018) The tracer method: the dynamic duo combining cognitive task analysis and eye tracking. In: Proceedings of the 2018 annual symposium on computer-human interaction in play companion extended abstracts, pp 585–593. https://doi.org/10.1145/3270316.327152
Rosyidah U, Haryanto H, Kardianawati A (2019) Usability evaluation using GOMS model for education game “Play and Learn English”. In: 2019 International seminar on application for technology of information and communication (iSemantic), pp 1–5. https://doi.org/10.1109/ISEMANTIC.2019.8884268
Schwind V, Mayer S, Comeau-Vermeersch A, Schweigert R, Henze N (2018) Up to the finger tip: the effect of avatars on mid-air pointing accuracy in virtual reality. In: Proceedings of the 2018 annual symposium on computer-human interaction in play, pp 477–488. https://doi.org/10.1145/3242671.3242675
Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(3):379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x
Shanthakumar VA, Peng C, Hansberger J, Cao L, Meacham S, Blakely V (2020) Design and evaluation of a hand gesture recognition approach for real-time interactions. Multimed Tools Appl. https://doi.org/10.1007/s11042-019-08520-1
Strazdins G, Pedersen BS, Zhang H, Major P (2017) Virtual reality using gesture recognition for deck operation training. In: OCEANS 2017-Aberdeen, pp 1–6. https://doi.org/10.1109/OCEANSE.2017.8084584
Sucar LE, Orihuela-Espina F, Velazquez RL, Reinkensmeyer DJ, Leder R, Hernández-Franco J (2013) Gesture therapy: an upper limb virtual reality-based motor rehabilitation platform. IEEE Trans Neural Syst Rehabil Eng 22(3):634–643. https://doi.org/10.1109/TNSRE.2013.2293673
Sudha MR, Sriraghav K, Jacob SG, Manisha S (2017) Approaches and applications of virtual reality and gesture recognition: a review. Int J Ambient Comput Intell 8(4):1–18. https://doi.org/10.4018/IJACI.2017100101
Sun JH, Ji TT, Zhang SB, Yang JK, Ji GR (2018) Research on the hand gesture recognition based on deep learning. In: 2018 12th International symposium on antennas, propagation and EM theory (ISAPE), pp 1–4. https://doi.org/10.1109/ISAPE.2018.8634348
Tadeja SK, Seshadri P, Kristensson PO (2020) AeroVR: an immersive visualisation system for aerospace design and digital twinning in virtual reality. Aeronaut J 124(1280):1615–1635. https://doi.org/10.1017/aer.2020.49
Wang K, Li K, Gao J, Liu B, Fang Z, Ke W (2021) A quantitative evaluation method of software usability based on improved GOMS model. In: 2021 IEEE 21st international conference on software quality, reliability and security companion (QRS-C), pp 691–697. https://doi.org/10.1109/QRS-C55045.2021.00104
Welford AT (1968) Fundamentals of skill. Am J Psychol 83(2):66. https://doi.org/10.2307/1421343
Weninger M, Grünbacher P, Gander E, Schörgenhumer A (2020) Evaluating an interactive memory analysis tool: findings from a cognitive walkthrough and a user study. Proc ACM Hum–Comput Interact 4:1–37. https://doi.org/10.1145/3394977
Yang MT, Liao WC, Shih YC (2013) VECAR: virtual English classroom with markerless augmented reality and intuitive gesture interaction. In: 2013 IEEE 13th international conference on advanced learning technologies, pp 439–440. https://doi.org/10.1109/ICALT.2013.134
Yu D, Liang HN, Lu F, Nanjappan V, Papangelis K, Wang W (2018) Target selection in head-mounted display virtual reality environments. J UCS 24(9):1217–1243
Ziak P, Holm A, Halicka J, Mojžiš P, Pinero DP (2017) Amblyopia treatment of adults with dichoptic training using the virtual reality oculus rift head mounted display: preliminary results. BMC Ophthalmol 17(1):1–8. https://doi.org/10.1186/s12886-017-0501-8
Funding
This work has received funding partly from the National Natural Science Foundation of China (Nos. 71901061, 71871056) and Postgraduate Research＆Practice Innovation Program of Jiangsu Province (Grant No. SJCX21_0045).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare there is no conflicts of interest regarding the publication of this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhou, X., Teng, F., Du, X. et al. H-GOMS: a model for evaluating a virtual-hand interaction system in virtual environments. Virtual Reality 27, 497–522 (2023). https://doi.org/10.1007/s10055-022-00674-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10055-022-00674-y