{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T04:10:15Z","timestamp":1726805415013},"reference-count":42,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2022,2,24]],"date-time":"2022-02-24T00:00:00Z","timestamp":1645660800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010418","name":"Institute for Information and Communications Technology Promotion","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neuroinform."],"abstract":"Brain\u2013computer interfaces (BCIs) based on electroencephalogram (EEG) have recently attracted increasing attention in virtual reality (VR) applications as a promising tool for controlling virtual objects or generating commands in a \u201chands-free\u201d manner. Video-oculography (VOG) has been frequently used as a tool to improve BCI performance by identifying the gaze location on the screen, however, current VOG devices are generally too expensive to be embedded in practical low-cost VR head-mounted display (HMD) systems. In this study, we proposed a novel calibration-free hybrid BCI system combining steady-state visual-evoked potential (SSVEP)-based BCI and electrooculogram (EOG)-based eye tracking to increase the information transfer rate (ITR) of a nine-target SSVEP-based BCI in VR environment. Experiments were repeated on three different frequency configurations of pattern-reversal checkerboard stimuli arranged in a 3 \u00d7 3 matrix. When a user was staring at one of the nine visual stimuli, the column containing the target stimulus was first identified based on the user\u2019s horizontal eye movement direction (left, middle, or right) classified using horizontal EOG recorded from a pair of electrodes that can be readily incorporated with any existing VR-HMD systems. Note that the EOG can be recorded using the same amplifier for recording SSVEP, unlike the VOG system. Then, the target visual stimulus was identified among the three visual stimuli vertically arranged in the selected column using the extension of multivariate synchronization index (EMSI) algorithm, one of the widely used SSVEP detection algorithms. In our experiments with 20 participants wearing a commercial VR-HMD system, it was shown that both the accuracy and ITR of the proposed hybrid BCI were significantly increased compared to those of the traditional SSVEP-based BCI in VR environment.<\/jats:p>","DOI":"10.3389\/fninf.2022.758537","type":"journal-article","created":{"date-parts":[[2022,2,24]],"date-time":"2022-02-24T07:48:24Z","timestamp":1645688904000},"update-policy":"http:\/\/dx.doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Novel Hybrid Brain-Computer Interface for Virtual Reality Applications Using Steady-State Visual-Evoked Potential-Based Brain\u2013Computer Interface and Electrooculogram-Based Eye Tracking for Increased Information Transfer Rate"],"prefix":"10.3389","volume":"16","author":[{"given":"Jisoo","family":"Ha","sequence":"first","affiliation":[]},{"given":"Seonghun","family":"Park","sequence":"additional","affiliation":[]},{"given":"Chang-Hwan","family":"Im","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2022,2,24]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1109\/TNSRE.2009.2039495","article-title":"BCI demographics: how many (and what kinds of) people can use an SSVEP BCI?","volume":"18","author":"Allison","year":"2010","journal-title":"IEEE Trans. Neural. Syst. Rehabil. Eng."},{"key":"B2","doi-asserted-by":"publisher","first-page":"741","DOI":"10.1109\/TPAMI.2010.86","article-title":"Eye movement analysis for activity recognition using electrooculography.","volume":"33","author":"Bulling","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"B3","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1007\/s13534-018-00093-6","article-title":"Wearable EEG and beyond","volume":"9","author":"Casson","year":"2019","journal-title":"Biomed. Eng. Lett."},{"key":"B4","doi-asserted-by":"publisher","first-page":"92","DOI":"10.3390\/computers9040092","article-title":"Recommendations for Integrating a P300-Based Brain\u2013Computer Interface in Virtual Reality Environments for Gaming: An Update.","volume":"9","author":"Cattan","year":"2020","journal-title":"Computers"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.3390\/s19122690","article-title":"Electrooculograms for Human-Computer Interaction: A Review.","volume":"19","author":"Chang","year":"2019","journal-title":"Sensors"},{"key":"B6","doi-asserted-by":"publisher","first-page":"227","DOI":"10.3390\/s16020227","article-title":"Removing the Interdependency between Horizontal and Vertical Eye-Movement Components in Electrooculograms.","volume":"16","author":"Chang","year":"2016","journal-title":"Sensors"},{"key":"B7","doi-asserted-by":"publisher","first-page":"E6058","DOI":"10.1073\/pnas.1508080112","article-title":"High-speed spelling with a noninvasive brain-computer interface.","volume":"112","author":"Chen","year":"2015","journal-title":"Proc. Natl. Acad. Sci. U.S.A."},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1155\/2019\/9680697","article-title":"Comparison of Visual Stimuli for Steady-State Visual Evoked Potential-Based Brain-Computer Interfaces in Virtual Reality Environment in terms of Classification Accuracy and Visual Comfort.","volume":"2019","author":"Choi","year":"2019","journal-title":"Comput. Intell. Neurosci."},{"key":"B9","doi-asserted-by":"crossref","DOI":"10.16910\/jemr.12.1.3","article-title":"Eye Tracking in Virtual Reality.","volume":"12","author":"Clay","year":"2019","journal-title":"J. Eye Mov. Res."},{"key":"B10","doi-asserted-by":"publisher","first-page":"10840","DOI":"10.1109\/ACCESS.2018.2809453","article-title":"Brain-computer interface control in a virtual reality environment and applications for the internet of things.","volume":"6","author":"Coogan","year":"2018","journal-title":"IEEE Access"},{"key":"B11","doi-asserted-by":"publisher","first-page":"1032","DOI":"10.1016\/S1474-4422(08)70223-0","article-title":"Brain\u2013computer interfaces in neurological rehabilitation.","volume":"7","author":"Daly","year":"2008","journal-title":"Lancet Neurol."},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1109\/IEMBS.2010.5627451","article-title":"A Comparison of Monopolar and Bipolar EEG Recordings for SSVEP Detection","author":"Diez","year":"2010","journal-title":"32nd Annual International Conference of the IEEE EMBS."},{"key":"B13","doi-asserted-by":"publisher","first-page":"418","DOI":"10.1007\/978-3-030-20521-8_35","article-title":"A VR-Based Hybrid BCI Using SSVEP and Gesture Input.","author":"Grichnik","year":"2019","journal-title":"Advances in Computational Intelligence, 15th International Work-Conference on Artificial Neural Networks"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2017.00035","article-title":"Hybrid Brain-Computer Interface Techniques for Improved Classification Accuracy and Increased Number of Commands: A Review.","volume":"11","author":"Hong","year":"2017","journal-title":"Front. Neurorobot."},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.16910\/jemr.14.3.3","article-title":"A low-cost, high-performance video-based binocular eye tracker for psychophysical research.","volume":"14","author":"Ivanchenko","year":"2021","journal-title":"J. Eye Mov. Res."},{"key":"B16","doi-asserted-by":"publisher","first-page":"885","DOI":"10.1016\/j.apmr.2019.10.195","article-title":"Effectiveness of Virtual Reality- and Gaming-Based Interventions for Upper Extremity Rehabilitation Poststroke: A Meta-analysis.","volume":"101","author":"Karamians","year":"2020","journal-title":"Arch. Phys. Med. Rehabil."},{"key":"B17","doi-asserted-by":"publisher","first-page":"578","DOI":"10.1109\/jetcas.2011.2178734","article-title":"A Chromatic Transient Visual Evoked Potential Based Encoding\/Decoding Approach for Brain\u2013Computer Interface.","volume":"1","author":"Lai","year":"2011","journal-title":"IEEE J. Emerg. Sel. Top. Circuits Syst."},{"key":"B18","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1109\/TNSRE.2016.2542524","article-title":"Real-time \u201cEye-Writing\u201d recognition using electrooculogram","volume":"25","author":"Lee","year":"2017","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR50242.2020.00084","article-title":"Virtual Reality Racket Sports: Virtual Drills for Exercise and Training","author":"Liu","year":"2020","journal-title":"2020 IEEE International Symposium on Mixed and Augmented Reality"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1016\/j.autcon.2020.103523","article-title":"Brain-computer interface for hands-free teleoperation of construction robots.","volume":"123","author":"Liu","year":"2021","journal-title":"Autom. Const."},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1145\/3172944.3172988","article-title":"Combining Brain-Computer Interface and Eye Tracking for High-Speed Text Entry in Virtual Reality","author":"Ma","year":"2018","journal-title":"23rd International Conference on Intelligent User Interfaces."},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.30","article-title":"A Review of Various State of Art Eye Gaze Estimation Techniques","author":"Modi","year":"2021","journal-title":"Advances in Intelligent Systems and Computing"},{"key":"B23","doi-asserted-by":"publisher","first-page":"2702","DOI":"10.1109\/TVCG.2021.3067687","article-title":"Hands-free interaction in immersive virtual reality: a systematic review.","volume":"27","author":"Monteiro","year":"2021","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B24","doi-asserted-by":"publisher","first-page":"482","DOI":"10.1007\/s40846-015-0056-1","article-title":"Monopolar and Bipolar Electrode Settings for SSVEP-Based Brain-Computer Interface.","volume":"35","author":"M\u00fcller","year":"2015","journal-title":"J. Med. Biol. Eng."},{"key":"B25","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1109\/THMS.2018.2830647","article-title":"A Survey on Unmanned Aerial Vehicle Remote Control Using Brain\u2013Computer Interface.","volume":"48","author":"Nourmohammadi","year":"2018","journal-title":"IEEE Trans. Hum.Mach. Syst."},{"key":"B26","doi-asserted-by":"publisher","first-page":"163604","DOI":"10.1109\/BCI48061.2020.9061633","article-title":"Development of an Online Home Appliance Control System Using Augmented Reality and an SSVEP-Based Brain\u2013Computer Interface.","volume":"7","author":"Park","year":"2019","journal-title":"IEEE Access"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.3389\/fnsys.2021.578875","article-title":"Progress in Brain Computer Interface: Challenges and Opportunities.","volume":"15","author":"Saha","year":"2021","journal-title":"Front. Syst. Neurosci."},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1016\/j.aei.2019.100994","article-title":"A high performance hybrid SSVEP based BCI speller system.","volume":"42","author":"Saravanakumar","year":"2019","journal-title":"Adv. Eng. Inform."},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1016\/j.aei.2020.101059","article-title":"A virtual speller system using SSVEP and electrooculogram.","volume":"44","author":"Saravanakumar","year":"2020","journal-title":"Adv. Eng. Inform."},{"key":"B30","doi-asserted-by":"publisher","first-page":"661","DOI":"10.1016\/j.dsx.2020.05.011","article-title":"Significant applications of virtual reality for COVID-19 pandemic.","volume":"14","author":"Singh","year":"2020","journal-title":"Diabetes Metab. Syndr."},{"key":"B31","doi-asserted-by":"publisher","first-page":"180","DOI":"10.3390\/healthcare9020180","article-title":"Eye-Tracking for Clinical Ophthalmology with Virtual Reality (VR): A Case Study of the HTC Vive Pro Eye\u2019s Usability.","volume":"9","author":"Sipatchin","year":"2021","journal-title":"Healthcare"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1109\/SMC.2018.00749","article-title":"SSVEP-based BCI in virtual reality - control of a vacuum cleaner robot","author":"Stawicki","year":"2018","journal-title":"Proceedings of the 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC)"},{"key":"B33","doi-asserted-by":"publisher","first-page":"35","DOI":"10.3390\/brainsci7040035","article-title":"A Novel Hybrid Mental Spelling Application Based on Eye Tracking and SSVEP-Based BCI.","volume":"7","author":"Stawicki","year":"2017","journal-title":"Brain Sci."},{"key":"B34","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1007\/978-3-030-35252-3_3","article-title":"Assisting Students to Understand Mathematical Graphs Using Virtual Reality Application","author":"Sundaram","year":"2020","journal-title":"Advanced Computational Intelligence Techniques for Virtual Reality in Healthcare"},{"key":"B35","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1088\/1741-2560\/10\/4\/046003","article-title":"Brain\u2013computer interface","author":"Vallabhaneni","year":"2013","journal-title":"Neural Engineering"},{"key":"B36","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1080\/10447318.2019.1612213","article-title":"Brain\u2013Computer Interface Games Based on Consumer-Grade EEG Devices: A Systematic Literature Review.","volume":"36","author":"Vasiljevic","year":"2019","journal-title":"Int. J. Hum.Comp. Interact."},{"key":"B37","doi-asserted-by":"publisher","first-page":"326","DOI":"10.1109\/86.712231","article-title":"EEG-Based Communication: Improved Accuracy by Response Verification.","volume":"6","author":"Wolpaw","year":"1998","journal-title":"IEEE Trans. Neural. Syst. Rehabil. Eng."},{"key":"B38","doi-asserted-by":"publisher","first-page":"2859","DOI":"10.1587\/transinf.2018EDL8122","article-title":"High-Speed Spelling in Virtual Reality with Sequential Hybrid BCIs.","author":"Yao","year":"2018","journal-title":"IEICE Trans. Inf. Syst."},{"key":"B39","doi-asserted-by":"publisher","first-page":"693","DOI":"10.1109\/TNSRE.2015.2403270","article-title":"A Hybrid Brain-Computer Interface Based on the Fusion of P300 and SSVEP Scores.","volume":"23","author":"Yin","year":"2015","journal-title":"IEEE Trans. Neural. Syst. Rehabil. Eng."},{"key":"B40","doi-asserted-by":"publisher","first-page":"226","DOI":"10.3389\/fnins.2020.00717","article-title":"The extension of multivariate synchronization index method for SSVEP-based BCI.","volume":"269","author":"Zhang","year":"2017","journal-title":"Neurocomputing"},{"key":"B41","doi-asserted-by":"publisher","first-page":"2881","DOI":"10.1109\/TBME.2020.2972747","article-title":"A Hybrid Asynchronous Brain-Computer Interface Combining SSVEP and EOG Signals.","volume":"67","author":"Zhou","year":"2020","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2020.583641","article-title":"A Hybrid BCI Based on SSVEP and EOG for Robotic Arm Control.","volume":"14","author":"Zhu","year":"2020","journal-title":"Front. Neurorobot."}],"container-title":["Frontiers in Neuroinformatics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fninf.2022.758537\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,19]],"date-time":"2024-09-19T04:28:19Z","timestamp":1726720099000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fninf.2022.758537\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,24]]},"references-count":42,"alternative-id":["10.3389\/fninf.2022.758537"],"URL":"https:\/\/doi.org\/10.3389\/fninf.2022.758537","relation":{},"ISSN":["1662-5196"],"issn-type":[{"type":"electronic","value":"1662-5196"}],"subject":[],"published":{"date-parts":[[2022,2,24]]}}}