{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,12,30]],"date-time":"2024-12-30T19:02:15Z","timestamp":1735585335560},"reference-count":61,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2021,8,27]],"date-time":"2021-08-27T00:00:00Z","timestamp":1630022400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2019R1F1A1058844","2021R1I1A3060828"],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Program for Excellence in Software at Handong Global University","award":["2017-0-00130"]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"Since the emergence of head-mounted displays (HMDs), researchers have attempted to introduce virtual and augmented reality (VR, AR) in brain\u2013computer interface (BCI) studies. However, there is a lack of studies that incorporate both AR and VR to compare the performance in the two environments. Therefore, it is necessary to develop a BCI application that can be used in both VR and AR to allow BCI performance to be compared in the two environments. In this study, we developed an opensource-based drone control application using P300-based BCI, which can be used in both VR and AR. Twenty healthy subjects participated in the experiment with this application. They were asked to control the drone in two environments and filled out questionnaires before and after the experiment. We found no significant (p > 0.05) difference in online performance (classification accuracy and amplitude\/latency of P300 component) and user experience (satisfaction about time length, program, environment, interest, difficulty, immersion, and feeling of self-control) between VR and AR. This indicates that the P300 BCI paradigm is relatively reliable and may work well in various situations.<\/jats:p>","DOI":"10.3390\/s21175765","type":"journal-article","created":{"date-parts":[[2021,9,1]],"date-time":"2021-09-01T02:58:15Z","timestamp":1630465095000},"page":"5765","source":"Crossref","is-referenced-by-count":36,"title":["P300 Brain\u2013Computer Interface-Based Drone Control in Virtual and Augmented Reality"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-5892-1537","authenticated-orcid":false,"given":"Soram","family":"Kim","sequence":"first","affiliation":[{"name":"School of Computer Science and Electrical Engineering, Handong Global University, Pohang 37554, Korea"}]},{"given":"Seungyun","family":"Lee","sequence":"additional","affiliation":[{"name":"School of Computer Science and Electrical Engineering, Handong Global University, Pohang 37554, Korea"}]},{"given":"Hyunsuk","family":"Kang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Electrical Engineering, Handong Global University, Pohang 37554, Korea"}]},{"given":"Sion","family":"Kim","sequence":"additional","affiliation":[{"name":"School of Computer Science and Electrical Engineering, Handong Global University, Pohang 37554, Korea"}]},{"ORCID":"http:\/\/orcid.org\/0000-0001-6512-0167","authenticated-orcid":false,"given":"Minkyu","family":"Ahn","sequence":"additional","affiliation":[{"name":"School of Computer Science and Electrical Engineering, Handong Global University, Pohang 37554, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Wolpaw, J., and Wolpaw, E.W. (2012). Brain-Computer Interfaces: Principles and Practice, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780195388855.001.0001"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1608","DOI":"10.1109\/TVCG.2018.2873737","article-title":"Towards BCI-Based Interfaces for Augmented Reality: Feasibility, Design and Evaluation","volume":"26","author":"Petit","year":"2020","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_3","first-page":"14","article-title":"P300 Brain Computer Interface: Current Challenges and Emerging Trends","volume":"5","author":"Allison","year":"2012","journal-title":"Front. Neuroeng."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"510","DOI":"10.1016\/0013-4694(88)90149-6","article-title":"Talking off the Top of Your Head: Toward a Mental Prosthesis Utilizing Event-Related Brain Potentials","volume":"70","author":"Farwell","year":"1988","journal-title":"Electroencephalogr. Clin. Neurophysiol."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Cattan, G., Andreev, A., and Visinoni, E. (2020). Recommendations for Integrating a P300-Based Brain\u2013Computer Interface in Virtual Reality Environments for Gaming: An Update. Computers, 9.","DOI":"10.3390\/computers9040092"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Aamer, A., Esawy, A., Swelam, O., Nabil, T., Anwar, A., and Eldeib, A. (2019, January 15\u201318). BCI Integrated with VR for Rehabilitation. Proceedings of the 2019 31st International Conference on Microelectronics (ICM), Cairo, Egypt.","DOI":"10.1109\/ICM48031.2019.9021752"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"13675","DOI":"10.1007\/s11042-018-6916-2","article-title":"Developing and Evaluating a BCI Video Game for Neurofeedback Training: The Case of Autism","volume":"78","author":"Mercado","year":"2019","journal-title":"Multimed. Tools Appl."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"McMahon, M., and Schukat, M. (2018, January 21\u201322). A Low-Cost, Open-Source, BCI-VR Prototype for Real-Time Signal Processing of EEG to Manipulate 3D VR Objects as a Form of Neurofeedback. Proceedings of the 2018 29th Irish Signals and Systems Conference (ISSC), Belfast, UK.","DOI":"10.1109\/ISSC.2018.8585373"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1140\/epjnbp\/s40366-015-0027-z","article-title":"BCI inside a Virtual Reality Classroom: A Potential Training Tool for Attention","volume":"3","author":"Rohani","year":"2015","journal-title":"EPJ Nonlinear Biomed. Phys."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ali, A., and Puthusserypady, S. (2015, January 25\u201329). A 3D Learning Playground for Potential Attention Training in ADHD: A Brain Computer Interface Approach. Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy.","DOI":"10.1109\/EMBC.2015.7318302"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Arpaia, P., De Benedetto, E., Donato, N., Duraccio, L., and Moccaldi, N. (2021, January 23\u201325). A Wearable SSVEP BCI for AR-Based, Real-Time Monitoring Applications. Proceedings of the 2021 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Lausanne, Switzerland.","DOI":"10.1109\/MeMeA52024.2021.9478593"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"6362","DOI":"10.1109\/TIM.2020.2970846","article-title":"Wearable Brain\u2013Computer Interface Instrumentation for Robot-Based Rehabilitation by Augmented Reality","volume":"69","author":"Arpaia","year":"2020","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Park, S., Cha, H., Kwon, J., Kim, H., and Im, C. (2020, January 26\u201328). Development of an Online Home Appliance Control System Using Augmented Reality and an SSVEP-Based Brain-Computer Interface. Proceedings of the 2020 8th International Winter Conference on Brain-Computer Interface (BCI), Gangwon, Korea.","DOI":"10.1109\/BCI48061.2020.9061633"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"26789","DOI":"10.1109\/ACCESS.2018.2825378","article-title":"A Wearable SSVEP-Based BCI System for Quadcopter Control Using Head-Mounted Device","volume":"6","author":"Wang","year":"2018","journal-title":"IEEE Access"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Kerous, B., and Liarokapis, F. (2017, January 9\u201313). BrainChat\u2014A Collaborative Augmented Reality Brain Interface for Message Communication. Proceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), Nantes, France.","DOI":"10.1109\/ISMAR-Adjunct.2017.91"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"60","DOI":"10.3389\/fnbot.2017.00060","article-title":"Closed-Loop Hybrid Gaze Brain-Machine Interface Based Robotic Arm Control with Augmented Reality Feedback","volume":"11","author":"Zeng","year":"2017","journal-title":"Front. Neurorobot."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1622","DOI":"10.1109\/TNSRE.2016.2626391","article-title":"Local and Remote Cooperation With Virtual and Robotic Agents: A P300 BCI Study in Healthy and People Living with Spinal Cord Injury","volume":"25","author":"Tidoni","year":"2017","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_18","first-page":"156","article-title":"Multimodal System for Training at Distance in a Virtual or Augmented Reality Environment for Users of Electric-Powered Wheelchairs","volume":"49","author":"Borges","year":"2016","journal-title":"IFAC Pap."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1996","DOI":"10.1109\/TITS.2013.2266135","article-title":"A Head-Up Display-Based P300 Brain\u2013Computer Interface for Destination Selection","volume":"14","author":"Bi","year":"2013","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Martens, N., Jenke, R., Abu-Alqumsan, M., Kapeller, C., Hinterm\u00fcller, C., Guger, C., Peer, A., and Buss, M. (2012, January 7\u201312). Towards Robotic Re-Embodiment Using a Brain-and-Body-Computer Interface. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6386259"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"60","DOI":"10.3389\/fnins.2011.00060","article-title":"Towards Intelligent Environments: An Augmented Reality\u2013Brain\u2013Machine Interface Operated with a See-Through Head-Mount Display","volume":"5","author":"Takano","year":"2011","journal-title":"Front. Neurosci."},{"key":"ref_22","unstructured":"Faller, J., Leeb, R., Pfurtscheller, G., and Scherer, R. (2010, January 14\u201316). Avatar Navigation in Virtual and Augmented Reality Environments Using an SSVEP BCI. Proceedings of the 1st International Conference on Apllied Bionics and Biomechanics (ICABB-2010), Venice, Italy."},{"key":"ref_23","unstructured":"Wong, K.W., Mendis, B.S.U., and Bouzerdoum, A. (2010). An Augmented-Reality Based Brain-Computer Interface for Robot Control. Neural Information Processing. Models and Applications, Springer."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"016066","DOI":"10.1088\/1741-2552\/ab4dc6","article-title":"An Online SSVEP-BCI System in an Optical See-through Augmented Reality Environment","volume":"17","author":"Ke","year":"2020","journal-title":"J. Neural Eng."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Barrera, A. (2011). Advances in Robot Navigation, BoD\u2013Books on Demand.","DOI":"10.5772\/705"},{"key":"ref_26","first-page":"388","article-title":"An Open Source-Based BCI Application for Virtual World Tour and Its Usability Evaluation","volume":"18","author":"Woo","year":"2021","journal-title":"Front. Hum. Neurosci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1329","DOI":"10.1016\/j.neunet.2009.07.003","article-title":"The MindGame: A P300-Based Brain\u2013Computer Interface Game","volume":"22","author":"Finke","year":"2009","journal-title":"Neural Netw."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1111\/j.1469-8986.2010.01054.x","article-title":"Mock Crime Application of the Complex Trial Protocol (CTP) P300-Based Concealed Information Test","volume":"48","author":"Winograd","year":"2011","journal-title":"Psychophysiol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"715","DOI":"10.1109\/TNSRE.2016.2591012","article-title":"A P300-Based Threshold-Free Brain Switch and Its Application in Wheelchair Control","volume":"25","author":"He","year":"2017","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1079","DOI":"10.1016\/j.medengphy.2008.01.004","article-title":"Stimulator Selection in SSVEP-Based BCI","volume":"30","author":"Wu","year":"2008","journal-title":"Med. Eng. Phys."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"344","DOI":"10.1109\/TNSRE.2017.2734164","article-title":"High Frequency SSVEP-BCI With Hardware Stimuli Control and Phase-Synchronized Comb Filter","volume":"26","author":"Chabuda","year":"2018","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1162\/pres.19.1.35","article-title":"OpenViBE: An Open-Source Software Platform to Design, Test, and Use Brain\u2013Computer Interfaces in Real and Virtual Environments","volume":"19","author":"Renard","year":"2010","journal-title":"Presence Teleoperators Virtual Environ."},{"key":"ref_33","unstructured":"(2021, March 23). Unity at 10: For Better\u2014Or Worse\u2014Game Development Has Never Been Easier|Ars Technica. Available online: https:\/\/arstechnica.com\/gaming\/2016\/09\/unity-at-10-for-better-or-worse-game-development-has-never-been-easier\/."},{"key":"ref_34","unstructured":"Komori, A. (2021, March 26). Comoc\/TelloForUnity. Available online: https:\/\/github.com\/comoc\/TelloForUnity."},{"key":"ref_35","unstructured":"BCILab (2021, March 26). AhnBCILab\/BCI_Drone. Available online: https:\/\/github.com\/AhnBCILab\/BCI_Drone."},{"key":"ref_36","unstructured":"(2021, March 22). Scipy. Signal. Butter\u2014SciPy v1.6.1 Reference Guide. Available online: https:\/\/docs.scipy.org\/doc\/scipy\/reference\/generated\/scipy.signal.butter.html."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"R1","DOI":"10.1088\/1741-2560\/4\/2\/R01","article-title":"Review of Classification Algorithms for EEG-Based Brain\u2013Computer Interfaces","volume":"4","author":"Lotte","year":"2007","journal-title":"J. Neural Eng."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1088\/1741-2560\/3\/4\/007","article-title":"A Comparison of Classification Techniques for the P300 Speller","volume":"3","author":"Krusienski","year":"2006","journal-title":"J. Neural Eng."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1016\/S0167-8760(97)00772-1","article-title":"On the Relationship between EEG and P300: Individual Differences, Aging, and Ultradian Rhythms","volume":"26","author":"Polich","year":"1997","journal-title":"Int. J. Psychophysiol."},{"key":"ref_40","unstructured":"(2021, March 29). DSI VR300. Available online: https:\/\/wearablesensing.com\/products\/vr300\/."},{"key":"ref_41","unstructured":"(2021, March 29). Tello. Available online: https:\/\/www.ryzerobotics.com\/kr\/tello."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1111\/1469-8986.3650583","article-title":"Substance Dependence and Externalizing Psychopathology in Adolescent Boys with Small, Average, or Large P300 Event-Related Potential Amplitude","volume":"36","author":"Carlson","year":"1999","journal-title":"Psychophysiology"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1016\/j.clinph.2015.04.054","article-title":"Effects of Training and Motivation on Auditory P300 Brain\u2013Computer Interface Performance","volume":"127","author":"Baykara","year":"2016","journal-title":"Clin. Neurophysiol."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing","volume":"57","author":"Benjamini","year":"1995","journal-title":"J. R. Stat. Soc. Ser. B Methodol."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"870","DOI":"10.1006\/nimg.2001.1037","article-title":"Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate","volume":"15","author":"Genovese","year":"2002","journal-title":"NeuroImage"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"261","DOI":"10.3389\/fnhum.2019.00261","article-title":"P300 Speller Performance Predictor Based on RSVP Multi-Feature","volume":"13","author":"Won","year":"2019","journal-title":"Front. Hum. Neurosci."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"59","DOI":"10.3389\/fnhum.2018.00059","article-title":"User\u2019s Self-Prediction of Performance in Motor Imagery Brain\u2013Computer Interface","volume":"12","author":"Ahn","year":"2018","journal-title":"Front. Hum. Neurosci."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Halder, S., Hammer, E.M., Kleih, S.C., Bogdan, M., Rosenstiel, W., Birbaumer, N., and K\u00fcbler, A. (2013). Prediction of Auditory and Visual P300 Brain-Computer Interface Aptitude. PLoS ONE, 8.","DOI":"10.1371\/journal.pone.0053513"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"066017","DOI":"10.1088\/1741-2560\/10\/6\/066017","article-title":"Prediction of SSVEP-Based BCI Performance by the Resting-State EEG Network","volume":"10","author":"Zhang","year":"2013","journal-title":"J. Neural Eng."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"298","DOI":"10.1109\/TSMC.2020.3041382","article-title":"Direct-Sense Brain\u2013Computer Interfaces and Wearable Computers","volume":"51","author":"Lin","year":"2021","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Allison, B.Z., Dunne, S., Leeb, R., Del R. Mill\u00e1n, J., and Nijholt, A. (2013). Combining BCI with Virtual Reality: Towards New Applications and Improved BCI. Towards Practical Brain-Computer Interfaces: Bridging the Gap from Research to Real-World Applications, Springer. Biological and Medical Physics, Biomedical Engineering.","DOI":"10.1007\/978-3-642-29746-5"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"144","DOI":"10.3389\/fnhum.2020.00144","article-title":"Editorial: Brain-Computer Interfaces and Augmented\/Virtual Reality","volume":"14","author":"Putze","year":"2020","journal-title":"Front. Hum. Neurosci."},{"key":"ref_53","first-page":"368","article-title":"A Comparison of the Power of the Paired Samples Rank Transform Statistic to That of Wilcoxon\u2019s Signed Ranks Statistic","volume":"10","author":"Blair","year":"1985","journal-title":"J. Educ. Stat."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"9","DOI":"10.22237\/jmasm\/1177992480","article-title":"Reliability and Statistical Power: How Measurement Fallibility Affects Power and Required Sample Sizes for Several Parametric and Nonparametric Statistics","volume":"6","author":"Kanyongo","year":"2007","journal-title":"J. Mod. Appl. Stat. Methods"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Lotte, F., Bougrain, L., and Clerc, M. (2015). Electroencephalography (EEG)-Based Brain-Computer Interfaces. Wiley Encyclopedia of Electrical and Electronics Engineering, John Wiley & Sons, Inc.","DOI":"10.1002\/047134608X.W8278"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"74385","DOI":"10.1109\/ACCESS.2020.2988057","article-title":"CNN with Large Data Achieves True Zero-Training in Online P300 Brain-Computer Interface","volume":"8","author":"Lee","year":"2020","journal-title":"IEEE Access"},{"key":"ref_57","unstructured":"Lotte, F., and Guan, C. (2009, January 10). An Efficient P300-Based Brain-Computer Interface with Minimal Calibration Time. Proceedings of the Assistive Machine Learning for People with Disabilities symposium (NIPS\u201909 Symposium), Vancouver, BC, Canada."},{"key":"ref_58","first-page":"123","article-title":"Adaptive Training Session for a P300 Speller Brain\u2013Computer Interface","volume":"105","author":"Rivet","year":"2011","journal-title":"J. Physiol. Paris"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"5391","DOI":"10.1002\/hbm.23730","article-title":"Deep Learning with Convolutional Neural Networks for EEG Decoding and Visualization","volume":"38","author":"Schirrmeister","year":"2017","journal-title":"Hum. Brain Mapp."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"25","DOI":"10.3389\/fnbot.2020.00025","article-title":"Current Status, Challenges, and Possible Solutions of EEG-Based Brain-Computer Interface: A Comprehensive Review","volume":"14","author":"Rashid","year":"2020","journal-title":"Front. Neurorobot."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"056013","DOI":"10.1088\/1741-2552\/aace8c","article-title":"EEGNet: A Compact Convolutional Neural Network for EEG-Based Brain\u2013Computer Interfaces","volume":"15","author":"Lawhern","year":"2018","journal-title":"J. Neural Eng."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/17\/5765\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,17]],"date-time":"2024-07-17T19:29:39Z","timestamp":1721244579000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/17\/5765"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,27]]},"references-count":61,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2021,9]]}},"alternative-id":["s21175765"],"URL":"https:\/\/doi.org\/10.3390\/s21175765","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,27]]}}}