Abstract
Aiming at the low detection and recognition rate of small and medium size targets in the current brain-controlled manipulator, this article proposes a method to solve the problem. Firstly, the self-made data set is expanded by using data enhancement technology and improving the robustness of the model, then improve the Faster-RCNN model by reducing the anchor boxes size, the mAP of the model has been increased by 2.62% on the original basis. After the improvement, combined with Augmented Reality (AR) technology to build a brain-controlled manipulator system. AR is used as a visual stimulator, The target position information is obtained through the improved target detection model, and the EEG signal of Steady-State Visual Evoked Potential (SSVEP) is recognized by Filter Bank Canonical Correlation Analysis (FBCCA), the grasping of the manipulator is controlled by decoding the EEG. 10 subjects participating in the grasping experiment, according to the experimental results, the grasping accuracy of the brain-controlled manipulator system is 92%, which verifies the effectiveness and portability of the system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wang, Y., Chen, X., Gao, X., et al.: A benchmark dataset for SSVEP-based brain–computer interfaces. IEEE Trans. Neural Syst. Rehabil. Eng. 25(10), 1746–1752 (2017)
Hortal, E., Planelles, D., Costa, A., et al.: SVM-based brain-machine interface for controlling a robot arm through four mental tasks. Neurocomputing 151, 116–121 (2015)
Chen, X.G., Zhao, B., Liu, M., et al.: Design and implementation of controlling robotic arms using steady-state visual evoked potential brain-computer interface. Biomedical Engineering and Clinical Medicine (2018)
Chen, X., Zhao, B., Wang, Y., et al.: Control of a 7-DOF robotic arm system with an SSVEP-based BCI. Int. J. Neural Syst. 28(8), 1850018 (2018)
Ge, S., Jiang, Y., Wang, P., et al.: Training-free steady-state visual evoked potential brain-computer interface based on filter bank canonical correlation analysis and spatiotemporal beamforming decoding. IEEE Trans. Neural Syst. Rehabil. Eng. 27(9), 1714–1723 (2019)
Zhang, R., Li, Y., Yan, Y., et al.: Control of a wheelchair in an indoor environment based on a brain-computer interface and automated navigation. IEEE Trans. Neural Syst. Rehabil. Eng. 24(1), 128–139 (2016)
Parmar, P., Joshi, V.G.A., Joshi, A.: Brain-computer interface: a review. In: Nirma University International Conference on Engineering. IEEE (2016)
Jeong, J.H., Shim, K.H., Kim, D.J., et al.: Brain-controlled robotic arm system based on multi-directional CNN-BiLSTM network using EEG signals. IEEE Trans. Neural Syst. Rehabil. Eng. 28(5), 1226–1238 (2020)
Li, L., Zeng, P.: Apple target detection based on improved faster-RCNN framework of deep learning. Mach. Des. Res. 35, 24–27 (2019)
Fan, T.: Research and realization of video target detection system based on deep learning. Int. J. Wavelets Multiresolut. Inf. Process. 18(1), 280–292 (2020)
Lv, S., Liu, K., Qiao, Y., et al.: Automatic defect detection based on improved faster RCNN for substation equipment. J. Phys. Conf. Ser. 1544, 012157 (2020)
Harianto, R.A., Pranoto, Y.M., Gunawan, T.P.: Data augmentation and faster RCNN improve vehicle detection and recognition. In: 2021 3rd East Indonesia Conference on Computer and Information Technology (2021)
Jiang, D., Li, G., Tan, C., et al.: Semantic segmentation for multiscale target based on object recognition using the improved Faster-RCNN model. Future Gener. Comput. Syst. 123(1), 94–104 (2021)
Mo, N., Yan, L.: Improved faster RCNN based on feature amplification and oversampling data augmentation for oriented vehicle detection in aerial images. Remote Sensing 12(16), 2558 (2020). https://doi.org/10.3390/rs12162558
Ke, Y., Liu, P., An, X., Song, X., Ming, D.: An online SSVEP-BCI system in an optical see-through augmented reality environment. J. Neural Eng. 17(1), 016066 (2020). https://doi.org/10.1088/1741-2552/ab4dc6
Shi, N., Wang, L., Chen, Y., et al.: Steady-state visual evoked potential (SSVEP)-based brain–computer interface (BCI) of Chinese speller for a patient with amyotrophic lateral sclerosis: a case report. J. Neurorestoratology 8(1), 40–52 (2020)
Lin, C.T., Chiu, C.Y., Singh, A.K., et al.: A wireless multifunctional SSVEP-based brain-computer interface assistive system. IEEE Trans. Cogn. Dev. Syst. 11(3), 375–383 (2019)
Acknowledgements
This project is supported by the National Natural Science Foundation of China (No. 61976133), Major scientific and technological innovation projects of Shan Dong Province (No. 2019JZZY021010), Shanghai Industrial Collaborative Technology Innovation Project (No. 2021-cyxt1-kj14), National Defense Basic Scientific Research Program of China (JCKY2021413B002).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Huang, Y., Yang, B., Wang, Z., Yao, Y., Xu, M., Xia, X. (2023). Brain Controlled Manipulator System Based on Improved Target Detection and Augmented Reality Technology. In: Ying, X. (eds) Human Brain and Artificial Intelligence. HBAI 2022. Communications in Computer and Information Science, vol 1692. Springer, Singapore. https://doi.org/10.1007/978-981-19-8222-4_15
Download citation
DOI: https://doi.org/10.1007/978-981-19-8222-4_15
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-8221-7
Online ISBN: 978-981-19-8222-4
eBook Packages: Computer ScienceComputer Science (R0)