Abstract
As deep learning research develops and models become larger and more complex, there are increasing concerns to deep learning about its’ low ability of explanations to humans and its black box characteristics. Model visualization research of DL (Deep Learning) has been attracting attention for a decade for a solution to this concern. VR (Virtual Reality) interaction research between humans and models is a practical means of model visualization research with great potential but is still in the early stages. The purpose of our study is to propose new methods for VR technology to contribute to the development of deep learning models by investigating and implementing the visualization technology of deep learning and VR research projects. In this paper, we also report our two experimental results. One is a web survey using demo images of a PC application and a VR goggles application, and the other is an evaluation experiment using VR goggles.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Liu, S., Wang, X., Liu, M., Zhu, J.: Towards better analysis of machine learning models: a visual analytics perspective. Visual Inform. 1(1), 48–56 (2017)
Zeiler, M., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53
Meissler, N., Wohlan, A., Hochgeschwender, N., Schreiber, A.: Using visualization of convolutional neural networks in virtual reality for machine learning newcomers (2019)
Choo, J., Liu, S.: Visual analytics for explainable deep learning. IEEE Comput. Graph. Appl. 38(4), 84–92 (2018)
Yuan, J., Chen, C., Yang, W., Liu, M., Xia, J., Liu, S.: A survey of visual analytics techniques for machine learning. Comput. Visual Media 7(1), 3–36 (2020). https://doi.org/10.1007/s41095-020-0191-7
Chen, C., et al.: Oodanalyzer: interactive analysis of out-of-distribution samples. IEEE Trans. Visual Comput. Graph. 27(7), 3335–3349 (2020)
Liu, S., Chen, C., Lu, Y., Ouyang, F., Wang, B.: An interactive method to improve crowdsourced annotations. IEEE Trans. Visual Comput. Graph. 25(1), 235–245 (2018)
Liu, M., Shi, J., Li, Z., Li, C., Zhu, J., Liu, S.: Towards better analysis of deep convolutional neural networks. IEEE Trans. Visual Comput. Graph. 23(1), 91–100 (2016)
Liu, M., Liu, S., Su, H., Cao, K., Zhu, J.: Analyzing the noise robustness of deep neural networks. In: 2018 IEEE Conference on Visual Analytics Science and Technology (VAST), pp. 60–71. IEEE (2018)
Yang, W., Wang, X., Lu, J., Dou, W., Liu, S.: Interactive steering of hierarchical clustering. IEEE Trans. Visual. Comput. Graph. (2020)
Jänicke, H., Borgo, R., Mason, J.S., Chen, M.: SoundRiver: semantically‐rich sound illustration. In: Computer Graphics Forum, vol. 29, no. 2, pp. 357–366. Blackwell Publishing Ltd., Oxford (2010)
Cui, W., et al.: Textflow: towards better understanding of evolving topics in text. IEEE Trans. Visual Comput. Graph. 17(12), 2412–2421 (2011)
Schuemie, M.J., Van Der Straaten, P., Krijn, M., Van Der Mast, C.A.: Research on presence in virtual reality: a survey. Cyberpsychol. Behav. 4(2), 183–201 (2001)
Miller, H.L., Bugnariu, N.L.: Level of immersion in virtual environments impacts the ability to assess and teach social skills in autism spectrum disorder. Cyberpsychol. Behav. Soc. Netw. 19(4), 246–256 (2016)
Sacks, R., Perlman, A., Barak, R.: Construction safety training using immersive virtual reality. Constr. Manag. Econ. 31(9), 1005–1017 (2013)
Witmer, B.G., Singer, M.J.: Measuring presence in virtual environments: a presence questionnaire. Presence 7(3), 225–240 (1998)
VanHorn, K.C., Zinn, M., Cobanoglu, M.C.: Deep learning development environment in virtual reality. arXiv preprint arXiv:1906.05925 (2019)
Hohman, F., et al.: Visual analytics in deep learning: an interrogative survey for the next frontiers. IEEE Trans. Visual Comput. Graph. 25(8), 2674–2693 (2018)
Bau, D., Zhou, B., Khosla, A., Oliva, A., Torralba, A.: Network dissection: quantifying interpretability of deep visual representations. In: CVPR (2017)
Tsai, C.-Y., Cox, D.D.: Characterizing visual representations within convolutional neural networks: toward a quantitative approach. In: ICML Workshop on Vis for Deep Learning (2016)
Deng, J., Guo, J., Xue, N., Zafeiriou, S.: Arcface: additive angular margin loss for deep face recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4690–4699 (2019)
Chollet, F., et al.: Keras. https://keras.io (2015)
Acknowledgement
We are grateful to all participants of our experiments. This work was supported by a Grant-in-Aid for Scientific Research (B) (No. 20H04476).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Naraha, T., Akimoto, K., Yairi, I.E. (2022). Survey of the VR Environment for Deep Learning Model Development. In: Takama, Y., et al. Advances in Artificial Intelligence. JSAI 2021. Advances in Intelligent Systems and Computing, vol 1423. Springer, Cham. https://doi.org/10.1007/978-3-030-96451-1_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-96451-1_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-96450-4
Online ISBN: 978-3-030-96451-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)