Abstract
English End-to-end spoken keyword systems (KWS) with limited keywords are commonly available in the literature. This paper aims to study the existing various keyword techniques in the Indian regional Bengali language under low-resource conditions. In this context, we study several KWS techniques which are common in the English language in Bengali namely: Conv1D, Conv2D+attention, Conv2D+multi head attention, VGG, Dense-net, and Vision transformer (ViT). In addition, we also study the effect of voice-activity detection (VAD) on the KWS under real-life scenarios especially when the speech signal could contain the front and tail short pause or silence i.e. without proper segmentation information even under clean conditions. Besides, we also consider cross-lingual transfer learning for tuning the parameters of a pre-trained state-of-the-art transformer model in English to Bengali. Finally, Experimental results demonstrate that VAD significantly improves the accuracy of the KWS detection system using both spectral features and raw audio data. Among the different traditional approaches (without transfer learning), the Densenet technique yields better system accuracy. Overall, cross-lingual transfer learning provides the highest KWS detection than others.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abdoli, S., Cardinal, P., Koerich, A.L.: End-to-end environmental sound classification using a 1D convolutional neural network. arXiv:1904.08990 (2019)
Basu, J., Bepari, M.S., Roy, R., Khan, S.: Design of telephonic speech data collection and transcription methodology for speech recognition systems. In: Proceedings of FRSM, pp. 147–153. India (2012)
Basu, J., Bepari, M.S., Roy, R., Khan, S.: Real time challenges to handle the telephonic speech recognition system. In: S, M., Kumar, S. (eds.) Proceedings of the Fourth International Conference on Signal and Image Processing (ICSIP), vol. 222, pp. 395–408. Springer, India (2013). https://doi.org/10.1007/978-81-322-1000-9_38
Berg, A., O’Connor, M., Cruz, M.T.: Keyword transformer: a self-attention model for keyword spotting. In: Proceedings of Interspeech, pp. 4249–4253 (2021)
Brian, M., et al.: librosa: audio and music signal analysis in python. In: Proceedings of the 14th Python in Science Conference, pp. 18–25 (2015)
D. Bruyne, L., Singh, P., D. Clercq, O., Lefever, E., Hoste, V.: How language-dependent is emotion detection? Evidence from multilingual BERT. In: Proceedings of the 2nd Workshop on Multi-lingual Representation Learning (MRL), pp. 76–85. Association for Computational Linguistics (2022)
Du, X., Zhu, M., Chai, M., Shi, X.: End to end model for keyword spotting with trainable window function and Densenet. In: Proceedings of IEEE International Conference on Digital Signal Processing, pp. 1–5 (2018)
Hsu, W.N., Bolte, B., Tsai, Y.H.H., Lakhotia, K., Salakhutdinov, R., Mohamed, A.: HuBERT: self-supervised speech representation learning by masked prediction of hidden units. arXiv:2106.07447 (2021)
Pastor, M., Ribas, D., Ortega, A., Miguel, A., Lleida, E.: Cross-corpus speech emotion recognition with HuBERT self-supervised representation. In: Proceedings of Interspeech, pp. 76–80 (2022)
Povey, D., et al.: The kaldi speech recognition toolkit. In: Proceedings of IEEE Workshop on Automatic Speech Recognition and Understanding (2011)
Rybakov, O., Kononenko, N., Subrahmanya, N., Visontai, M., Laurenzo, S.: Streaming keyword spotting on mobile devices. In: Proceedings of Interspeech, pp. 2277–2281 (2020)
Shan, C., Zhang, J., Wang, Y., Xie, L.: Attention-based end-to-end models for small-footprint keyword spotting. arXiv:1803.10916 (2018)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2015)
Tan, Z.H., Sarkar, A.K., Dehak, N.: rVAD: an unsupervised segment-based robust voice activity detection method. Comput. Speech Lang. 59, 1–21 (2020)
Uçan, A., Dörterler, M., Akçapınar Sezer, E.: A study of Turkish emotion classification with pretrained language models. J. Inf. Sci. 48(6), 857–865 (2022)
Warden, P.: Speech commands: a dataset for limited-vocabulary speech recognition. arXiv:1804.03209 (2018)
Yang, J.: Ensemble deep learning with HuBERT for speech emotion recognition. In: Proceedings of IEEE 17th International Conference on Semantic Computing (ICSC), pp. 153–154 (2023)
Acknowledgments
We gratefully acknowledge the support and funding provided by the Ministry of Electronics and Information Technology (MeitY), Government of India, which made this research possible. We would like to extend our heartfelt thanks to IIT Madras and IIT Guwahati for their invaluable feedback and unwavering support throughout the course of our activities. Additionally, we would like to express our deep gratitude to all the native speakers from West Bengal who generously shared their voice samples for our research endeavors. Without the collective efforts of these individuals and organizations, this work would not have been achievable. A part of this work is supported by NLTM BHASHINI project funding 11(1)/2022-HCC(TDIL) from MeitY, Govt. of India.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sarkar, A.K. et al. (2023). Study of Various End-to-End Keyword Spotting Systems on the Bengali Language Under Low-Resource Condition. In: Karpov, A., Samudravijaya, K., Deepak, K.T., Hegde, R.M., Agrawal, S.S., Prasanna, S.R.M. (eds) Speech and Computer. SPECOM 2023. Lecture Notes in Computer Science(), vol 14339. Springer, Cham. https://doi.org/10.1007/978-3-031-48312-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-48312-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-48311-0
Online ISBN: 978-3-031-48312-7
eBook Packages: Computer ScienceComputer Science (R0)