Abstract
The existence of jammer and the limited buffer space bring major challenge to data transmission efficiency in high-frequency (HF) commuication. The data transmission problem of how to select transmission strategy with multi-channel and different buffer states to maximize the system throughput is studied in this paper. We model the data transmission problem as a Makov decision process (MDP). Then, a modified Q-learning with additional value is proposed to help transmitter to learn the appropriate strategy and improve the system throughput. The simulation results show the proposed Q-learning algorithm can converge to the optimal Q value. Simultaneously, the QL algorithm compared with the sensing algorithm has better system throughput and less packet loss.
This work was supported by the National Natural Science Foundation of China under Grant No. 61771488, No. 61671473 and No. 61631020, in part by the Natural Science Foundation for Distinguished Young Scholars of Jiangsu Province under Grant No. BK20160034.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wang, J.: Research and Development of HF Digital Communications. Science Press, Beijing, China (2013)
Hanson, R.: Military applications of HF communications. Mil. Technol. 10, 70–77 (2010)
Xu, K., Jiang, B., Su, Z., et al.: High frequency communication network with diversity: system structure and key enabling techniques. China Commun. 15(9), 46–59 (2018)
Haykin, S.: Cognitive radio: brain-empowered wireless communications. IEEE J. Sel. Areas Commun. 23(2), 201–220 (2015)
Yucek, T., Arslan, H.: A survey of spectrum sensing algorithms for cognitive radio applications. IEEE Commun. Surv. Tutor. 11(1), 116–130 (2009)
Bkassiny, M., Li, Y., Jayaweera, S.K.: A survey on machine learning techniques in cognitive radios. IEEE Commun. Surv. Tutor. 15(3), 1136–1159 (2013)
Luong, N.C., Hoang, D.T., Gong, S., et al.: Applications of deep reinforcement learning in communications and networking: a survey. arXiv:1810.07862
Yang, D., Xue, G., Zhang, J., Richa, A., Fang, X.: Coping with a smart jammer in wireless networks: a Stackelberg game approach. IEEE Trans. Wirel. Commun. 12(8), 4038–4047 (2013)
Jia, L., Xu, Y., Sun, Y., Feng, S., Anpalagan, A.: Stackelberg game approaches for anti-jamming defence in wireless networks. IEEE Trans. Wirel. Commun. 25(6), 120–128 (2018)
Feng, Z., Ren, G., Chen, J.: Power control in relay-assisted anti-jamming systems: a Bayesian three-layer stackelberg game approach. IEEE Access 7, 14623–14636 (2019)
Michael, R.M., et al.: Adaptive coding for frequency-Hop transmission over fading channels with partial-Band interference. IEEE Trans. Commun. 59(3), 854–862 (2011)
Duan, R.J., et al.: Research on spectrum allocation of HF access network based on intelligent frequency hopping. In: 8th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, pp. 295–300 (2015)
Johnson, E., et al.: Third-Generation and Wideband HF Radio Communications. Artech House, Norwood (2013)
Wang, J., Ding, G., Wang, H.: HF communications: past, present, and future. China Commun. 15(9), 1–9 (2018)
Kong, L., Xu, Y., Zhang, Y., et al.: A reinforcement learning approach for dynamic spectrum anti-jamming in fading environment. In: IEEE 18th International Conference on Communication Technology (ICCT), Chongqing, pp. 51–58 (2018)
Slimeni, F., Chtourou, Z., Schaeers, B., Nir, V.L., Attia, R.: Cooperative Q-learning based channel selection for cognitive radio. Wirel. Netw. 4, 1–11 (2018)
Xiao, L., Lu, X., Xu, D., Tang, Y., Wang, L., Zhuang, W.: Uav relay in vanets against smart jamming with reinforcement learning. IEEE Trans. Veh. Technol. 67(5), 4087–4097 (2018)
Liu, X., Xu, Y., Jia, L., et al.: Anti-jamming communications using spectrum waterfall: a deep reinforcement learning approach. IEEE Commun. Lett. 22(5), 998–1001 (2018)
Liu, X., Xu, Y., Cheng, Y., et al.: A heterogeneous information fusion deep reinforcement learning for intelligent frequency selection of HF communication. China Commun. 15(9), 73–84 (2018)
Chen, Y., Li, Y., Xu, D., Xiao, L.: DQN-based power control for IOT transmission against jamming. In: IEEE 87th Vehicular Technology Conference (VTC Spring), Porto, pp. 1–5 (2018)
Zhu, J., Song, Y., Jiang, D., Song, H.: A new deep-Q-learning-based transmission scheduling mechanism for the cognitive internet of things. IEEE Internet Things J. 5(4), 2375–2385 (2018)
Lin, X., Tan, Y., Zhang, J.: A MDP-based energy efficient policy for wireless transmission. Commun. Netw. 36(7), 1433–1438 (2014)
Li, W., Ruan, L., Xu, Y., et al.: Exploring channel diversity in HF communication systems: a matching-potential game approach. China Commun. 15(9), 60–72 (2018)
Wang, H.S., Moayeri, N.: Finite-state Markov channel-a useful model for radio communication channels. IEEE Trans. Veh. Technol. 44(1), 163–171 (1995)
Zhang, Q., Kassam, S.A.: Finite-state Markov model for Rayleigh fading channels. IEEE Trans. Commun. 47(11), 1688–1692 (1999)
Monahan, G.E.: State of the art-a survey of partially observable Markov decision processes: theory, models, and algorithms. Manag. Sci. 28(1), 1–16 (1982)
Luong, N.C., Hoang, D.T., Gong, S., et al.: Applications of deep reinforcement learning in communications and networking: a survey (2018). arXiv:1810.07862
Browne, C.B., Powley, E., Whitehouse, D., et al.: A survey of Monte Carlo tree search methods, 4(1), 1–43 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Li, W. et al. (2019). A Q-Learning-Based Channel Selection and Data Scheduling Approach for High-Frequency Communications in Jamming Environment. In: Zhai, X., Chen, B., Zhu, K. (eds) Machine Learning and Intelligent Communications. MLICOM 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 294. Springer, Cham. https://doi.org/10.1007/978-3-030-32388-2_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-32388-2_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32387-5
Online ISBN: 978-3-030-32388-2
eBook Packages: Computer ScienceComputer Science (R0)