{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,30]],"date-time":"2024-08-30T21:50:21Z","timestamp":1725054621787},"reference-count":48,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2023,10,31]],"date-time":"2023-10-31T00:00:00Z","timestamp":1698710400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Major Program Project of Xiangjiang Laboratory","award":["22XJ01010"]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["42301381","42271481"],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"High-Performance Computing Platform of Central South University"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"Traffic flow forecasting is an important function of intelligent transportation systems. With the rise of deep learning, building traffic flow prediction models based on deep neural networks has become a current research hotspot. Most of the current traffic flow prediction methods are designed from the perspective of model architectures, using only the traffic features of future moments as supervision signals to guide the models to learn the spatiotemporal dependence in traffic flow. However, traffic flow data themselves contain rich spatiotemporal features, and it is feasible to obtain additional self-supervised signals from the data to assist the model to further explore the underlying spatiotemporal dependence. Therefore, we propose a self-supervised traffic flow prediction method based on a spatiotemporal masking strategy. A framework consisting of symmetric backbone models with asymmetric task heads were applied to learn both prediction and spatiotemporal context features. Specifically, a spatiotemporal context mask reconstruction task was designed to force the model to reconstruct the masked features via spatiotemporal context information, so as to assist the model to better understand the spatiotemporal contextual associations in the data. In order to avoid the model simply making inferences based on the local smoothness in the data without truly learning the spatiotemporal dependence, we performed a temporal shift operation on the features to be reconstructed. The experimental results showed that the model based on the spatiotemporal context masking strategy achieved an average prediction performance improvement of 1.56% and a maximum of 7.72% for longer prediction horizons of more than 30 min compared with the backbone models.<\/jats:p>","DOI":"10.3390\/sym15112002","type":"journal-article","created":{"date-parts":[[2023,10,31]],"date-time":"2023-10-31T16:44:03Z","timestamp":1698770643000},"page":"2002","source":"Crossref","is-referenced-by-count":3,"title":["Self-Supervised Spatiotemporal Masking Strategy-Based Models for Traffic Flow Forecasting"],"prefix":"10.3390","volume":"15","author":[{"given":"Gang","family":"Liu","sequence":"first","affiliation":[{"name":"China Academy of Electronic Information Technology, Beijing 100041, China"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-0144-2524","authenticated-orcid":false,"given":"Silu","family":"He","sequence":"additional","affiliation":[{"name":"School of Geosciences and Info-Physics, Central South University, Changsha 410083, China"}]},{"given":"Xing","family":"Han","sequence":"additional","affiliation":[{"name":"School of Geosciences and Info-Physics, Central South University, Changsha 410083, China"}]},{"given":"Qinyao","family":"Luo","sequence":"additional","affiliation":[{"name":"School of Geosciences and Info-Physics, Central South University, Changsha 410083, China"}]},{"given":"Ronghua","family":"Du","sequence":"additional","affiliation":[{"name":"College of Automotive and Mechanical Engineering, Changsha University of Science and Technology, Changsha 410114, China"}]},{"given":"Xinsha","family":"Fu","sequence":"additional","affiliation":[{"name":"School of Civil Engineering and Transportation, South China University of Technology, Guangzhou 510640, China"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-6103-1113","authenticated-orcid":false,"given":"Ling","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Geosciences and Info-Physics, Central South University, Changsha 410083, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,10,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1007\/s41019-020-00151-z","article-title":"A survey of traffic prediction: From spatio-temporal data to intelligent transportation","volume":"6","author":"Yuan","year":"2021","journal-title":"Data Sci. Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.pmcj.2018.07.004","article-title":"Survey on traffic prediction in smart cities","volume":"50","author":"Nagy","year":"2018","journal-title":"Pervasive Mob. Comput."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Hashemi, S.M., Botez, R.M., and Grigorie, T.L. (2020). New Reliability Studies of Data-Driven Aircraft Trajectory Prediction. Aerospace, 7.","DOI":"10.3390\/aerospace7100145"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Tedjopurnomo, D.A., Bao, Z., Zheng, B., Choudhury, F.M., and Qin, A.K. (2023, January 3\u20137). A Survey on Modern Deep Neural Network for Traffic Prediction: Trends, Methods and Challenges (Extended Abstract). Proceedings of the 2023 IEEE 39th International Conference on Data Engineering (ICDE), Anaheim, CA, USA.","DOI":"10.1109\/ICDE55515.2023.00324"},{"key":"ref_5","unstructured":"Li, Y., Yu, R., Shahabi, C., and Liu, Y. (2017). Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Yu, B., Yin, H., and Zhu, Z. (2017). Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv.","DOI":"10.24963\/ijcai.2018\/505"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3848","DOI":"10.1109\/TITS.2019.2935152","article-title":"T-gcn: A temporal graph convolutional network for traffic prediction","volume":"21","author":"Zhao","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"736","DOI":"10.1111\/tgis.12644","article-title":"Traffic transformer: Capturing the continuity and periodicity of time series for traffic forecasting","volume":"24","author":"Cai","year":"2020","journal-title":"Trans. GIS"},{"key":"ref_9","first-page":"17804","article-title":"Adaptive graph convolutional recurrent network for traffic forecasting","volume":"33","author":"Bai","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Wu, Z., Pan, S., Long, G., Jiang, J., and Zhang, C. (2019). Graph wavenet for deep spatial-temporal graph modeling. arXiv.","DOI":"10.24963\/ijcai.2019\/264"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zhang, J., Zheng, Y., and Qi, D. (2017, January 4\u20139). Deep spatio-temporal residual networks for citywide crowd flows prediction. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.10735"},{"key":"ref_12","first-page":"3529","article-title":"Multi-Range Attentive Bicomponent Graph Convolutional Network for Traffic Forecasting","volume":"34","author":"Chen","year":"2020","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_13","first-page":"3656","article-title":"Spatiotemporal Multi-Graph Convolution Network for Ride-Hailing Demand Forecasting","volume":"33","author":"Geng","year":"2019","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_14","unstructured":"Qin, Y., Fang, Y., Luo, H., Zhao, F., and Wang, C. (2021). DMGCRN: Dynamic Multi-Graph Convolution Recurrent Network for Traffic Forecasting. arXiv."},{"key":"ref_15","first-page":"4189","article-title":"Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting","volume":"35","author":"Li","year":"2021","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_16","first-page":"914","article-title":"Spatial-Temporal Synchronous Graph Convolutional Networks: A New Framework for Spatial-Temporal Network Data Forecasting","volume":"34","author":"Song","year":"2020","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"128913","DOI":"10.1016\/j.physa.2023.128913","article-title":"STGC-GNNs: A GNN-based traffic prediction framework with a spatial\u2013temporal Granger causality graph","volume":"623","author":"He","year":"2023","journal-title":"Phys. Stat. Mech. Its Appl."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"108199","DOI":"10.1016\/j.knosys.2022.108199","article-title":"Adaptive Spatio-temporal Graph Neural Network for traffic forecasting","volume":"242","author":"Ta","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ji, J., Wang, J., Huang, C., Wu, J., Xu, B., Wu, Z., Zhang, J., and Zheng, Y. (2022). Spatio-Temporal Self-Supervised Learning for Traffic Flow Prediction. arXiv.","DOI":"10.1609\/aaai.v37i4.25555"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Liu, X., Liang, Y., Huang, C., Zheng, Y., Hooi, B., and Zimmermann, R. (2022, January 1\u20134). When do contrastive learning signals help spatio-temporal graph forecasting?. Proceedings of the 30th International Conference on Advances in Geographic Information Systems, Seattle, WA, USA.","DOI":"10.1145\/3557915.3560939"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Shao, Z., Zhang, Z., Wang, F., and Xu, Y. (2022, January 14\u201318). Pre-Training Enhanced Spatial-Temporal Graph Neural Network for Multivariate Time Series Forecasting. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD \u201922, Washington, DC, USA.","DOI":"10.1145\/3534678.3539396"},{"key":"ref_22","unstructured":"Hwang, D., Park, J., Kwon, S., Kim, K.M., Ha, J.W., and Kim, H.J. (2020, January 6\u201312). Self-Supervised Auxiliary Learning with Meta-Paths for Heterogeneous Graphs. Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS\u201920, Vancouver, BC, Canada."},{"key":"ref_23","first-page":"865","article-title":"Traffic flow prediction with big data: A deep learning approach","volume":"16","author":"Lv","year":"2014","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1049\/iet-its.2016.0208","article-title":"LSTM network: A deep learning approach for short-term traffic forecast","volume":"11","author":"Zhao","year":"2017","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Fu, R., Zhang, Z., and Li, L. (2016, January 11\u201313). Using LSTM and GRU neural network methods for traffic flow prediction. Proceedings of the 2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC), Wuhan, China.","DOI":"10.1109\/YAC.2016.7804912"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Liu, Y., Zheng, H., Feng, X., and Chen, Z. (2017, January 11\u201313). Short-term traffic flow prediction with Conv-LSTM. Proceedings of the 2017 9th International Conference on Wireless Communications and Signal Processing (WCSP), Nanjing, China.","DOI":"10.1109\/WCSP.2017.8171119"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ma, X., Dai, Z., He, Z., Ma, J., Wang, Y., and Wang, Y. (2017). Learning traffic as images: A deep convolutional neural network for large-scale transportation network speed prediction. Sensors, 17.","DOI":"10.3390\/s17040818"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zonoozi, A., Kim, J.j., Li, X.L., and Cong, G. (2018, January 13\u201319). Periodic-CRN: A convolutional recurrent model for crowd density prediction with recurring periodic patterns. Proceedings of the IJCAI, Stockholm, Sweden.","DOI":"10.24963\/ijcai.2018\/519"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3101","DOI":"10.1109\/TITS.2020.2979634","article-title":"Predicting citywide road traffic flow using deep spatiotemporal neural networks","volume":"22","author":"Jia","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_30","first-page":"802","article-title":"Convolutional LSTM network: A machine learning approach for precipitation nowcasting","volume":"28","author":"Shi","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/TNNLS.2020.2978386","article-title":"A comprehensive survey on graph neural networks","volume":"32","author":"Wu","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_32","unstructured":"Bruna, J., Zaremba, W., Szlam, A., and LeCun, Y. (2013). Spectral networks and locally connected networks on graphs. arXiv."},{"key":"ref_33","unstructured":"Atwood, J., and Towsley, D. (2016). Diffusion-convolutional neural networks. Adv. Neural Inf. Process. Syst., 29."},{"key":"ref_34","unstructured":"Veli\u010dkovi\u0107, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. (2017). Graph attention networks. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1049\/itr2.12044","article-title":"Spatial-temporal attention wavenet: A deep learning framework for traffic prediction considering spatial-temporal dependencies","volume":"15","author":"Tian","year":"2021","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zhou, Q., Chen, N., and Lin, S. (2022). FASTNN: A Deep Learning Approach for Traffic Flow Prediction Considering Spatiotemporal Features. Sensors, 22.","DOI":"10.3390\/s22186921"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Jin, G., Liang, Y., Fang, Y., Huang, J., Zhang, J., and Zheng, Y. (2023). Spatio-temporal graph neural networks for predictive learning in urban computing: A survey. arXiv.","DOI":"10.1109\/TKDE.2023.3333824"},{"key":"ref_38","unstructured":"Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv."},{"key":"ref_39","unstructured":"Xu, M., Dai, W., Liu, C., Gao, X., Lin, W., Qi, G.J., and Xiong, H. (2020). Spatial-temporal transformer networks for traffic flow forecasting. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Hashemi, S.M., Hashemi, S.A., Botez, R.M., and Ghazi, G. (2023). Aircraft Trajectory Prediction Enhanced through Resilient Generative Adversarial Networks Secured by Blockchain: Application to UAS-S4 Eh\u00e9catl. Appl. Sci., 13.","DOI":"10.3390\/app13179503"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Hashemi, S.M., Hashemi, S.A., Botez, R.M., and Ghazi, G. (2023). A Novel Fault-Tolerant Air Traffic Management Methodology Using Autoencoder and P2P Blockchain Consensus Protocol. Aerospace, 10.","DOI":"10.3390\/aerospace10040357"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"108990","DOI":"10.1016\/j.knosys.2022.108990","article-title":"TFGAN: Traffic forecasting using generative adversarial network with multi-graph convolutional network","volume":"249","author":"Khaled","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Xu, B., Wang, X., Liu, Z., and Kang, L. (2023, January 26\u201328). A GAN Combined with Graph Contrastive Learning for Traffic Forecasting. Proceedings of the 2023 4th International Conference on Computing, Networks and Internet of Things, CNIOT \u201923, Xiamen China.","DOI":"10.1145\/3603781.3603933"},{"key":"ref_44","first-page":"857","article-title":"Self-supervised learning: Generative or contrastive","volume":"35","author":"Liu","year":"2021","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"046020","DOI":"10.1088\/1741-2552\/abca18","article-title":"Uncovering the structure of clinical EEG signals with self-supervised learning","volume":"18","author":"Banville","year":"2021","journal-title":"J. Neural Eng."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Chung, Y.A., Hsu, W.N., Tang, H., and Glass, J. (2019). An unsupervised autoregressive model for speech representation learning. arXiv.","DOI":"10.21437\/Interspeech.2019-1473"},{"key":"ref_47","unstructured":"Bai, J., Wang, W., Zhou, Y., and Xiong, C. (2020). Representation learning for sequence data with deep autoencoding predictive components. arXiv."},{"key":"ref_48","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/15\/11\/2002\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,25]],"date-time":"2023-12-25T00:50:47Z","timestamp":1703465447000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/15\/11\/2002"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,31]]},"references-count":48,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2023,11]]}},"alternative-id":["sym15112002"],"URL":"https:\/\/doi.org\/10.3390\/sym15112002","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,31]]}}}