Abstract
Sequential recommendation models user preferences based on the sequence of user interactions. A key challenge in this context is the variability of user behavior. While Transformer-based models demonstrate unique superiority, existing models still struggle with modeling temporal information and adequately capturing users’ long-term and short-term preferences. This paper proposes a Time-Aware Squeeze-Excitation Transformer for sequential recommendation (TASESRec). The model has two salient features: (1) TASESRec preserves the continuity dependency within timestamps from both duration and spectrum perspectives using a time window function, and integrates timestamps and user interactions through a multi-layer encoder-decoder structure. (2) Given the non-uniqueness of users’ latent purchasing behavior, characterized by multiple potential purchasing behaviors, the model utilizes Squeeze-Excitation Attention (sigmoid activation) to comprehensively capture relevant items, thus enhancing prediction accuracy. Extensive experiments validate the superiority of the proposed model over various state-of-the-art models under several widely used evaluation metrics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Joglekar, M.R., et al.: Neural input search for large scale recommendation models. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2387–2397 (2020)
Huang, J.T., et al.: Embedding-based retrieval in facebook search. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2553–2561 (2020)
He, R., McAuley, J.: Fusing similarity models with markov chains for sparse sequential recommendation. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 191–200 (2016)
Hidasi, B., Karatzoglou, A.: Recurrent neural networks with top-k gains for session-based recommendations. In: Proceedings of the 27th Acm International Conference on Information and Knowledge Management, pp. 843–852 (2018)
Liu, Q., Zeng, Y., Mokhosi, R., Zhang, H.: STAMP: short-term attention/memory priority model for session-based recommendation. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1831–1839 (2018)
Tang, J., Wang, K.: Personalized top-n sequential recommendation via convolutional sequence embedding. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pp. 565–573 (2018)
Wang, C., Zhang, M., Ma, W., Liu, Y., Ma, S.: Make it a chorus: knowledge-and time-aware item modeling for sequential recommendation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 109–118 (2020)
Yu, Z., Lian, J., Mahmoody, A., Liu, G., Xie, X.: Adaptive user modeling with long and short-term preferences for personalized recommendation. In : IJCAI, pp. 4213–4219 (2019)
Sun, F., et al.: BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1441–1450 (2019)
Kang, W.C., McAuley, J.: Self-attentive sequential recommendation. In: 2018 IEEE International Conference on Data Mining (ICDM), pp. 197–206. IEEE (2018)
Zhang, T., et al.: Feature-level deeper self-attention network for sequential recommendation. In: IJCAI, pp. 4320–4326 (2019)
Li, J., Wang, Y., McAuley, J.: Time interval aware self-attention for sequential recommendation. In: Proceedings of the 13th International Conference on Web Search and Data Mining, pp. 322–330 (2020)
Lin, Y., Lin, F., Yang, L., Zeng, W., Liu, Y., Pengcheng, W.: Context-aware reinforcement learning for course recommendation. Appl. Soft Comput. 125, 109189 (2022)
Yang, B., Chen, J., Kang, Z., Li, D.: Memory-aware gated factorization machine for top-n recommendation. Knowl.-Based Syst. 201, 106048 (2020)
Zhang, P., Kim, S.: A survey on incremental update for neural recommender systems. arXiv preprint arXiv:2303.02851 (2023)
Shu, W., Tang, Y., Zhu, Y., Wang, L., Xie, X., Tan, T.: Session-based recommendation with graph neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 346–353 (2019)
Vaswani, A., et al.: Attention is all you need. Adv. Neural Inf. Proc. Syst. 30 (2017)
Chen, L., et al.: SCA-CNN: spatial and channel-wise attention in convolutional networks for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5659–5667 (2017)
Chen, K., Wang, R., Utiyama, M., Sumita, E., Zhao, T.: Syntax-directed attention for neural machine translation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Ji, M., Joo, W., Song, K., Kim, Y.-Y., Moon, I.-C.: Sequential recommendation with relation-aware kernelized self-attention. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 4304–4311 (2020)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)
Zhu, H., et al.: Learning tree-based deep model for recommender systems. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1079–1088 (2018)
He, R., Kang, W.C., McAuley, J.: Translation-based recommendation. In: Proceedings of the Eleventh ACM Conference on Recommender Systems, pp. 161–169 (2017)
Hidasi, B., Karatzoglou, A., Baltrunas, L., Tikk, D.: Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, H., Liu, L., Chen, Z., Li, X. (2024). Time-Aware Squeeze-Excitation Transformer for Sequential Recommendation. In: Wand, M., Malinovská, K., Schmidhuber, J., Tetko, I.V. (eds) Artificial Neural Networks and Machine Learning – ICANN 2024. ICANN 2024. Lecture Notes in Computer Science, vol 15024. Springer, Cham. https://doi.org/10.1007/978-3-031-72356-8_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-72356-8_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72355-1
Online ISBN: 978-3-031-72356-8
eBook Packages: Computer ScienceComputer Science (R0)