Abstract
Click-Through Rate (CTR) prediction is a critical problem in recommendation systems since it involves enormous business interest. Most deep CTR model follows an Embedding & Feature Interaction paradigm. However, the feature interaction module cannot work well without a good embedding representation of features. Due to the long-tail phenomenon in real scenes, few samples are provided in the dataset for a large proportion of features. In this paper, we present ProtoMix, a model-agnostic framework for learnable data augmentation on few-shot features in CTR prediction. ProtoMix automatically extracts information from co-occurred features within the same instance to assign prototype embedding with vector quantization for few-shot features and further synthesize the embedding representation of the augmented virtual instance for training. Original embedding, feature interaction module, and the embedding generator are jointly trained on a well-designed objective in an end-to-end manner in ProtoMix. We experimentally validate the effectiveness and compatibility of ProtoMix by comparing it with baseline and other data augmentation methods on different deep CTR models and multiple real-world CTR benchmark datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
\(\textrm{sg}\) stand for the stop gradient operator that is defined as identity at forward computation time and has zero partial derivatives.
- 2.
- 3.
- 4.
References
Bian, S., Zhao, W.X., Wang, J., Wen, J.R.: A relevant and diverse retrieval-enhanced data augmentation framework for sequential recommendation. In: Proceedings of the 31st ACM International Conference on Information and Knowledge Management, pp. 2923–2932 (2022)
Feng, S.Y., et al.: A survey of data augmentation approaches for MLP. arXiv preprint arXiv:2105.03075 (2021)
Guo, W., et al.: Dual graph enhanced embedding neural network for CR prediction. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 496–504 (2021)
Pan, F., Li, S., Ao, X., Tang, P., He, Q.: Warm up cold-start advertisements: Improving CTR predictions via learning to learn id embeddings. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 695–704 (2019)
Van Den Oord, A., Vinyals, O., et al.: Neural discrete representation learning. In: Advances in Neural Information Processing Systems 30 (2017)
Wang, R., et al.: DCN v2: Improved deep AND cross network and practical lessons for web-scale learning to rank systems. In: Proceedings of the Web Conference 2021, pp. 1785–1797 (2021)
Xie, X., Sun, F., Liu, Z., Gao, J., Ding, B., Cui, B.: Contrastive pre-training for sequential recommendation. arXiv preprint arXiv:2010.14395 (2020)
Xu, X., et al.: Alleviating cold-start problem in CTR prediction with a variational embedding learning framework. arXiv preprint arXiv:2201.10980 (2022)
Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017)
Zhao, T., Liu, Y., Neves, L., Woodford, O., Jiang, M., Shah, N.: Data augmentation for graph neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 35, pp. 11015–11023 (2021)
Zhu, J., Liu, J., Yang, S., Zhang, Q., He, X.: Open benchmarking for click-through rate prediction. In: Proceedings of the 30th ACM International Conference on Information and Knowledge Management, pp. 2759–2769 (2021)
Zhu, Y., et al.: Learning to warm up cold item embeddings for cold-start recommendation with meta scaling and shifting networks. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1167–1176 (2021)
Acknowledgements
This work is partially supported by the Special Project on High-performance Computing under the National Key R &D Program (No.2016YFB0200602), and the Natural Science Foundation of Guangdong Province, China (No.2022A1515010831).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, H., Xu, R., Wang, CD., Jiang, Y. (2023). ProtoMix: Learnable Data Augmentation on Few-Shot Features with Vector Quantization in CTR Prediction. In: Yang, X., et al. Advanced Data Mining and Applications. ADMA 2023. Lecture Notes in Computer Science(), vol 14176. Springer, Cham. https://doi.org/10.1007/978-3-031-46661-8_47
Download citation
DOI: https://doi.org/10.1007/978-3-031-46661-8_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-46660-1
Online ISBN: 978-3-031-46661-8
eBook Packages: Computer ScienceComputer Science (R0)