When Alignment Makes a Difference: A Content-Based Variational Model for Cold-Start CTR Prediction | SpringerLink
Skip to main content
Springer Nature Link
Log in

When Alignment Makes a Difference: A Content-Based Variational Model for Cold-Start CTR Prediction

  • Conference paper
  • First Online:
Advanced Data Mining and Applications (ADMA 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14176))

Included in the following conference series:

  • 1139 Accesses

Abstract

Click-Through Rate (CTR) prediction is a core task in recommendation systems. Despite VAE-based models have shown promising accuracy performance, they are still weak in supporting cold-start CTR prediction due to limited personal interactions. To this end, this paper proposes a content-based variational CTR model, which jointly models content information and interactions behaviors in a shared probability space via variational inference. Specifically, a three-step scheme is designed to fully utilize content information for the improved ability of preference modeling toward cold-start users. First, our method adopts VAE to model user preferences from personal interactions by probabilistic distributions, instead of a fixed embedding vector for representing the user’s interest. Then, we transform content information into variational probabilistic distribution to model the implicit preferences of cold-start users. Finally, a variational alignment strategy is applied to maximize the similarity between variational preference distributions obtained from interactions behaviors and content information respectively, so that the interest of the cold user can be recovered. Besides, we adopt a self-attention mechanism to reasonably balance the importance of latent features for CTR prediction. Experiments on two public real datasets show the effectiveness of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11210
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14013
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://grouplens.org/datasets/movielens/.

  2. 2.

    https://tianchi.aliyun.com/dataset/dataDetail?dataId=56.

References

  1. Chen, Y., de Rijke, M.: A collective variational autoencoder for top-n recommendation with side information. In: DLRS@RecSys, pp. 3–9 (2018)

    Google Scholar 

  2. Cheng, H.T., et al.: Wide and deep learning for recommender systems. In: DLRS@RecSys, pp. 7–10 (2016)

    Google Scholar 

  3. Dong, M., Yuan, F., Yao, L., Xu, X., Zhu, L.: MAMO: memory-augmented meta-optimization for cold-start recommendation. In: KDD, pp. 688–697 (2020)

    Google Scholar 

  4. Gaurav, P., Ambedkar, D.: Variational methods for conditional multimodal deep learning. In: IJCNN, pp. 308–315 (2017)

    Google Scholar 

  5. Gu, L.: Ad click-through rate prediction: a survey. In: DASFAA, pp. 140–153 (2021)

    Google Scholar 

  6. Guo, H., Tang, R., Ye, Y., Li, Z., He, X.: Deepfm: a factorization-machine based neural network for CTR prediction. In: IJCAI, pp. 1725–1731 (2017)

    Google Scholar 

  7. Hu, X., Xu, J., Wang, W., Li, Z., Liu, A.: A graph embedding based model for fine-grained POI recommendation. Neurocomputing 428, 376–384 (2021)

    Article  Google Scholar 

  8. Kingma, D.P., Welling, M.: Auto-Encoding Variational Bayes (2014)

    Google Scholar 

  9. Lee, H., Im, J., Jang, S., Cho, H., Chung, S.: Melu: meta-learned user preference estimator for cold-start recommendation. In: KDD, pp. 1073–1082 (2019)

    Google Scholar 

  10. Li, F., Chen, Z., Wang, P., Ren, Y., Zhang, D., Zhu, X.: Graph intention network for click-through rate prediction in sponsored search. In: SIGIR, pp. 961–964 (2019)

    Google Scholar 

  11. Li, Y., Xu, J., Zhao, P., Fang, J., Chen, W., Zhao, L.: Atlrec: an attentional adversarial transfer learning network for cross-domain recommendation. J. Comput. Sci. Technol. 35(4), 794–808 (2020)

    Article  Google Scholar 

  12. Liang, D., Krishnan, R.G., Hoffman, M.D., Jebara, T.: Variational autoencoders for collaborative filtering. In: WWW, pp. 689–698 (2018)

    Google Scholar 

  13. Pan, F., Li, S., Ao, X., Tang, P., He, Q.: Warm up cold-start advertisements: improving CTR predictions via learning to learn id embeddings. In: SIGIR, pp. 695–704 (2019)

    Google Scholar 

  14. Qu, Y., et al.: Product-based neural networks for user response prediction over multi-field categorical data. ACM Trans. Inf. Syst. 37(1), 5:1–5:35 (2019)

    Google Scholar 

  15. Rendle, S.: Factorization machines. In: ICDM, pp. 995–1000 (2010)

    Google Scholar 

  16. Shen, F., Yan, S., Zeng, G.: Neural style transfer via meta networks. In: CVPR, pp. 8061–8069 (2018)

    Google Scholar 

  17. Sohn, K., Lee, H., Yan, X.: Learning structured output representation using deep conditional generative models. In: NIPS, pp. 3483–3491 (2015)

    Google Scholar 

  18. Song, J., Xu, J., Zhou, R., Chen, L., Li, J., Liu, C.: CBML: a cluster-based meta-learning model for session-based recommendation. In: CIKM, pp. 1713–1722 (2021)

    Google Scholar 

  19. Sun, H., Xu, J., Zheng, K., Zhao, P., Chao, P., Zhou, X.: MFNP: a meta-optimized model for few-shot next POI recommendation. In: IJCAI, pp. 3017–3023 (2021)

    Google Scholar 

  20. Sun, H., Xu, J., Zhou, R., Chen, W., Zhao, L., Liu, C.: HOPE: a hybrid deep neural model for out-of-town next POI recommendation. WWW 24(5), 1749–1768 (2021)

    Google Scholar 

  21. Vahdat, A., Kautz, J.: NVAE: a deep hierarchical variational autoencoder. In: NIPS, pp. 19667–19679 (2020)

    Google Scholar 

  22. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems 30 (2017)

    Google Scholar 

  23. Volkovs, M., Yu, G., Poutanen, T.: DropoutNet: addressing cold start in recommender systems. In: NIPS, pp. 4957–4966 (2017)

    Google Scholar 

  24. Wang, P., Jiang, Y., Xu, C., Xie, X.: Overview of content-based click-through rate prediction challenge for video recommendation. In: ACM Multimedia, pp. 2593–2596 (2019)

    Google Scholar 

  25. Wang, R., Fu, B., Fu, G., Wang, M.: Deep and cross network for ad click predictions. In: ADKDD@KDD, pp. 1–7 (2017)

    Google Scholar 

  26. Wang, X., Du, Y., Zhang, L., Li, X., Zhang, M., Dong, J.: Exploring content-based video relevance for video click-through rate prediction. In: ACM Multimedia, pp. 2602–2606 (2019)

    Google Scholar 

  27. Xie, R., Wang, R., Zhang, S., Yang, Z., Xia, F., Lin, L.: Real-time relevant recommendation suggestion. In: WSDM, pp. 112–120 (2021)

    Google Scholar 

  28. Xu, X., et al.: Alleviating cold-start problem in CTR prediction with a variational embedding learning framework. In: WWW, pp. 27–35 (2022)

    Google Scholar 

  29. Yu, B., Li, X., Fang, J., Tai, C., Cheng, W., Xu, J.: Memory-augmented meta-learning framework for session-based target behavior recommendation. WWW 26(1), 233–251 (2023)

    Google Scholar 

  30. Zhao, J., Zhao, P., Zhao, L., Liu, Y., Sheng, V.S., Zhou, X.: Variational self-attention network for sequential recommendation. In: ICDE, pp. 1559–1570 (2021)

    Google Scholar 

  31. Zhou, R., Liu, C., Wan, J., Fan, Q., Ren, Y., Zhang, J., Xiong, N.: A hybrid neural network architecture to predict online advertising click-through rate behaviors in social networks. IEEE Trans. Netw. Sci. Eng. 8(4), 3061–3072 (2021)

    Article  Google Scholar 

  32. Zhu, Q., Bi, W., Liu, X., Ma, X., Li, X., Wu, D.: A batch normalized inference network keeps the KL vanishing away. In: ACL, pp. 2636–2649 (2020)

    Google Scholar 

  33. Zhu, Y., et al.: Learning to warm up cold item embeddings for cold-start recommendation with meta scaling and shifting networks. In: SIGIR, pp. 1167–1176 (2021)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Soochow University, Suzhou, China

    Jianyu Ren & Ruoqian Zhang

Authors
  1. Jianyu Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruoqian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ruoqian Zhang .

Editor information

Editors and Affiliations

  1. Northeastern University, Shenyang, China

    Xiaochun Yang

  2. The University of Indonesia, Depok, Indonesia

    Heru Suhartanto

  3. Beijing Institute of Technology, Beijing, China

    Guoren Wang

  4. Northeastern University, Shenyang, China

    Bin Wang

  5. University of Technology Sydney, Sydney, NSW, Australia

    Jing Jiang

  6. Agency for Science, Technology and Research (A*STAR), Singapore, Singapore

    Bing Li

  7. Sun Yat-sen University, Guangzhou, China

    Huaijie Zhu

  8. Anhui University, Hefei, China

    Ningning Cui

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ren, J., Zhang, R. (2023). When Alignment Makes a Difference: A Content-Based Variational Model for Cold-Start CTR Prediction. In: Yang, X., et al. Advanced Data Mining and Applications. ADMA 2023. Lecture Notes in Computer Science(), vol 14176. Springer, Cham. https://doi.org/10.1007/978-3-031-46661-8_48

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-46661-8_48

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-46660-1

  • Online ISBN: 978-3-031-46661-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation