Empowering neural collaborative filtering with contextual features for multimedia recommendation | Multimedia Systems Skip to main content
Springer Nature Link
Log in

Empowering neural collaborative filtering with contextual features for multimedia recommendation

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

A rapid growth in multimedia on various application platforms has made essential the provision of additional assistive technologies to handle information overload issues. Consequently, various multimedia recommendation systems have been developed by the research community. Among these Neural Collaborative Filtering (NCF) is one of the most commonly adopted recommendation frameworks. In this research, we argue that weighing contextual features can help the underlined learning model to develop a better understanding of a user’s behavior. We propose a Weighted Context-based Neural Collaborative Filtering (WNCF) model to supplement weighted contextual information into NCF for learning the user–item interaction function with respect to the different contextual conditions. We introduced an interactive mechanism for addressing the user ratings on items in various contextual situations. Learned contextual weights describe the importance of each item in specific contextual conditions. The proposed model can also assign different weights to the contextual conditions depending on their significance. We performed extensive experiments on three real-world datasets and the outcomes demonstrate the significance of our proposal in comparison with the state-of-the-art models. Empirical results highlight that integrating weighted contextual information with NCF has enhanced recommendation performance. Also, the in-depth analysis leads us toward a completely new research direction on context-aware recommender systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

Dataset could be flourished upon request.

References

  1. Adomavicius, G., Mobasher, B., Ricci, F., Tuzhilin, A.: Context-aware recommender systems. AI Mag. 32(3), 67–80 (2011)

    Google Scholar 

  2. Alhamid, M.F., Rawashdeh, M., Dong, H., Hossain, M.A., Alelaiwi, A., El-Saddik, A.: Recam: a collaborative context-aware framework for multimedia recommendations in an ambient intelligence environment. Multimed. Syst. 22(5), 587–601 (2016)

    Google Scholar 

  3. Ali, W., Kumar, J., Mawuli, C.B., She, L., Shao, J.: Dynamic context management in context-aware recommender systems. Comput. Electr. Eng. 107, 108622 (2023)

    Google Scholar 

  4. Barragáns-Martínez, A.B., Costa-Montenegro, E., Burguillo-Rial, J.C., Rey-López, M., Mikic-Fonte, F.A., Peleteiro-Ramallo, A.: A hybrid content-based and item-based collaborative filtering approach to recommend TV programs enhanced with singular value decomposition. Inf. Sci. 180(22), 4290–4311 (2010)

    Google Scholar 

  5. Braunhofer, M., Ricci, F.: Selective contextual information acquisition in travel recommender systems. J. Inf. Technol. Tour. 17(1), 5–29 (2017)

    Google Scholar 

  6. Chang, L., Chen, W., Huang, J., Bin, C., Wang, W.: Exploiting multi-attention network with contextual influence for point-of-interest recommendation. Appl. Intell. 51(4), 1904–1917 (2021)

    Google Scholar 

  7. Chen, J., Zhang, H., He, X., Nie, L., Liu, W., Chua, T.: Attentive collaborative filtering: multimedia recommendation with item- and component-level attention. In: Kando, N., Sakai, T., Joho, H., Li, H., de Vries, A.P., White, R.W. (eds.) Proceedings of the 40th international ACM SIGIR Conference on research and development in information retrieval, Shinjuku, Tokyo, Japan, pp. 335–344. ACM (2017)

    Google Scholar 

  8. Chen, W., Chen, W., Song, L.: Enhancing deep multimedia recommendations using graph embeddings. In: 3rd IEEE conference on multimedia information processing and retrieval, MIPR 2020, Shenzhen, China, August 6-8, 2020. IEEE (2020)

    Google Scholar 

  9. Cui, Q., Wu, S., Liu, Q., Wang, L.: A visual and textual recurrent neural network for sequential prediction. CoRR abs/1611.06668 (2016)

  10. Ding, K., Wang, R., Wang, S.: Social media popularity prediction: a multiple feature fusion approach with deep neural networks. In: Amsaleg, L., Huet, B., Larson, M.A., Gravier, G., Hung, H., Ngo, C., Ooi, W.T. (eds.) Proceedings of the 27th ACM International Conference on Multimedia, MM 2019, Nice, France, October 21-25, 2019, pp. 2682–2686. ACM (2019)

    Google Scholar 

  11. Djenouri, Y., Belhadi, A., Srivastava, G., Lin, J.C.: Deep learning based hashtag recommendation system for multimedia data. Inf. Sci. 609, 1506–1517 (2022)

    Google Scholar 

  12. Han, H., Qin, X., Zhao, Q.: Interactive attention graph convolution networks for aspect-level sentiment classification. In: 3rd International conference on artificial intelligence and advanced manufacture, AIAM 2021, Manchester, United Kingdom, October 23-25, 2021, pp. 271–275. IEEE (2021)

    Google Scholar 

  13. He, X., He, Z., Song, J., Liu, Z., Jiang, Y., Chua, T.: NAIS: neural attentive item similarity model for recommendation. IEEE Trans. Knowl. Data Eng. 30(12), 2354–2366 (2018)

    Google Scholar 

  14. He, X., Liao, L., Zhang, H., Nie, L., Hu, X., Chua, T.: Neural collaborative filtering. In: Barrett, R., Cummings, R., Agichtein, E., Gabrilovich, E. (eds.) Proceedings of the 26th international conference on world wide web, WWW 2017, Perth, Australia, April 3-7, 2017, pp. 173–182. ACM (2017)

    Google Scholar 

  15. Hidasi, B., Karatzoglou, A.: Recurrent neural networks with top-k gains for session-based recommendations. In: Cuzzocrea, A., Allan, J., Paton, N.W., Srivastava, D., Agrawal, R., Broder, A.Z., Zaki, M.J., Candan, K.S., Labrinidis, A., Schuster, A., Wang, H. (eds.) Proceedings of the 27th ACM international conference on information and knowledge management, CIKM 2018, Torino, Italy, October 22-26, 2018, pp. 843–852. ACM (2018)

    Google Scholar 

  16. Hu, M., Peng, Y., Huang, Z., Qiu, X., Wei, F., Zhou, M.: Reinforced mnemonic reader for machine reading comprehension. In: Lang, J. (ed.) Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, pp. 4099–4106. ijcai.org (2018)

  17. Kabbur, S., Ning, X., Karypis, G.: FISM: factored item similarity models for top-n recommender systems. In: Dhillon, I.S., Koren, Y., Ghani, R., Senator, T.E., Bradley, P., Parekh, R., He, J., Grossman, R.L., Uthurusamy, R. (eds.) The 19th ACM SIGKDD international conference on knowledge discovery and data mining, KDD 2013, Chicago, IL, USA, August 11-14, 2013, pp. 659–667. ACM (2013)

    Google Scholar 

  18. Khan, Z., Iltaf, N., Afzal, H., Abbas, H.: Enriching non-negative matrix factorization with contextual embeddings for recommender systems. Neurocomputing 380, 246–258 (2020)

    Google Scholar 

  19. Li, B., Wang, G., Cheng, Y., Sun, Y., Bi, X.: An event recommendation model using ELM in event-based social network. Neural Comput. Appl. 32(18), 14375–14384 (2020)

    Google Scholar 

  20. Li, H., Min, M.R., Ge, Y., Kadav, A.: A context-aware attention network for interactive question answering. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, Halifax, NS, Canada, August 13-17, 2017, pp. 927–935. ACM (2017)

    Google Scholar 

  21. Li, J., Wang, Y., McAuley, J.J.: Time interval aware self-attention for sequential recommendation. In: Caverlee, J., Hu, X.B., Lalmas, M., Wang, W. (eds.) WSDM ’20: the thirteenth ACM international conference on web search and data mining, Houston, TX, USA, February 3-7, 2020, pp. 322–330. ACM (2020)

    Google Scholar 

  22. Li, X., Cong, G., Li, X., Pham, T.N., Krishnaswamy, S.: Rank-geofm: a ranking based geographical factorization method for point of interest recommendation. In: Baeza-Yates, R., Lalmas, M., Moffat, A., Ribeiro-Neto, B.A. (eds.) Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, Santiago, Chile, August 9-13, 2015, pp. 433–442. ACM (2015)

    Google Scholar 

  23. Liu, C., Wu, X.: Large-scale recommender system with compact latent factor model. Expert Syst. Appl. 64, 467–475 (2016)

    Google Scholar 

  24. Liu, J., Wu, C., Wang, J.: Gated recurrent units based neural network for time heterogeneous feedback recommendation. Inf. Sci. 423, 50–65 (2018)

    Google Scholar 

  25. Lops, P., de Gemmis, M., Semeraro, G.: Content-based recommender systems: state of the art and trends. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender systems handbook, pp. 73–105. Springer (2011)

    Google Scholar 

  26. Mahajan, P., Kaur, P.D.: Three-tier iot-edge-cloud (3T-IEC) architectural paradigm for real-time event recommendation in event-based social networks. J. Ambient. Intell. Humaniz. Comput. 12(1), 1363–1386 (2021)

    Google Scholar 

  27. Mahajan, P., Kaur, P.D.: 3t-iec*: a context-aware recommender system architecture for smart social networks (EBSN and SBSN). J. Intell. Inf. Syst. 60(1), 199–233 (2023)

    Google Scholar 

  28. Miao, H., Luo, B., Sun, Z.: An improved context-aware recommender algorithm. In: Huang, D., Bevilacqua, V., Premaratne, P. (eds.) Intelligent computing theories and application - 12th international conference, ICIC 2016, Lanzhou, China, August 2-5, 2016, proceedings, part I. Lecture notes in computer science, vol. 9771, pp. 153–162. Springer (2016)

    Google Scholar 

  29. Moon, C.B., Lee, J.Y., Kim, D., Kim, B.M.: Multimedia content recommendation in social networks using mood tags and synonyms. Multimed. Syst. 26(2), 139–156 (2020)

    Google Scholar 

  30. Nassar, N., Jafar, A., Rahhal, Y.: A novel deep multi-criteria collaborative filtering model for recommendation system. Knowl. Based Syst. 187, 104811 (2020)

    Google Scholar 

  31. Ning, X., Karypis, G.: SLIM: sparse linear methods for top-n recommender systems. In: Cook, D.J., Pei, J., Wang, W., Zaïane, O.R., Wu, X. (eds.) 11th IEEE international conference on data mining, ICDM 2011, Vancouver, BC, Canada, December 11-14, 2011, pp. 497–506. IEEE Computer Society (2011)

    Google Scholar 

  32. Odic, A., Tkalcic, M., Tasic, J.F., Kosir, A.: Predicting and detecting the relevant contextual information in a movie-recommender system. Interact. Comput. 25(1), 74–90 (2013)

    Google Scholar 

  33. Pei, W., Yang, J., Sun, Z., Zhang, J., Bozzon, A., Tax, D.M.J.: Interacting attention-gated recurrent networks for recommendation. In: Lim, E., Winslett, M., Sanderson, M., Fu, A.W., Sun, J., Culpepper, J.S., Lo, E., Ho, J.C., Donato, D., Agrawal, R., Zheng, Y., Castillo, C., Sun, A., Tseng, V.S., Li, C. (eds.) Proceedings of the 2017 ACM on conference on information and knowledge management, CIKM 2017, Singapore, November 06-10, 2017, pp. 1459–1468. ACM (2017)

    Google Scholar 

  34. ur Rehman, I., Ali, W., Jan, Z., Ali, Z., Xu, H., Shao, J.: Caml: contextual augmented meta-learning for cold-start recommendation. Neurocomputing 533, 178–190 (2023)

    Google Scholar 

  35. Sassi, I.B., Yahia, S.B.: How does context influence music preferences: a user-based study of the effects of contextual information on users’ preferred music. Multimed. Syst. 27(2), 143–160 (2021)

    Google Scholar 

  36. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Guyon, I., von Luxburg, U., Bengio, S., Wallach, H.M., Fergus, R., Vishwanathan, S.V.N., Garnett, R. (eds.) Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 5998–6008. ACM (2017)

    Google Scholar 

  37. Wang, S., Hu, L., Cao, L., Huang, X., Lian, D., Liu, W.: Attention-based transactional context embedding for next-item recommendation. In: McIlraith, S.A., Weinberger, K.Q. (eds.) Proceedings of the thirty-second AAAI conference on artificial intelligence, (AAAI-18), the 30th innovative applications of artificial intelligence (IAAI-18), and the 8th AAAI symposium on educational advances in artificial intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pp. 2532–2539. AAAI Press (2018)

  38. Wang, Y., Xu, H., Ali, W., Li, M., Zhou, X., Shao, J.: Fedftha: a fine-tuning and head aggregation method in federated learning. IEEE Internet Things J. (2023). https://doi.org/10.1109/JIOT.2023.3254052

    Article  Google Scholar 

  39. Waqar, A., Din, S.U., Khan, A.A., Tumrani, S., Wang, X., Shao, J.: Context-aware collaborative filtering framework for rating prediction based on novel similarity estimation. Comput. Mater. Contin. 63(2), 1065–1078 (2020)

    Google Scholar 

  40. Wu, C., Ahmed, A., Beutel, A., Smola, A.J., Jing, H.: Recurrent recommender networks. In: de Rijke, M., Shokouhi, M., Tomkins, A., Zhang, M. (eds.) Proceedings of the tenth ACM international conference on web search and data mining, WSDM 2017, Cambridge, United Kingdom, February 6-10, 2017, pp. 495–503. ACM (2017)

    Google Scholar 

  41. Xu, H., Gong, L., Xuan, H., Zheng, X., Gao, Z., Wen, X.: Multiview clustering via consistent and specific nonnegative matrix factorization with graph regularization. Multimed. Syst. 28(5), 1559–1572 (2022)

    Google Scholar 

  42. Yang, C., Yu, X., Liu, Y., Nie, Y., Wang, Y.: Collaborative filtering with weighted opinion aspects. Neurocomputing 210, 185–196 (2016)

    Google Scholar 

  43. Yu, S., Yang, M., Qu, Q., Shen, Y.: Contextual-boosted deep neural collaborative filtering model for interpretable recommendation. Expert Syst. Appl. 136, 365–375 (2019)

    Google Scholar 

  44. Zhang, H., Kong, X., Zhang, Y.: Cross-domain collaborative recommendation without overlapping entities based on domain adaptation. Multimed. Syst. 28(5), 1621–1637 (2022)

    Google Scholar 

  45. Zhang, S., Yao, L., Sun, A., Wang, S., Long, G., Dong, M.: Neurec: On nonlinear transformation for personalized ranking. In: Lang, J. (ed.) Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, pp. 3669–3675. ijcai.org (2018)

  46. Zheng, Y.: Context-aware collaborative filtering using context similarity: an empirical comparison. Information 13(1), 42 (2022)

    Google Scholar 

  47. Zheng, Y., Burke, R.D., Mobasher, B.: Recommendation with differential context weighting. In: Carberry, S., Weibelzahl, S., Micarelli, A., Semeraro, G. (eds.) User modeling, adaptation, and personalization - 21th international conference, UMAP 2013, Rome, Italy, June 10-14, 2013, proceedings. Lecture notes in computer science, vol. 7899, pp. 152–164. Springer (2013)

    Google Scholar 

  48. Zheng, Y., Mobasher, B., Burke, R.D.: Incorporating context correlation into context-aware matrix factorization. In: Jannach, D., Mengin, J., Mobasher, B., Passerini, A., Viappiani, P. (eds.) Proceedings of the IJCAI 2015 Joint Workshop on Constraints and Preferences for Configuration and Recommendation and Intelligent Techniques for Web Personalization co-located with the 24th International Joint Conference on Artificial Intelligence (IJCAI 2015), Buenos Aires, Argentina, July 27, 2015. CEUR workshop proceedings, volume 1440. CEUR-WS.org (2015)

  49. Zheng, Y., Mobasher, B., Burke, R.D.: Similarity-based context-aware recommendation. In: Wang, J., Cellary, W., Wang, D., Wang, H., Chen, S., Li, T., Zhang, Y. (eds.) Web information systems engineering-WISE 2015-16th international conference, Miami, FL, USA, November 1-3, 2015, proceedings, part I. Lecture notes in computer science, vol. 9418, pp. 431–447. Springer (2015)

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Sichuan Science and Technology Program (No. 2022YFWZ0006). Also, we thank the Higher Education Commission of Pakistan and Yibin GRACE Co., Ltd., China for the financial and technical assistance.

Author information

Authors and Affiliations

  1. Department of Computer Science, Islamia College Peshawar, Peshawar, 25000, Pakistan

    Israr ur Rehman & Zahoor Jan

  2. UCP Business School, University of Central Punjab, Lahore, 54000, Pakistan

    Muhammad Shehzad Hanif

  3. Department of Computer Science, National University of Technology, Islamabad, 44000, Pakistan

    Zulfiqar Ali

  4. Yangtze Delta Region Institute, University of Electronic Science and Technology of China, Huzhou, 313001, China

    Cobbinah Bernard Mawuli

  5. University of Electronic Science and Technology of China, Chengdu, 611731, China

    Waqar Ali

  6. Yibin GRACE Group Co. Ltd., Yibin, 644002, China

    Waqar Ali

  7. Sichuan Artificial Intelligence Research Institute, Yibin, 644000, China

    Waqar Ali

Authors
  1. Israr ur Rehman
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Muhammad Shehzad Hanif
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Zulfiqar Ali
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Zahoor Jan
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Cobbinah Bernard Mawuli
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Waqar Ali
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

IuR: writing—original draft, conceptualization, methodology, and software. MSH: writing—review and editing, and supervision. ZA: validation and formal analysis. ZJ: investigation, prepared figures, and software supervision. CBM: conceptualization and reviewed. WA: main idea, supervision, conceptualization, and methodology.

Corresponding author

Correspondence to Waqar Ali.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rehman, I.u., Hanif, M.S., Ali, Z. et al. Empowering neural collaborative filtering with contextual features for multimedia recommendation. Multimedia Systems 29, 2375–2388 (2023). https://doi.org/10.1007/s00530-023-01107-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-023-01107-9

Keywords