Abstract
Over the last decade, the volume of videos available on the web has increased exponentially. In order to help users cope with the ever-growing video volume, recommendation systems have emerged that can provide personalized suggestions to users based on their past preferences and relevant online metrics. However, such approaches require user profiling, which raises privacy issues while often providing delayed suggestions as various metrics have to be firstly collected such as ratings and number of views. In this paper, we propose a system specifically targeting video content generated in a conference event, where a series of talks and presentations are held and a separate video for each is recorded. Through audience analysis, our system is able to predict the online views of each video and thus recommend the most popular videos to users. This way, online users don’t have to search through all the videos of a conference event thus saving time while not missing the most impactful videos. The proposed system employs several complementary techniques for audience analysis based on video and audio streams. Experimental evaluation of real data demonstrates the potential of the proposed approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Symeonidis, P., et al.: Recommending the video to watch next: an offline and online evaluation at YOUTV.de. In: Forteenth ACM Conference on Recommender Systems, pp. 299–308 (2020)
Medrek, J., Otto, C., Ewerth, R.: Recommending scientific videos based on metadata enrichment using linked open data. In: Méndez, E., Crestani, F., Ribeiro, C., David, G., Lopes, J.C. (eds.) TPDL 2018. LNCS, vol. 11057, pp. 286–292. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00066-0_25
Zhu, Q., Shyu, M., Wang, H.: VideoTopic: content-based video recommendation using a topic model. In: IEEE International Symposium on Multimedia, pp. 219–222, (2013)
Yunyu, S., Haisheng, Y., Ming, G., Xiang, L., Yongxiang, X.: A fast and robust key frame extraction method for video copyright protection. J. Electr. Comput. Eng. 2017, 1–7 (2017). Article ID 1231794
Tang, B., Chen, W.: A description scheme for video overview based on scene detection and face clustering. J. Circuits Syst. Comput. 30(1), 2150002 (2021)
Yan, T., Ra, I., Wen, H., Weng, M., Zhang, Q., Che, Y.: CTU layer rate control algorithm in scene change video for free-viewpoint video. IEEE Access 8, 24549–24560 (2020)
Guo, G., Wang, H., Yan, Y., Zheng, J., Li, B.: A fast face detection method via convolutional neural network. Neurocomputing 395, 128–137 (2019)
Jimenez, M., Kalogeiton, V., Suarez, P., Zisserman, A.: LAEO-Net: revisiting people looking at each other in videos. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp. 3472–3480 (2019)
Soler, R., Cuevas, F., Salinas, R., Jimenez, J.: RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild. Neural Comp. and App. (2020). https://doi.org/10.1007/s00521-020-05511-4
Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)
Jaiswal, S., Nandi, G.C.: Robust real-time emotion detection system using CNN architecture. Neural Comput. Appl. 32(15), 11253–11262 (2019). https://doi.org/10.1007/s00521-019-04564-4
Goodfellow, J., Erthan, D., Carrier, P., Cournville, et al.: Challenges in representation learning: a report on three machine learning contests. Lecture Notes in Computer Science, vol. 8228 (2013). https://doi.org/10.1007/978-3-642-42051-1_16
Lucey, P., Cohn, J., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition – Workshops, San Francisco, CA, pp. 94–101 (2010)
Dachapally, P.: Facial emotion detection using convolutional neural networks and representational autoencoder units. arXiv, abs/1706.01509 (2017)
Answar, I., Islam, N.: Learned features are better for ethnicity classification. Cybern. Inf. Technol. 17(3), 152–164 (2017)
Deng, M., Meng, T., Cao, J., Wang, S., Zhang, J., Fan, H.: Heart sound classification based on improved MFCC features and convolutional recurrent neural networks. Neural Netw. 130, 22–32 (2020)
Ntalampiras, S.: A transfer learning framework for predicting the emotional content of generalized sound events. J. Acoust. Soc. Am. 141(3), 1694–1701 (2017)
Khamparia, A., Gupta, D., Nguyen, N., Khanna, A., Pandey, B., Tiwari, P.: Sound classification using convolutional neural network and tensor deep stacking network. IEEE Access 7, 7717–7727 (2019)
Piczak, K.: ESC: dataset for environmental sound classification. In: Proceedings of the 23rd ACM International Conference on Multimedia (2015)
Pinto, H., Almeida, J.M., Goncalves, M.A.: Using early view patterns to predict the popularity of Youtube videos. In: Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, pp. 365–374 (2013)
Chen, X., Chen, J., Ma, L., Liu, W., Luo, J., Zhang, T.: Fine-grained video attractiveness prediction using multimodal deep learning on a large real-world dataset. In: Companion Proceedings of the Web Conference, pp. 671–678 (2018)
Li, Y., Eng, K., Zhang, L.: Youtube Videos Prediction: Will this video be popular? Stanford University (2019)
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollar, P.: Focal loss for dense object detection. In: Proceedings of the IEEE, (ICCV), pp. 2980–2988 (2017)
Ruiz, N., Chong, E., Rehg, J.M.: Fine-grained head pose estimation without keypoints. In: IEEE Computer Vision and Pattern Recognition Workshops (2018)
Mohammad, S., Mohammad, H., Barlet, K., Trinh, P., Cohn, F.: DISFA: a spontaneous facial action intensity database. IEEE Trans. Affect. Comput. 4(2), 151–160 (2012)
Shao, Z., Liu, Z., Cail, J., Ma, L.: JAA-Net: joint facial action unit detection and face alignment via adaptive attention. Int. J. Comput. Vision 129, 321–340 (2021)
Vafeiadis, A., et al.: Acoustic scene classification: from a hybrid classifier to deep learning. In: DCASE Workshop, Munich, Germany, pp. 123–127 (2017)
Koestinger, M., Wohlhart, P., Roth, P., Bischof, H.: Annotated facial landmarks in the wild: a largescale, real-world database for facial landmark localization. In: IEEE International Conference on Computer Vision Workshops, Barcelona, pp. 2144–2151 (2011)
Sasaki, Y.: The truth of the F-measure. Teach Tutor Master (2007)
Willmott, C.J., Matsuura, K.: Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Res. 30, 79–82 (2005)
Acknowledgments
This research has been financed by the European Regional Development Fund of the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH –CREATE –INNOVATE (project code: LiveMedia++, T1EDK-04943). This support is gratefully acknowledged.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; nor in the decision to publish the results.
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Vrochidis, A., Dimitriou, N., Krinidis, S., Panagiotidis, S., Parcharidis, S., Tzovaras, D. (2021). A Multi-modal Audience Analysis System for Predicting Popularity of Online Videos. In: Iliadis, L., Macintyre, J., Jayne, C., Pimenidis, E. (eds) Proceedings of the 22nd Engineering Applications of Neural Networks Conference. EANN 2021. Proceedings of the International Neural Networks Society, vol 3. Springer, Cham. https://doi.org/10.1007/978-3-030-80568-5_38
Download citation
DOI: https://doi.org/10.1007/978-3-030-80568-5_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-80567-8
Online ISBN: 978-3-030-80568-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)