A video engine supported by social buzz to automatically create TV summaries | Multimedia Tools and Applications Skip to main content
Springer Nature Link
Log in

A video engine supported by social buzz to automatically create TV summaries

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Viewers post a lot of TV program-related information on social networks while they are watching TV, especially during its key moments. Therefore, this social buzz has the potential to be used as an automatic editorial criterion. Having this premise in consideration, this paper reports on the nowUP solution, a service developed with the main goal of automatically creating TV summaries of popular television programs (like football matches, talent or reality shows) based on the Twitter activity and integrating a part of that activity in the TV show summary. A data-mining engine continuously processes the activity of this social network looking for tweets associated with the TV shows. Based on the program metadata it indexes the twitter activity; correlates tweets; and creates clusters of peaks, being the relevant clusters associated with the highlights of the TV show. With this, the video engine automatically creates a full video summary (an edited sequence of TV highlights) and publishes it in an online platform and on a Catch-up TV service. The paper reports on the nowUP development and on the results of its evaluation, namely comparing its outputs with official editorial/professional video summaries. The results show that the solution was very successful in achieving the project main goal and users want to have access to this type of social buzz-based video summaries. The nowUP solution also promises potential gains in the value chain of TV producers and broadcasters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Abel F, Gao Q, Houben GJ, Tao K (2011) Semantic enrichment of twitter posts for user profile construction on the social web. Proceedings of the 8th extended semantic web conference on The semanic web: research and applications - Volume Part II (ESWC'11). Springer-Verlag, Berlin, 375–389

  2. Abreu J, Almeida P, Teles B, Reis M (2013) Viewer behaviors and practices in the (new) Television Environment. Proceedings of the 11th European conference on Interactive TV and video (EuroITV '13). ACM, New York, 5–12. doi: 10.1145/2465958.2465970

  3. Abreu J, Nogueira J, Becker V (2016) Survey of Catch-up TV and Other Time-Shift Services: A Comprehensive Analysis and Taxonomy of Linear and Nonlinear Television. Telecommun Syst 64:57–74. https://doi.org/10.1007/s11235-016-0157-3

    Article  Google Scholar 

  4. Alonso O, Shiells K (2013) Timelines as summaries of popular scheduled events. Proceedings of the 22nd International World Wide Web Conference Committee (IW3C2):1037–1044. https://doi.org/10.1145/2487788.2488114

  5. Beuker I (2012) The Social EPG: The Next Step Towards Social TV? Viral Blog. http://www.viralblog.com/social-tv/the-social-epg-the-next-step-towards-social-tv/. Accessed 21 April 2017

  6. Bradley M, Lang P (1994) Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. Journal Behav. Ther. Exp. Psychiatry, Mar; 25(1):49–59. http://www.ncbi.nlm.nih.gov/pubmed/7962581

  7. Capon G (2014) Twitter Amplify - Harmonize the two-screen experience. The Guardian. http://www.theguardian.com/media-network/media-network-blog/2014/may/23/twitter-amplify-influence-tv. Accessed 21 April 2017

  8. Cremonesi P et al. (2013) TV program detection in tweets. Proceedings of the 11th European Conference on Interactive TV and Video (EuroITV '13). 45–54. doi: 10.1145/2465958.2465960

  9. Crisci A, Grasso V, Nesi P et al (2017) Predicting TV programme audience by using twitter based metrics. Multimed Tools Appl, Published Online:1–30. https://doi.org/10.1007/s11042-017-4880-x

  10. Doman K et al. (2014) Event Detection based on Twitter Enthusiasm Degree for Generating a Sports Highlight Video. Proceedings of 22nd ACM International Conference on Multimedia (ACM-MM2014), pp.949–952. doi: https://doi.org/10.1145/2647868.2654973

  11. Dumont E, Quénot G (2012) Automatic Story Segmentation for TV News Video Using Multiple Modalities. Int J Digit Multimed Broadcast 2012:11. https://doi.org/10.1155/2012/732514

    Google Scholar 

  12. Flick U (2009) An introduction to qualitative research (4th ed.). Sage Publications, London

  13. Forlines C, Peker K, Divakaran A (2006) Subjective assessment of consumer video summarization. SPIE Proceedings - Special Session: Evaluating Video Summarization, Browsing, and Retrieval Techniques 6073. https://doi.org/10.1117/12.648554

  14. Genc Y, Sakamoto Y, Nickerson J (2011) Discovering context: classifying tweets through a semantic transform based on wikipedia. Proceedings of the 6th international conference on Foundations of augmented cognition: directing the future of adaptive systems (FAC'11). Springer-Verlag, Berlin, 484–492

  15. Jiang W, Cotton C, Loui A (2011) Automatic consumer video summarization by audio and visual analysis. Proceedings of International Conference on Multimedia and Expo (ICME):1–6. https://doi.org/10.1109/ICME.2011.6011841

  16. Kantar Media (2015) Who's Tweeting about TV in the UK? Kantar Media. http://www.kantarmedia.com/uk/thinking-resources/latest-thinking/who-is-tweeting-about-tv-in-the-uk. Accessed 21 April 2017

  17. Kim H-G et al (2008) Real-Time Highlight Detection in Baseball Video for TVs with Time-shift Function. IEEE Transactions on Consumer Electronics, Volume 54(2):831–838. https://doi.org/10.1109/TCE.2008.4560167

    Article  Google Scholar 

  18. Marlow S et al. (2002) Audio processing for automatic TV sports program highlights detection. Proceedings of Irish Signals and Systems Conference (ISSC 2002), 25–26. http://doras.dcu.ie/326/

  19. Nielsen Social (2017) Social TV Analytics & Solutions. Nielsen Social. http://www.nielsensocial.com/. Accessed 21 April 2017

  20. Nielsen Social (2017) TV Season in review: the top social moments of the 2016–17 season. Nielsen Social. http://www.nielsen.com/us/en/insights/news/2017/tv-season-in-review-the-top-social-moments-of-the-2016-17-season.html. Accessed 30 Aug 2017

  21. Nielsen Social (2017) What We Do. Nielsen Social. http://www.nielsensocial.com/what-we-do/. Accessed 21 April 2017

  22. Otani M, Nakashima Y, Sato T et al (2017) Video summarization using textual descriptions for authoring video blogs. Multimed Tools Appl 76(9):12097–12115. https://doi.org/10.1007/s11042-016-4061-3

    Article  Google Scholar 

  23. Outhwaite W, Turner S (2007) The Sage Handbook of Social Science Methodology, 1st ed., Sage Publications

  24. Ozdikis O, Senkul P, Oguztuzun H (2012) Semantic expansion of hashtags for enhanced event detection in Twitter. Proceedings of the International Conference Advances in Social Networks Analysis and Mining (ASONAM). IEEE, 20–24. doi: 10.1109/ASONAM.2012.14

  25. Sgarbi E, Borges D (2005) Structure in Soccer Videos: Detecting and Classifying Highlights for Automatic Summarization. In: Sanfeliu A, Cortés ML (eds) Progress In Pattern Recognition, Image Analysis And Applications, Ciarp 2005, LNCS, vol 3773. Springer, Heidelberg, pp 691–700. https://doi.org/10.1007/11578079_72

    Google Scholar 

  26. Tellyo (2017) Media sharing. Tellyo. https://tellyo.com/. Accessed 21 April 2017

  27. Twitter Biz (2016) Twitter Amplify will create enormous value for broadcasters and brands. Twitter Business. https://business.twitter.com/pt/twitter-amplify. Accessed 11 Jan 2016

  28. Ulanoff L (2015) Twitter experiments with TV Timelines. Mashable. http://mashable.com/2015/03/12/twitter-experiment-tv-timelines/#Ip8nGr1eJqq3. Accessed 21 April 2017

  29. Viacom (2013) When Networks Network - TV Gets Social. Viacom Inc. http://vimninsights.viacom.com/post/61773538381/when-networks-network-tv-gets-social-in-our. Accessed 11 Jan 2016

  30. Vilaça A, Antunes M, Gomes D (2015) TV-Pulse: Detecting TV highlights in Social Networks. Proceedings of ConfTele 2015 - 10th Conference on Telecommunications, Aveiro, Portugal

  31. Vilaça A, Antunes M, Gomes D (2015) TV-Pulse: Improvements on detecting TV highlights in Social Networks using metadata and semantic similarity. Proceedings of 14th Conferência sobre Redes de Computadores, Évora, Portugal

  32. WildMoka (2014) Moments Capture. WildMoka Website. http://wildmoka.com/solutions/moments-capture/. Accessed 11 Jan 2016

  33. WildMoka (2014) Moments Share. WildMoka Website. http://wildmoka.com/solutions/moments-share/. Accessed 11 Jan 2016

  34. WildMoka (2014) Platform. WildMoka Website. http://wildmoka.com/platform/. Accessed 11 Jan 2016

  35. WildMoka (2014) Press Release – CANAL+ Group selects Moments Share. WildMoka Website. http://wildmoka.com/press-release-canal-group-selects-moments-share-wildmokas-social-tv-solution/. Accessed 11 Jan 2016

  36. WildMoka (2014) Moments Replay. WildMoka Website. htttp://wildmoka.com/solutions/moments-replay/. Accessed 11 Jan 2016

  37. Yang J, Huang Y, Tsai C et al. (2009) An Automatic Multimedia Content Summarization System for Video Recommendation. Educational Technol & Soc, 12 (1), 49–61. http://eric.ed.gov/?id=EJ833416

  38. Yang J, Li J, Liu S (2017) A novel technique applied to the economic investigation of recommender system. Multimed Tools Appl, Published Online:1–16. https://doi.org/10.1007/s11042-017-4752-4

  39. Zhang J, Li X, Nie W et al (2017) Automatic report generation based on multi-modal information. Multimed Tools Appl 76(9):12005–12015. https://doi.org/10.1007/s11042-016-3936-7

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to acknowledge the remaining partners of the nowUP project (Institute of Telecommunications of the University of Aveiro and Altice Labs).

Author information

Authors and Affiliations

  1. Digimedia, Department of Communication and Art, University of Aveiro, Aveiro, Portugal

    Pedro Almeida, Jorge Ferraz de Abreu & Rita Oliveira

  2. Institute of Telecommunications, Department of Electronics, Telecommunications and Informatics, University of Aveiro, Aveiro, Portugal

    Diogo Gomes

Authors
  1. Pedro Almeida
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jorge Ferraz de Abreu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rita Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Diogo Gomes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pedro Almeida.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Almeida, P., de Abreu, J.F., Oliveira, R. et al. A video engine supported by social buzz to automatically create TV summaries. Multimed Tools Appl 77, 24313–24331 (2018). https://doi.org/10.1007/s11042-018-5723-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-5723-0

Keywords

Search

Navigation