{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,23]],"date-time":"2024-08-23T17:47:18Z","timestamp":1724435238320},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"4","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2021,11,30]]},"abstract":"\n Live video broadcasting normally requires a multitude of skills and expertise with domain knowledge to enable multi-camera productions. As the number of cameras keeps increasing, directing a live sports broadcast has now become more complicated and challenging than ever before. The broadcast directors need to be much more concentrated, responsive, and knowledgeable, during the production. To relieve the directors from their intensive efforts, we develop an innovative automated sports broadcast directing system, called Smart Director, which aims at mimicking the typical human-in-the-loop broadcasting process to automatically create near-professional broadcasting programs in real-time by using a set of advanced multi-view video analysis algorithms. Inspired by the so-called \u201cthree-event\u201d construction of sports broadcast [\n 14<\/jats:xref>\n ], we build our system with an event-driven pipeline consisting of three consecutive novel components: (1) the\n Multi-View Event Localization<\/jats:italic>\n to detect events by modeling multi-view correlations, (2) the\n Multi-View Highlight Detection<\/jats:italic>\n to rank camera views by the visual importance for view selection, and (3) the\n Auto-Broadcasting Scheduler<\/jats:italic>\n to control the production of broadcasting videos. To our best knowledge, our system is the first end-to-end automated directing system for multi-camera sports broadcasting, completely driven by the semantic understanding of sports events. It is also the first system to solve the novel problem of multi-view joint event detection by cross-view relation modeling. We conduct both objective and subjective evaluations on a real-world multi-camera soccer dataset, which demonstrate the quality of our auto-generated videos is comparable to that of the human-directed videos. Thanks to its faster response, our system is able to capture more fast-passing and short-duration events which are usually missed by human directors.\n <\/jats:p>","DOI":"10.1145\/3448981","type":"journal-article","created":{"date-parts":[[2021,11,12]],"date-time":"2021-11-12T21:16:06Z","timestamp":1636751766000},"page":"1-18","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["Smart Director: An Event-Driven Directing System for Live Broadcasting"],"prefix":"10.1145","volume":"17","author":[{"given":"Yingwei","family":"Pan","sequence":"first","affiliation":[{"name":"JD AI Research, Beijing, China"}]},{"given":"Yue","family":"Chen","sequence":"additional","affiliation":[{"name":"JD AI Research, Beijing, China"}]},{"given":"Qian","family":"Bao","sequence":"additional","affiliation":[{"name":"JD AI Research, Beijing, China"}]},{"given":"Ning","family":"Zhang","sequence":"additional","affiliation":[{"name":"JD AI Research, USA"}]},{"given":"Ting","family":"Yao","sequence":"additional","affiliation":[{"name":"JD AI Research, Beijing, China"}]},{"given":"Jingen","family":"Liu","sequence":"additional","affiliation":[{"name":"JD AI Research, USA"}]},{"given":"Tao","family":"Mei","sequence":"additional","affiliation":[{"name":"JD AI Research, Beijing, China"}]}],"member":"320","published-online":{"date-parts":[[2021,11,12]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1177\/2167479513479107"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.675"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01172"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICME.2013.6607445"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2019.00305"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2019.00184"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2018.00053"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3343031.3350937"},{"key":"e_1_3_3_10_2","first-page":"193","volume-title":"Proceedings of the Korean Society of Broadcast Engineers Conference","author":"Choi Kyu-Hyoung","year":"2009","unstructured":"Kyu-Hyoung Choi, Sang-Wook Lee, and Yong-Duek Seo. 2009. Automatic broadcast video generation for ball sports from multiple views. In Proceedings of the Korean Society of Broadcast Engineers Conference. The Korean Institute of Broadcast and Media Engineers, 193\u2013198."},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVMP.2011.8"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00712"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.65"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01216-8_5"},{"key":"e_1_3_3_15_2","volume-title":"Playing for Keeps: Sport, the Media and Society","author":"Goldlust John","year":"2018","unstructured":"John Goldlust. 2018. Playing for Keeps: Sport, the Media and Society. Hybrid Publishers."},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.632"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2016.2573042"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2018.2806224"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.390"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073653"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-94211-7_9"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-05716-9_18"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3328994"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00782"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2019.2939711"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3123266.3123343"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00043"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-018-1120-4"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.228"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781315770000"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.5555\/3061053.3061155"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/2766462.2767725"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01098"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/2647868.2656404"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.5555\/876865.877098"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10599-4_35"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.590"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1186\/s40064-015-1065-9"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/354384.354443"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2017.2655624"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.119"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00036"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00530-008-0112-6"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/1027527.1027535"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/2964284.2967265"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISM.2014.44"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00135"},{"key":"e_1_3_3_48_2","volume-title":"CVPR","author":"Yao Ting","year":"2016","unstructured":"Ting Yao, Tao Mei, and Yong Rui. 2016. Highlight detection with pairwise deep ranking for first-person video summarization. In CVPR."},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01264-9_42"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00271"},{"key":"e_1_3_3_51_2","article-title":"SeCo: Exploring sequence supervision for unsupervised representation learning","volume":"2008","author":"Yao Ting","year":"2020","unstructured":"Ting Yao, Yiheng Zhang, Zhaofan Qiu, Yingwei Pan, and Tao Mei. 2020. SeCo: Exploring sequence supervision for unsupervised representation learning. CoRR abs\/2008.00975. arxiv:2008.00975https:\/\/arxiv.org\/abs\/2008.00975","journal-title":"CoRR"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.1997.638604"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46478-7_47"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01237-3_24"},{"key":"e_1_3_3_55_2","article-title":"Robust visual object tracking with two-stream residual convolutional networks","volume":"2005","author":"Zhang Ning","year":"2020","unstructured":"Ning Zhang, Jingen Liu, Ke Wang, Dan Zeng, and Tao Mei. 2020. Robust visual object tracking with two-stream residual convolutional networks. CoRR abs\/2005.06536. arxiv:2005.06536https:\/\/arxiv.org\/abs\/2005.06536","journal-title":"CoRR"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/BTAS.2017.8272675"},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01017"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.317"},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394171.3414453"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3448981","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T22:03:56Z","timestamp":1672610636000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3448981"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,12]]},"references-count":58,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,11,30]]}},"alternative-id":["10.1145\/3448981"],"URL":"https:\/\/doi.org\/10.1145\/3448981","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,12]]},"assertion":[{"value":"2020-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-02-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-11-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}