Abstract
The biggest obstacles facing multi-document summarization include much more complicated input and excessive redundancy in source contents. Most state-of-the-art systems have attempted to tackle the redundancy problem, treating the entire input as a flat sequence. However, correlations among documents are often neglected. In this paper, we propose an end-to-end summarization model called MLT, which can effectively learn interactions at multiple levels and avoid redundant information. Specifically, we utilize a word-level transformer layer to encode contextual information within each sentence. Also, we design a sentence-level transformer layer for learning relations between sentences within a single document, as well as a document-level layer for learning interactions among input documents. Moreover, we use a neural method to enhance Max Marginal Relevance (MMR), a powerful algorithm for redundancy reduction. We incorporate MMR into our model and measure the redundancy quantitively based on the sentence representations. On benchmark datasets, our system compares favorably to strong summarization baselines judged by automatic metrics and human evaluators.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Antognini, D., Faltings, B.: Learning to create sentence semantic relation graphs for multi-document summarization. EMNLP-IJCNLP 2019, 32 (2019)
Carbonell, J.G., Goldstein, J.: The use of MMR, diversity-based reranking for reordering documents and producing summaries. In: SIGIR, pp. 335–336. ACM (1998)
Cho, S., Lebanoff, L., Foroosh, H., Liu, F.: Improving the similarity measure of determinantal point processes for extractive multi-document summarization. In: ACL, vol. 1, pp. 1027–1038. Association for Computational Linguistics (2019)
Chu, E., Liu, P.J.: MeanSum: a neural model for unsupervised multi-document abstractive summarization. In: ICML. Proceedings of Machine Learning Research, vol. 97, pp. 1223–1232. PMLR (2019)
Erkan, G., Radev, D.R.: LexRank: graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res. 22, 457–479 (2004)
Fabbri, A.R., Li, I., She, T., Li, S., Radev, D.R.: Multi-news: a large-scale multi-document summarization dataset and abstractive hierarchical model. In: ACL, vol. 1, pp. 1074–1084. Association for Computational Linguistics (2019)
Gehrmann, S., Deng, Y., Rush, A.M.: Bottom-up abstractive summarization. In: EMNLP, pp. 4098–4109. Association for Computational Linguistics (2018)
Hao, T.: Overview of DUC 2005. In: Proceedings of the Document Understanding Conference (DUC 2005) (2005)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (Poster) (2015)
Kiritchenko, S., Mohammad, S.: Best-worst scaling more reliable than rating scales: a case study on sentiment intensity annotation. In: ACL, vol. 2, pp. 465–470. Association for Computational Linguistics (2017)
Lebanoff, L., Song, K., Liu, F.: Adapting the neural encoder-decoder framework from single to multi-document summarization. In: EMNLP, pp. 4131–4141. Association for Computational Linguistics (2018)
Lin, C.Y.: ROUGE: a package for automatic evaluation of summaries. In: Text Summarization Branches Out, pp. 74–81 (2004)
Liu, P.J., et al.: Generating Wikipedia by summarizing long sequences. In: ICLR (Poster). OpenReview.net (2018)
Liu, Y., Lapata, M.: Hierarchical transformers for multi-document summarization. In: ACL, vol. 1, pp. 5070–5081. Association for Computational Linguistics (2019)
Louviere, J.J., Flynn, T.N., Marley, A.A.J.: Best-Worst Scaling: Theory, Methods and Applications. Cambridge University Press, Cambridge (2015)
Mihalcea, R., Tarau, P.: TextRank: bringing order into text. In: EMNLP, pp. 404–411. ACL (2004)
See, A., Liu, P.J., Manning, C.D.: Get to the point: summarization with pointer-generator networks. In: ACL, vol. 1, pp. 1073–1083. Association for Computational Linguistics (2017)
Vaswani, A., et al.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)
Yasunaga, M., Zhang, R., Meelu, K., Pareek, A., Srinivasan, K., Radev, D.R.: Graph-based neural multi-document summarization. In: CoNLL, pp. 452–462. Association for Computational Linguistics (2017)
Zhang, J., Tan, J., Wan, X.: Adapting neural single-document summarization model for abstractive multi-document summarization: a pilot study. In: INLG, pp. 381–390. Association for Computational Linguistics (2018)
Acknowledgments
This research work has been funded by the National Natural Science Foundation of China (Grant No. 61772337, U1736207) and the National Key R&D Program of China (2018YFC0830700).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, Y., Fan, X., Zhou, J., Liu, G. (2020). Learning Interactions at Multiple Levels for Abstractive Multi-document Summarization. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Communications in Computer and Information Science, vol 1332. Springer, Cham. https://doi.org/10.1007/978-3-030-63820-7_77
Download citation
DOI: https://doi.org/10.1007/978-3-030-63820-7_77
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63819-1
Online ISBN: 978-3-030-63820-7
eBook Packages: Computer ScienceComputer Science (R0)