{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T16:38:12Z","timestamp":1726850292253},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,7]]},"abstract":"Current pre-training works in natural language generation pay little attention to the problem of exposure bias on downstream tasks. To address this issue, we propose an enhanced multi-flow sequence to sequence pre-training and fine-tuning framework named ERNIE-GEN, which bridges the discrepancy between training and inference with an infilling generation mechanism and a noise-aware generation method. To make generation closer to human writing patterns, this framework introduces a span-by-span generation flow that trains the model to predict semantically-complete spans consecutively rather than predicting word by word. Unlike existing pre-training methods, ERNIE-GEN incorporates multi-granularity target sampling to construct pre-training data, which enhances the correlation between encoder and decoder. Experimental results demonstrate that ERNIE-GEN achieves state-of-the-art results with a much smaller amount of pre-training data and parameters on a range of language generation tasks, including abstractive summarization (Gigaword and CNN\/DailyMail), question generation (SQuAD), dialogue generation (Persona-Chat) and generative question answering (CoQA). The source codes and pre-trained models have been released at https:\/\/github.com\/PaddlePaddle\/ERNIE\/ernie-gen.<\/jats:p>","DOI":"10.24963\/ijcai.2020\/553","type":"proceedings-article","created":{"date-parts":[[2020,7,8]],"date-time":"2020-07-08T12:12:10Z","timestamp":1594210330000},"page":"3997-4003","source":"Crossref","is-referenced-by-count":49,"title":["ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation"],"prefix":"10.24963","author":[{"given":"Dongling","family":"Xiao","sequence":"first","affiliation":[{"name":"Baidu, Inc"}]},{"given":"Han","family":"Zhang","sequence":"additional","affiliation":[{"name":"Baidu, Inc"}]},{"given":"Yukun","family":"Li","sequence":"additional","affiliation":[{"name":"Baidu, Inc"}]},{"given":"Yu","family":"Sun","sequence":"additional","affiliation":[{"name":"Baidu, Inc"}]},{"given":"Hao","family":"Tian","sequence":"additional","affiliation":[{"name":"Baidu, Inc"}]},{"given":"Hua","family":"Wu","sequence":"additional","affiliation":[{"name":"Baidu, Inc"}]},{"given":"Haifeng","family":"Wang","sequence":"additional","affiliation":[{"name":"Baidu, Inc"}]}],"member":"10584","event":{"number":"28","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)"],"acronym":"IJCAI-PRICAI-2020","name":"Twenty-Ninth International Joint Conference on Artificial Intelligence and Seventeenth Pacific Rim International Conference on Artificial Intelligence {IJCAI-PRICAI-20}","start":{"date-parts":[[2020,7,11]]},"theme":"Artificial Intelligence","location":"Yokohama, Japan","end":{"date-parts":[[2020,7,17]]}},"container-title":["Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2020,7,9]],"date-time":"2020-07-09T02:15:54Z","timestamp":1594260954000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2020\/553"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2020,7]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2020\/553","relation":{},"subject":[],"published":{"date-parts":[[2020,7]]}}}