Abstract
Information Extraction (IE) aims to extract structural knowledge (e.g., entities, relations, events) from natural language texts. Recently, Large Language Models (LLMs) with code-style prompts have demonstrated powerful capabilities in IE tasks. However, adopting code LLMs to conduct IE tasks still has two challenges: (1) It still lacks a unified code-style prompt for different IE tasks since existing methods use task-specific prompts for separate IE tasks. (2) It still lacks an effective in-context learning (ICL) method to encourage LLMs to conduct IE tasks precisely, considering some powerful LLMs are close-sourced and not trainable. Therefore, this paper proposes a code generation framework for Universal IE (UIE) tasks called Code4UIE. Specifically, for the first challenge, Code4UIE designs a unified code-style schema for various IE tasks via Python classes. By so doing, different IE tasks can be associated, and LLMs can learn from various IE tasks effectively. For the second challenge, Code4UIE adopts a retrieval-augmented mechanism to comprehensively utilize the ICL ability of LLMs. Extensive experiments on five representative IE tasks across nine datasets demonstrate the effectiveness of the Code4UIE framework.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Brown, T., Mann, B., Ryder, N. et al.: Language models are few-shot learners. In: Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901 (2020)
Christopher, W., Stephanie, S., Julie, M., Kazuaki, M.: ACE 2005 multilingual training corpus LDC2006T06. In: Philadelphia: Linguistic Data Consortium, Web Download (2006)
Chung, H.W., Hou, L., Longpre, S., et al.: Scaling instruction-finetuned language models. J. Mach. Learn. Res. 25(70), 1–53 (2024)
Doddington, G.R., Mitchell, A., Przybocki, M.A. et al.: The automatic content extraction (ACE) program - tasks, data, and evaluation. In: Proceedings of the Fourth International Conference on Language Resources and Evaluation (2004)
Du, X. Cardie, C.: Event extraction by answering (almost) natural questions. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 671–683 (2020)
Guo, X., Wang, S., Zhao, H., et al.: Intelligent online selling point extraction for e-commerce recommendation. Proc. AAAI Conf. Artif. Intell. 36(11), 12360–12368 (2022)
Gurulingappa, H., Rajput, A.M., Roberts, A., et al.: Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports. J. Biomed. Inform. 45(5), 885–892 (2012)
Hsu, I.H., Huang, K.H., Boschee, E. et al.: DEGREE: a data-efficient generation-based event extraction model. In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1890–1908 (2022)
Li, B., Fang, G., Yang, Y. et al.: Evaluating ChatGPT’s information extraction capabilities: an assessment of performance, explainability, calibration, and faithfulness. arXiv preprint arXiv:2304.11633 (2023)
Li, P., Sun, T., Tang, Q. et al.: CodeIE: large code generation models are better few-shot information extractors. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15339–15353 (2023)
Lou, J., Lu, Y., Dai, D., et al.: Universal information extraction as unified semantic matching. Proc. AAAI Conf. Artif. Intell. 37(11), 13318–13326 (2023)
Lu, Y., Liu, Q., Dai, D. et al.: Unified structure generation for universal information extraction. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5755–5772 (2022)
Luan, Y., He, L., Ostendorf, M. Hajishirzi, H.: Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3219–3232 (2018)
Ouyang, L., Wu, J., Jiang, X. et al.: Training language models to follow instructions with human feedback. In: Advances in Neural Information Processing Systems, vol. 35, pp. 27730–27744 (2022)
Raffel, C., Shazeer, N., Roberts, A., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(140), 1–67 (2020)
Riedel, S., Yao, L. McCallum, A.: Modeling relations and their mentions without labeled text. In: Machine Learning and Knowledge Discovery in Databases, pp. 148–163 (2010)
Roth, D. Yih, W.t.: A linear programming formulation for global inference in natural language tasks. In: Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004, pp. 1–8 (2004)
Schweter, S. Akbik, A.: FLERT: document-level features for named entity recognition. arXiv preprint arXiv:2011.06993 (2020)
Song, K., Tan, X., Qin, T. et al.: MPNet: masked and permuted pre-training for language understanding. In: Advances in Neural Information Processing Systems, vol. 33, pp. 16857–16867 (2020)
Taneeya, W., Satyapanich, F.F., Finin, T.: CASIE: extracting cybersecurity event information from text. In: Proceeding of the 34th AAAI Conference on Artificial Intelligence (2020)
Tjong Kim Sang, E.F., De Meulder, F.: Introduction to the CoNLL-2003 shared task: language-independent named entity recognition. In: Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pp. 142–147 (2003)
Wadhwa, S., Amir, S., Wallace, B.: Revisiting relation extraction in the era of large language models. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15566–15589 (2023)
Wang, X., Zhou, W., Zu, C., et al.: InstructUIE: multi-task instruction tuning for unified information extraction. arXiv preprint arXiv:2304.08085 (2023)
Wang, X., Li, S., Ji, H.: Code4Struct: code generation for few-shot event structure prediction. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3640–3663 (2023)
Wei, X., Cui, X., Cheng, N., et al.: Zero-shot information extraction via chatting with ChatGPT. arXiv preprint arXiv:2302.10205 (2023)
Yan, Z., Tang, D., Duan, N., et al.: Assertion-based QA with question-aware open information extraction. Proc. AAAI Conf. Artif. Intell. 32(1) (2018)
Acknowledgments
The work is supported by the National Natural Science Foundation of China under grant No. 62306299, the JCJQ Project of China, the National Key Research and Development Program of China under grant No. 2022QY0703, the Beijing Academy of Artificial Intelligence under grant No. BAAI2019ZD0306, the KGJ Project under grant No. JCKY2022130C039, the Lenovo-CAS Joint Lab Youth Scientist Project, and the Strategic Priority Research Program of the CAS under grant No. XDB0680102. We thank anonymous reviewers for their insightful comments and suggestions.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Guo, Y. et al. (2025). Retrieval-Augmented Code Generation for Universal Information Extraction. In: Wong, D.F., Wei, Z., Yang, M. (eds) Natural Language Processing and Chinese Computing. NLPCC 2024. Lecture Notes in Computer Science(), vol 15360. Springer, Singapore. https://doi.org/10.1007/978-981-97-9434-8_3
Download citation
DOI: https://doi.org/10.1007/978-981-97-9434-8_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-9433-1
Online ISBN: 978-981-97-9434-8
eBook Packages: Computer ScienceComputer Science (R0)
