Abstract
In recent years, the challenge of extracting information from business documents has emerged as a critical task, finding applications across numerous domains. This effort has attracted substantial interest from both industry and academy, highlighting its significance in the current technological landscape. Most datasets in this area are primarily focused on Key Information Extraction (KIE), where the extraction process revolves around extracting information using a specific, predefined set of keys. Unlike most existing datasets and benchmarks, our focus is on discovering key-value pairs (KVPs) without relying on predefined keys, navigating through an array of diverse templates and complex layouts. This task presents unique challenges, primarily due to the absence of comprehensive datasets and benchmarks tailored for non-predetermined KVP extraction. To address this gap, we introduce KVP10k , a new dataset and benchmark specifically designed for KVP extraction. The dataset contains 10707richly annotated images. In our benchmark, we also introduce a new challenging task that combines elements of KIE as well as KVP in a single task. KVP10k sets itself apart with its extensive diversity in data and richly detailed annotations, paving the way for advancements in the field of information extraction from complex business documents.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Beltagy,I., Peters, M.E., Cohan, A.: Longformer: the longdocument transformer. arXiv preprint arXiv:2004.05150 (2020)
Ding, Y., et al.: Form-NLU: dataset for the form natural language understanding. In: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2807–2816 (2023)
Hong, T., et al.: Bros: A pre-trained language model for understanding texts in document. (2020)
Huang, Y., et al.: Layoutlmv3: Pre-training for document AI with unified text and image masking. In: Proceedings of the 30th ACM International Conference on Multimedia, pp. 4083–4091 (2022)
Huang, Z., et al.: Icdar2019 competition on scanned receipt OCR and information extraction. In: 2019 International Conference on Document Analysis and Recognition (ICDAR). IEEE, pp. 1516–1520 (2019)
Hwang, W., et al.: Spatial dependency parsing for semi-structured document information extraction. arXiv preprint arXiv:2005.00642 (2020)
Jaume, G., Ekenel, H.K., Thiran, J.P.: Funsd: a dataset for form understanding in noisy scanned documents. In: 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW), Vol. 2, pp. 1–6. IEEE (2019)
Jiang, A.Q., et al.: Mistral 7B. arXiv preprint arXiv:2310.06825 (2023)
Lee, C.Y., et al.: FormNetV2: multimodal Graph Contrastive Learning for Form Document Information Extraction. arXiv preprint arXiv:2305.02549 (2023)
Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Mathew, M., Karatzas, D., Jawahar, C.V.: Docvqa: a dataset for VQA on document images. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2200–2209(2021)
Mathur, P., et al.: LayerDoc: layer-wise extraction of spatial hierarchical structure in visually-rich documents. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3610–3620 (2023)
Naparstek, O., et al.: BusiNet-a light and fast text detection network for business documents. I arXiv preprint arXiv:2207.01220 (2022)
Park, S., et al.: CORD: a consolidated receipt dataset for post- OCR parsing. In: Workshop on Document Intelligence at NeurIPS 2019 (2019)
Perot, V., et al.: LMDX: Language Model-based Document Information Extraction and Localization. arXiv preprint arXiv:2309.10952 (2023)
Smith, R.: An overview of the Tesseract OCR engine. In: Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), Vol. 2, pp. 629–633. IEEE (2007)
Stanisławek, T., et al.: Kleister: key information extraction datasets involving long documents with complex layouts. In: International Conference on Document Analysis and Recognition, pp. 564–579. Springer (2021). https://doi.org/10.1007/978-3-030-86549-8_36
Wang, J., et al.: Towards robust visual information extraction in real world: new dataset and novel solution. In: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35 no. 4 , pp. 2738–2745 (2021)
Wang, Z., et al.: DocStruct: a multimodal method to extract hierarchy structure in document for general form understanding. arXiv preprint arXiv:2010.11685 (2020)
Wang, Z., et al.: VRDU: a benchmark for visually-rich document understanding. In: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 5184–5193 (2023)
Xu, Y., et al.: XFUND: a benchmark dataset for multilingual visually rich form understanding. In: Findings of the Association for Computational Linguistics: ACL 2022, pp. 3214–3224 (2022)
Yang, Z., et al.: Modeling entities as semantic points for visual information extraction in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15358–15367 (2023)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendices
A Annotation Format
An exaple for flat KVP annotation is given in listing 1.2. An exaple for unkeyed value annotation is given in listing 1.3 and an example with a section is given in 1.4. Note that in the annotation the x any y coordinates are relative to the page size and take values between 0 and 1
B Additional Statistics
Here we provide mode detailed statistics of the data. Figure 7 is a histogram that specifically focuses on documents sourced from public files. Figure 8, on the other hand, is a histogram that specifically focuses on documents sourced from web crawl.
These plots provides an overview of the variability in the number of entities in total, including key-value pairs and other types of entities, across different documents and within specific sources. The aim is to offer a comprehensive view of the dataset’s characteristics, highlighting variations in document complexity across different sources.
Distribution of entities per page in KVP10k sourced from Public files.
In addition to the overall distribution, we have generated two more pie charts to examine the distribution of entity labels within specific subsets of the dataset:
-
Figure 9 focuses on documents sourced from public files.
-
Figure 10 specifically examines documents sourced from web crawl.
The following pie charts offer a more intuitive representation of the distribution of entity labels in total across different documents and within specific sources, facilitating a better understanding of the characteristics of the dataset.
Distribution of entities per page in KVP10k sourced from web crawl.
Distribution of entity labels in KVP10k sourced from Public files.
Distribution of entity labels in KVP10k sourced from web crawl.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Naparstek, O. et al. (2024). KVP10k : A Comprehensive Dataset for Key-Value Pair Extraction in Business Documents. In: Barney Smith, E.H., Liwicki, M., Peng, L. (eds) Document Analysis and Recognition - ICDAR 2024. ICDAR 2024. Lecture Notes in Computer Science, vol 14804. Springer, Cham. https://doi.org/10.1007/978-3-031-70533-5_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-70533-5_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-70532-8
Online ISBN: 978-3-031-70533-5
eBook Packages: Computer ScienceComputer Science (R0)
