Abstract
Merging information from physical and digital documents is essential in an era when information is becoming even more relevant. Different strategies have been used to combine knowledge from these two data sources. One state-of-the-art data extraction approach for this problem is the Named Entity Recognition (NER) strategy. However, even for those advanced models, the performance is still highly dependent on the Optical Character Recognition (OCR) system used to read the text from the physical documents. This paper investigates this dependence and how altering OCR systems between the training and inference phases influences NER performance. We verified that changing the OCR system negatively impacts the performance of data extraction models. Furthermore, we also show that models trained on less accurate OCR are more robust to OCR changes in the inference phase. The most accurate one regarding OCR errors should be preferred in scenarios where the OCR system is the same in the training and inference stages. We also propose a solution to mitigate this problem by mixing OCRs during the training phase. This approach enhances the model’s robustness while simultaneously preserving a high F1-score.
This study was financed in part by the founding public agencies: Coordenação de Aperfeiçoamento de Pessoal de Nível Superior—Brasil (CAPES)—Finance Code 001, CNPq, and FACEPE. In addition, we acknowledge all the support of Di2Win (www.di2win.com) during the development of this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. CoRR (2018). http://arxiv.org/abs/1810.04805
Dosovitskiy, A., et al.: An image is worth \(16 \times 16\) words: transformers for image recognition at scale (2020). https://doi.org/10.48550/ARXIV.2010.11929
Gaizauskas, R., Wilks, Y.: Information extraction: beyond document retrieval. J. Documentation 54, 70–105 (1998)
Garncarek, L., Powalski, R., Stanislawek, T., Topolski, B., Halama, P., Gralinski, F.: LAMBERT: layout-aware language modeling using BERT for information extraction. CoRR (2020). https://arxiv.org/abs/2002.08087
Hamdi, A., Jean-Caurant, A., Sidere, N., Coustaty, M., Doucet, A.: An analysis of the performance of named entity recognition over OCRed documents. In: Proceedings of the 18th Joint Conference on Digital Libraries, JCDL 20919, pp. 333–334. IEEE Press (2020). https://doi.org/10.1109/JCDL.2019.00057
Hamdi, A., Jean-Caurant, A., Sidère, N., Coustaty, M., Doucet, A.: Assessing and minimizing the impact of OCR quality on named entity recognition. In: Hall, M., Merčun, T., Risse, T., Duchateau, F. (eds.) TPDL 2020. LNCS, vol. 12246, pp. 87–101. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-54956-5_7
Hamdi, A., Linhares Pontes, E., Sidère, N., Coustaty, M., Doucet, A.: In-depth analysis of the impact of OCR errors on named entity recognition and linking. Nat. Lang. Eng. 29(2), 425–448 (2022). https://doi.org/10.1017/S1351324922000110. https://hal.science/hal-03615997
Huang, Y., Lv, T., Cui, L., Lu, Y., Wei, F.: LayoutLMv3: pre-training for document AI with unified text and image masking (2022). https://doi.org/10.48550/ARXIV.2204.08387
Kim, W., Son, B., Kim, I.: ViLT: vision-and-language transformer without convolution or region supervision (2021). https://doi.org/10.48550/ARXIV.2102.03334
Koudoro-Parfait, C., Lejeune, G., Roe, G.: Spatial named entity recognition in literary texts: what is the influence of OCR noise? In: Proceedings of the 5th ACM SIGSPATIAL International Workshop on Geospatial Humanities, GeoHumanities 2021, pp. 13–21. Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3486187.3490206
Lewis, D., Agam, G., Argamon, S., Frieder, O., Grossman, D., Heard, J.: Building a test collection for complex document information processing. In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2006, pp. 665–666. Association for Computing Machinery, New York (2006). https://doi.org/10.1145/1148170.1148307
Li, C., et al.: PP-OCRv3: more attempts for the improvement of ultra lightweight OCR system (2022). https://doi.org/10.48550/ARXIV.2206.03001
Lin, T., Dollár, P., Girshick, R.B., He, K., Hariharan, B., Belongie, S.J.: Feature pyramid networks for object detection. CoRR (2016). http://arxiv.org/abs/1612.03144
Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach (2019)
Park, S., et al.: Cord: a consolidated receipt dataset for post-OCR parsing (2019)
Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, vol. 32, pp. 8024–8035. Curran Associates, Inc. (2019). http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
Vaswani, A., et al.: Attention is all you need. CoRR (2017). http://arxiv.org/abs/1706.03762
Wang, J., Jin, L., Ding, K.: Lilt: a simple yet effective language-independent layout transformer for structured document understanding (2022). https://doi.org/10.48550/ARXIV.2202.13669
Wolf, T., et al.: Huggingface’s transformers: state-of-the-art natural language processing. CoRR (2019). http://arxiv.org/abs/1910.03771
Xie, S., Girshick, R., Dollar, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2017). https://doi.org/10.1109/cvpr.2017.634
Xu, Y., et al.: LayoutLMv2: multi-modal pre-training for visually-rich document understanding. CoRR (2020). https://arxiv.org/abs/2012.14740
Xu, Y., Li, M., Cui, L., Huang, S., Wei, F., Zhou, M.: LayoutLM: pre-training of text and layout for document image understanding. CoRR (2019). http://arxiv.org/abs/1912.13318
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Macedo, J., Bezerra, B., Zanchettin, C. (2024). How Does Changing the Optical Character Recognition System Impact the Layout-Aware Named Entity Recognition Models?. In: Sfikas, G., Retsinas, G. (eds) Document Analysis Systems. DAS 2024. Lecture Notes in Computer Science, vol 14994. Springer, Cham. https://doi.org/10.1007/978-3-031-70442-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-70442-0_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-70441-3
Online ISBN: 978-3-031-70442-0
eBook Packages: Computer ScienceComputer Science (R0)
