Abstract
Transformers gave a considerable boost to Natural Language Processing, but their application to specific scenarios still poses some practical issues. We present an approach for extracting information from technical documents on different domains, with minimal effort. It leverages on generic models for Question Answering and on questions formulated with target properties in mind. These are made to specific sections where the answer, then used as the value for the property, should reside. We further describe how this approach was applied to documents of two very different domains: toxicology and finance. For both, results extracted from a sample of documents were assessed by domain experts, who also provided feedback on the benefits of this approach. F-Scores of 0.73 and 0.90, respectively in the toxicological and financial domain, confirm the potential and flexibility of the approach suggesting that, while it cannot yet be fully automated and replace human work, it can support expert decisions, thus reducing time and manual effort.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
deepset/bert-base-cased-squad2.
- 4.
deepset/roberta-base-squad2.
- 5.
recobo/chemical-bert-uncased-squad2.
- 6.
dmis-lab/biobert-base-cased-v1.1-squad.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
References
Adnan, K., Akbar, R.: Limitations of information extraction methods and techniques for heterogeneous unstructured big data. Int. J. Eng. Bus. Manag. 11, 1847979019890771 (2019)
Arici, T., Kumar, K., Çeker, H., Saladi, A.S., Tutar, I.: Solving price per unit problem around the world: formulating fact extraction as question answering. arXiv preprint arXiv:2204.05555 (2022)
Baradaran, R., Ghiasi, R., Amirkhani, H.: A survey on machine reading comprehension systems. Nat. Lang. Eng. 28(6), 683–732 (2022)
Bhutani, N., Suhara, Y., Tan, W., Halevy, A.Y., Jagadish, H.V.: Open information extraction from question-answer pairs. CoRR abs/1903.00172 (2019). http://arxiv.org/abs/1903.00172
Brown, T., et al.: Language models are few-shot learners. In: Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901. Curran Associates, Inc. (2020)
Chithrananda, S., Grand, G., Ramsundar, B.: ChemBERTa: large-scale self-supervised pretraining for molecular property prediction. CoRR abs/2010.09885 (2020). https://arxiv.org/abs/2010.09885
Dedić, N., Stanier, C.: Towards differentiating business intelligence, big data, data analytics and knowledge discovery. In: Piazolo, F., Geist, V., Brehm, L., Schmidt, R. (eds.) ERP Future 2016. LNBIP, vol. 285, pp. 114–122. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58801-8_10
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Minnesota (Volume 1: Long and Short Papers), pp. 4171–4186. ACL (2019)
Lee, J., et al.: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. CoRR abs/1901.08746 (2019). http://arxiv.org/abs/1901.08746
Li, F., et al.: Event extraction as multi-turn question answering. In: Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 829–838. ACL (2020)
Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. CoRR abs/1907.11692 (2019). http://arxiv.org/abs/1907.11692
Lomotey, R.K., Deters, R.: Topics and terms mining in unstructured data stores. In: 2013 IEEE 16th International Conference on Computational Science and Engineering, pp. 854–861 (2013). https://doi.org/10.1109/CSE.2013.129
McKinsey &Company: The great reset: North American asset management in 2022. Global Asset Management Practice (2022). https://mck.co/3lXfxVm
Nguyen, M.T., Le, D.T., Le, L.: Transformers-based information extraction with limited data for domain-specific business documents. Eng. Appl. Artif. Intell. 97, 104100 (2021)
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(140), 1–67 (2020)
Rajpurkar, P., Jia, R., Liang, P.: Know what you don’t know: unanswerable questions for SQuAD. In: Proceedings of 56th Annual Meeting of the Association for Computational Linguistics (Vol 2: Short Papers), pp. 784–789. ACL (2018)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Wang, Y., Kung, L., Byrd, T.A.: Big data analytics: understanding its capabilities and potential benefits for healthcare organizations. Technol. Forecast. Soc. Chang. 126, 3–13 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 Springer Nature Switzerland AG
About this paper
Cite this paper
Ferreira, B.C.L., Oliveira, H.G., Silva, C. (2024). Leveraging Question Answering for Domain-Agnostic Information Extraction. In: Vasconcelos, V., Domingues, I., Paredes, S. (eds) Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. CIARP 2023. Lecture Notes in Computer Science, vol 14469. Springer, Cham. https://doi.org/10.1007/978-3-031-49018-7_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-49018-7_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-49017-0
Online ISBN: 978-3-031-49018-7
eBook Packages: Computer ScienceComputer Science (R0)
