Abstract
Patent documents serve as a valuable source of information for companies and researchers when making decisions. Therefore, there is a demand for classifying patent documents and analyzing them. Topic modeling is a major approach to automatically extracting topics from documents, and it can be used for document clustering. Although we can apply existing topic modeling methods to the text body of patent documents for our intended purpose, most models do not fully take into account the relationships between patent documents and metadata entities, such as titles, abstracts, and classification codes. This paper explores a topic modeling approach that considers the relationships between patent documents and metadata entities, by developing and evaluating a novel method, named Metadata-Assisted Topic Modeling (MATopic). MATopic (1) generates more coherent and diverse topics by considering both the metadata of patents and their text, and (2) infers evidence about what components make up the topics by representing the detected topics as nodes in a knowledge graph. The experiments using real-world patent document datasets show that MATopic outperforms existing methods in topic diversity and coherence.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3(Jan), 993–1022 (2003)
Brambilla, M., Altinel, B.: Improving topic modeling for textual content with knowledge graph embeddings. In: AAAI Spring Symposium: Combining Machine Learning with Knowledge Engineering (2019)
Campbell, R.S.: Patent trends as a technological forecasting tool. World Patent Inf. 5(3), 137–143 (1983)
Choi, S., Park, H., Kang, D., Lee, J.Y., Kim, K.: An SAO-based text mining approach to building a technology tree for technology planning. Expert Syst. Appl. 39(13), 11443–11455 (2012)
Choi, Y., Park, S., Lee, S.: Identifying emerging technologies to envision a future innovation ecosystem: a machine learning approach to patent data. Scientometrics 126(7), 5431–5476 (2021). https://doi.org/10.1007/s11192-021-04001-1
Deng, D.: DBSCAN clustering algorithm based on density. In: 7th International Forum on Electrical Engineering and Automation (IFEEA), pp. 949–953. IEEE (2020)
Fang, L., Zhang, L., Wu, H., Xu, T., Zhou, D., Chen, E.: Patent2vec: multi-view representation learning on patent-graphs for patent classification. World Wide Web 24, 1791–1812 (2021)
Grootendorst, M.: Bertopic: neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794 (2022)
Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864 (2016)
Kim, G., Bae, J.: A novel approach to forecast promising technology through patent analysis. Technol. Forecast. Soc. Chang. 117, 228–237 (2017)
Kim, Y.G., Suh, J.H., Park, S.C.: Visualization of patent analysis for emerging technology. Expert Syst. Appl. 34(3), 1804–1812 (2008)
Li, D., Zamani, S., Zhang, J., Li, P.: Integration of knowledge graph embedding into topic modeling with hierarchical Dirichlet process. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pp. 940–950 (2019)
Madani, F., Weber, C.: The evolution of patent mining: applying bibliometrics analysis and keyword network analysis. World Patent Inf. 46, 32–48 (2016)
McInnes, L., Healy, J.: Accelerated hierarchical density based clustering. In: 2017 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 33–42 (2017)
McInnes, L., Healy, J., Melville, J.: UMAP: uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426 (2018)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Ramage, D., Hall, D., Nallapati, R., Manning, C.D.: Labeled LDA: a supervised topic model for credit attribution in multi-labeled corpora. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 248–256 (2009)
Reimers, N., Gurevych, I.: Sentence-bert: sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019)
Rose, S., Engel, D., Cramer, N., Cowley, W.: Automatic keyword extraction from individual documents. In: Text Mining: Applications and Theory, pp. 1–20 (2010)
Rosen-Zvi, M., Griffiths, T., Steyvers, M., Smyth, P.: The author-topic model for authors and documents. arXiv preprint arXiv:1207.4169 (2012)
Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)
Tenenbaum, J.B., Silva, V.D., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)
Tseng, Y.H., Lin, C.J., Lin, Y.I.: Text mining techniques for patent analysis. Inf. Process. Manag. 43(5), 1216–1247 (2007)
Wang, X., Zhang, Y., Wang, X., Chen, J.: A knowledge graph enhanced topic modeling approach for herb recommendation. In: Database Systems for Advanced Applications (DSAA), pp. 709–724 (2019)
Yan, S., Xu, D., Zhang, B., Zhang, H.J., Yang, Q., Lin, S.: Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Trans. Pattern Anal. Mach. Intell. 29(1), 40–51 (2006)
Yao, L., et al.: Incorporating knowledge graph embeddings into topic modeling. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), vol. 31, no. 1 (2017)
Yoon, B., Park, Y.: A text-mining-based patent network: analytical tool for high-technology trend. J. High Technol. Managem. Res. 15(1), 37–50 (2004)
Acknowledgement
This work was supported in part by JST CREST (JPMJCR22M2), Grants-in-Aid for Scientific Research (22H00508, 22K17944), and Kumagai Gumi Co., Ltd.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Nakamura, R., Ito, H., Morishima, A. (2025). Metadata-Assisted Topic Modeling for Patent Analysis. In: Oliver, G., Frings-Hessami, V., Du, J.T., Tezuka, T. (eds) Sustainability and Empowerment in the Context of Digital Libraries. ICADL 2024. Lecture Notes in Computer Science, vol 15494. Springer, Singapore. https://doi.org/10.1007/978-981-96-0868-3_1
Download citation
DOI: https://doi.org/10.1007/978-981-96-0868-3_1
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-96-0867-6
Online ISBN: 978-981-96-0868-3
eBook Packages: Computer ScienceComputer Science (R0)