Abstract
In scientific research, basic research that is both curiosity-driven and use-inspired is known as Pasteur’s Quadrant. The research on the impact and attention of Pasteur’s Quadrant is an essential research topic in academia. In view of the many milestone breakthroughs that Artificial Intelligence (AI) has brought to humanity, this paper delves into Pasteur’s Quadrant in AI through the citation of papers by patents. We empirically analyse the scientific impact of 3322 patent-cited papers and 6587 non-patent-cited papers published from 1999 to 2013, where scientific impact is measured by scientific citations and usage counts. Our main results show that patent-cited papers have a stronger scientific impact than non-patent-cited papers, and this impact is further enhanced in conference publications than in journal publications. Further, the relationship between the multidimensional characteristics of patent citations and scientific impact is investigated in terms of patent-cited papers. We find an inverted U-shaped relationship between the intensity of a paper’s patent citations and its scientific citations, as well as between the breadth of a paper’s patent citations and its scientific citations. In addition, the patent citation lag of a paper negatively relates to its scientific impact.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The definition of science and technology is complex and multifaceted; a definitive conclusion is lacking. According to Kuhn (1962), science is not a straightforward accumulation of knowledge but rather a dynamic process characterized by paradigm shifts and periodic revolutions. Arthur (2009) defines technology as a combination of three elements: a knowledge base, a hardware foundation, and social organization. He suggests that technology is not merely a collection of tools but a dynamic system that evolves through innovation and societal interaction.
The relationship between science and technology is complex and diverse (Stokes, 1997), and not all papers in Pasteur’s Quadrant are cited by patents. For example, the technological translation potential of highly innovative academic research at the forefront of science may not have been fully tapped, or some academic research may involve sensitive issues or confidential information that cannot be cited in patents.
The 4-digit code is used because it is more sensitive than the 3-digit code in determining the variety of patents, and more numeric codes can lead to meaningless dispersion (Hu & Rousseau, 2015).
The diachronous approach is commonly used to measure citation impact (Fukuzawa & Ida, 2016). This article adopts this method.
A classic thesis proposed by Merton (1968) is that scientific achievements exhibit a “Matthew effect”: Famous scientists are more likely to receive more resources and honours, which in turn strengthen their status and influence. The theory of the Matthew effect has been further studied and expanded by subsequent scholars (Bol et al., 2018; Drivas & Kremmydas, 2020; Larivière & Gingras, 2010; Wang, 2014).
The groundbreaking research of Spence (1973) on signaling theory examines how individuals in the job market effectively convey their qualities to prospective employers. The study demonstrates that education can serve as an investment by job applicants to signal their quality, effectively setting them apart from other candidates in the applicant pool.
The restriction on the publication year is for the following reasons. Firstly, when we limit the publication types and WoS categories, the number of papers selected in the period from 1999 onwards is relatively large, providing a more accurate and reliable estimate. Secondly, the time lag for the translation of scientific knowledge into technological applications may vary greatly across technology areas (Ahmadpoor & Jones, 2017; Huang et al., 2015). To retain a sufficient number of samples and ensure that papers are fully cited by patents, this study selects a patent citation time of at least eight years; that is, papers in 2013 can be cited by patents in 2013-2020.
According to Montesi and Owen (2008), the term “extended versions of conference papers” refers to papers originally presented at conferences, workshops, seminars, or the like, which are subsequently modified for publication in academic journals.
The method for calculating the average citation frequency is to first determine a specific publication source (such as a journal, conference, etc.), then count the number of citations for all papers on that source, and finally divide the total number of citations by the total number of papers to obtain the average citation frequency.
There are some extreme values in the data. For example, in the full sample, the 99th percentile values for citation count and usage count of papers are 1579 and 220, respectively. There are 99 papers with citation count exceeding 1579, out of which 7 have more than 10,000 citations. Similarly, there are 99 papers with usage count surpassing 220, out of which 12 have more than 1000 uses.
Specifically, we used the Winsorize method to process the data, replacing values less than the 1st percentile and greater than the 99th percentile with the 1st percentile and 99th percentile, respectively.
Negative binomial models and Poisson models are commonly used for count data analysis. The main assumption of the Poisson model is that the mean equals the variance. However, in the full sample and subsample, the coefficient of variance indicates that the standard deviation is nearly double the mean. In addition, in subsequent empirical tests, alpha is the estimate of the dispersion parameter, and Lnalpha is the calculated result of taking the logarithm of alpha. Significant Lnalpha statistics indicate a problem of overdispersion of the explained variables, and therefore the negative binomial model fits our estimation better than the Poisson model (Ning & Guo, 2022).
References
Ahmadpoor, M., & Jones, B. F. (2017). The dual frontier: Patented inventions and prior scientific advance. Science, 357(6351), 583–587. https://doi.org/10.1126/science.aam9527
Angori, G., Marzocchi, C., Ramaciotti, L., & Rizzo, U. (2023). A patent-based analysis of the evolution of basic, mission-oriented, and applied research in European universities. Journal of Technology Transfer. https://doi.org/10.1007/s10961-023-10001-5
Archibugi, D. (1992). Patenting as an indicator of technological innovation: A review. Science and Public Policy, 19(6), 357–368. https://doi.org/10.1093/spp/19.6.357
Arthur, W. B. (2009). The nature of technology: What it is and how it evolves. Free Press.
Bol, T., de Vaan, M., & van de Rijt, A. (2018). The Matthew effect in science funding. Proceedings of the National Academy of Sciences of the United States of America, 115(19), 4887–4890. https://doi.org/10.1073/pnas.1719557115
Borner, K., Rouse, W. B., Trunfio, P., & Stanley, H. E. (2018). Forecasting innovations in science, technology, and education. Proceedings of the National Academy of Sciences of the United States of America, 115(50), 12573–12581. https://doi.org/10.1073/pnas.1818750115
Bornmann, L., & Daniel, H. D. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation, 64(1), 45–80. https://doi.org/10.1108/00220410810844150
Bornmann, L., Schier, H., Marx, W., & Daniel, H. D. (2012). What factors determine citation counts of publications in chemistry besides their quality? Journal of Informetrics, 6(1), 11–18. https://doi.org/10.1016/j.joi.2011.08.004
Bush, V. (1945). Science - The endless frontier. U.S. Government Printing Office, Washington, DC.
Callaert, J., Van Looy, B., Verbeek, A., Debackere, K., & Thijs, B. (2006). Traces of Prior Art: An analysis of non-patent references found in patent documents. Scientometrics, 69(1), 3–20. https://doi.org/10.1007/s11192-006-0135-8
Chakraborty, T. (2018). Role of interdisciplinarity in computer sciences: Quantification, impact and life trajectory. Scientometrics, 114(3), 1011–1029. https://doi.org/10.1007/s11192-017-2628-z
Chen, C. M. (2012). Predictive effects of structural variation on citation counts. Journal of the American Society for Information Science and Technology, 63(3), 431–449. https://doi.org/10.1002/asi.21694
Collins, P., & Wyatt, S. (1988). Citations in patents to the basic research literature. Research Policy, 17(2), 65–74. https://doi.org/10.1016/0048-7333(88)90022-4
Cozzens, S. E. (1989). What do citations count? The Rhetoric-First Model. Scientometrics, 15(5), 437–447. https://doi.org/10.1007/BF02017064
Didegah, F., & Thelwall, M. (2013). Determinants of research citation impact in nanoscience and nanotechnology. Journal of the American Society for Information Science and Technology, 64(5), 1055–1064. https://doi.org/10.1002/asi.22806
Ding, C. G., Hung, W. C., Lee, M. C., & Wang, H. J. (2017). Exploring paper characteristics that facilitate the knowledge flow from science to technology. Journal of Informetrics, 11(1), 244–256. https://doi.org/10.1016/j.joi.2016.12.004
Donner, P., & Schmoch, U. (2020). The implicit preference of bibliometrics for basic research. Scientometrics, 124(2), 1411–1419. https://doi.org/10.1007/s11192-020-03516-3
Drivas, K., & Kremmydas, D. (2020). The Matthew effect of a journal’s ranking. Research Policy, 49(4), 103951. https://doi.org/10.1016/j.respol.2020.103951
Du, J., Li, P. X., Haunschild, R., Sun, Y. N., & Tang, X. L. (2020). Patent citation linkages as early signs for predicting delayed recognized knowledge: Macro and micro evidence. Journal of Informetrics, 14(2), 101017. https://doi.org/10.1016/j.joi.2020.101017
Duy, J., & Vaughan, L. (2006). Can electronic journal usage data replace citation data as a measure of journal use? An empirical examination. Journal of Academic Librarianship, 32(5), 512–517. https://doi.org/10.1016/j.acalib.2006.05.005
Eckmann, M., Rocha, A., & Wainer, J. (2012). Relationship between high-quality journals and conferences in computer vision. Scientometrics, 90(2), 617–630. https://doi.org/10.1007/s11192-011-0527-2
Finardi, U. (2011). Time relations between scientific production and patenting of knowledge: The case of nanotechnologies. Scientometrics, 89(1), 37–50. https://doi.org/10.1007/s11192-011-0443-5
Fortnow, L. (2009). Time for computer science to grow up. Communications of the ACM, 52(8), 33–35. https://doi.org/10.1145/1536616.1536631
Frank, M. R., Wang, D. S., Cebrian, M., & Rahwan, I. (2019). The evolution of citation graphs in artificial intelligence research. Nature Machine Intelligence, 1(2), 79–85. https://doi.org/10.1038/s42256-019-0024-5
Freyne, J., Coyle, L., Smyth, B., & Cunningham, P. (2010). Relative status of journal and conference publications in computer science. Communications of the ACM, 53(11), 124–132. https://doi.org/10.1145/1839676.1839701
Fukuzawa, N., & Ida, T. (2016). Science linkages between scientific articles and patents for leading scientists in the life and medical sciences field: The case of Japan. Scientometrics, 106(2), 629–644. https://doi.org/10.1007/s11192-015-1795-z
Gazni, A. (2020). The growing number of patent citations to scientific papers: Changes in the world, nations, and fields. Technology in Society, 62, 101276. https://doi.org/10.1016/j.techsoc.2020.101276
Gazni, A., & Ghaseminik, Z. (2019). The increasing dominance of science in the economy: Which nations are successful? Scientometrics, 120(3), 1411–1426. https://doi.org/10.1007/s11192-019-03161-5
Geuna, A., & Nesta, L. J. J. (2006). University patenting and its effects on academic research: The emerging European evidence. Research Policy, 35(6), 790–807. https://doi.org/10.1016/j.respol.2006.04.005
Glänzel, W., & Zhou, P. (2011). Publication activity, citation impact and bi-directional links between publications and patents in biotechnology. Scientometrics, 86(2), 505–525. https://doi.org/10.1007/s11192-010-0269-6
Gu, T. T., Zhang, S. F., & Cai, R. R. (2022). Can Artificial intelligence boost employment in service industries? Empirical analysis based on China. Applied Artificial Intelligence, 36(1), 2080336. https://doi.org/10.1080/08839514.2022.2080336
Haans, R. F. J., Pieters, C., & He, Z. L. (2016). Thinking about U: Theorizing and testing U-and inverted U-shaped relationships in strategy research. Strategic Management Journal, 37(7), 1177–1195. https://doi.org/10.1002/smj.2399
Harzing, A. W., & van der Wal, R. (2009). A google scholar h-index for journals: An Alternative metric to measure journal impact in economics and business. Journal of the American Society for Information Science and Technology, 60(1), 41–46. https://doi.org/10.1002/asi.20953
Hicks, D., Breitzman, A., Sr., Hamilton, K., & Narin, F. (2000). Research excellence and patented innovation. Science and Public Policy, 27(5), 310–320. https://doi.org/10.3152/147154300781781805
Hu, X. J., & Rousseau, R. (2015). A simple approach to describe a company’s innovative activities and their technological breadth. Scientometrics, 102(2), 1401–1411. https://doi.org/10.1007/s11192-014-1464-7
Huang, M. H., Yang, H. W., & Chen, D. Z. (2015). Increasing science and technology linkage in fuel cells: A cross citation analysis of papers and patents. Journal of Informetrics, 9(2), 237–249. https://doi.org/10.1016/j.joi.2015.02.001
Jefferson, O. A., Jaffe, A., Ashton, D., Warren, B., Koellhofer, D., Dulleck, U., & Ward, K. (2018). Mapping the global influence of published research on industry and innovation. Nature Biotechnology, 36(1), 31–39. https://doi.org/10.1038/nbt.4049
Jin, T., Duan, H. Q., Lu, X. F., Ni, J., & Guo, K. (2021). Do research articles with more readable abstracts receive higher online attention? Evidence from Science. Scientometrics, 126(10), 8471–8490. https://doi.org/10.1007/s11192-021-04112-9
Ke, Q. (2018). Comparing scientific and technological impact of biomedical research. Journal of Informetrics, 12(3), 706–717. https://doi.org/10.1016/j.joi.2018.06.010
Ke, Q. (2020). Technological impact of biomedical research: The role of basicness and novelty. Research Policy, 49(7), 104071. https://doi.org/10.1016/j.respol.2020.104071
Kuhn, T. S. (1962). The structure of scientifi revolutions. University of Chicago Press.
Larivière, V., & Gingras, Y. (2010). The impact factor’s Matthew effect: A natural experiment in bibliometrics. Journal of the American Society for Information Science and Technology, 61(2), 424–427. https://doi.org/10.1002/asi.21232
Lind, J. T., & Mehlum, H. (2010). With or without U? The appropriate test for a U-shaped relationship. Oxford Bulletin of Economics and Statistics, 72(1), 109–118. https://doi.org/10.1111/j.1468-0084.2009.00569.x
Lu, X. F. (2012). The relationship of lexical richness to the quality of ESL learners’ oral narratives. The Modern Language Journal, 96(2), 190–208. https://doi.org/10.1111/j.1540-4781.2011.01232_1.x
Mammola, S., Piano, E., Doretto, A., Caprio, E., & Chamberlain, D. (2022). Measuring the influence of non-scientific features on citations. Scientometrics, 127(7), 4123–4137. https://doi.org/10.1007/s11192-022-04421-7
Manjunath, A., Li, H. Y., Song, S. C., Zhang, Z. X., Liu, S., Kahrobai, N., & Kumar, I. (2021). Comprehensive analysis of 24 million patent-to-research citations maps the biomedical innovation and translation landscape. Nature Biotechnology, 39(6), 678–683. https://doi.org/10.1038/s41587-021-00940-5
Merton, R. K. (1968). The Matthew effect in science: The reward and communication systems of science are considered. Science, 159(3810), 56–63. https://doi.org/10.1126/science.159.3810.56
Meyer, M. (2002). Tracing knowledge flows in innovation systems. Scientometrics, 54(2), 193–212. https://doi.org/10.1023/A:1016057727209
Meyer, M., Debackere, K., & Glänzel, W. (2010). Can applied science be ‘good science’? Exploring the relationship between patent citations and citation impact in nanoscience. Scientometrics, 85(2), 527–539. https://doi.org/10.1007/s11192-009-0154-3
Mingers, J., Macri, F., & Petrovici, D. (2012). Using the h-index to measure the quality of journals in the field of business and management. Information Processing & Management, 48(2), 234–241. https://doi.org/10.1016/j.ipm.2011.03.009
Montesi, M., & Owen, J. M. (2008). From conference to journal publication: How conference papers in software engineering are extended for publication in journals. Journal of the American Society for Information Science and Technology, 59(5), 816–829. https://doi.org/10.1002/asi.20805
Nelson, R. R. (1993). National innovation systems: a comparative analysis. Oxford University Press.
Ning, L., & Guo, R. (2022). Technological diversification to green domains: Technological relatedness, invention impact and knowledge integration capabilities. Research Policy, 51(1), 104406. https://doi.org/10.1016/j.respol.2021.104406
Onodera, N., & Yoshikane, F. (2015). Factors affecting citation rates of research articles. Journal of the Association for Information Science and Technology, 66(4), 739–764. https://doi.org/10.1002/asi.23209
Papazoglou, M. E., & Spanos, Y. E. (2018). Bridging distant technological domains: A longitudinal study of the determinants of breadth of innovation diffusion. Research Policy, 47(9), 1713–1728. https://doi.org/10.1016/j.respol.2018.06.006
Park, H. W., & Kang, J. (2009). Patterns of scientific and technological knowledge flows based on scientific papers and patents. Scientometrics, 81(3), 811–820. https://doi.org/10.1007/s11192-008-2224-3
Park, M. S., Park, J., Kim, H., Lee, J. H., & Park, H. (2023). Measuring the impacts of quantity and trustworthiness of information on COVID-19 vaccination intent. Journal of the Association for Information Science and Technology, 74(7), 846–865. https://doi.org/10.1002/asi.24760
Poege, F., Harhoff, D., Gaessler, F., & Baruffaldi, S. (2019). Science quality and the value of inventions. Science Advances, 5(12), eaay7323.
Qi, Y., Zhu, N., Zhai, Y., & Ding, Y. (2018). The mutually beneficial relationship of patents and scientific literature: Topic evolution in nanoscience. Scientometrics, 115(2), 893–911. https://doi.org/10.1007/s11192-018-2693-y
Qian, Y. F., Rong, W. G., Nan, J., Tang, J., & Xiong, Z. (2017). Citation regression analysis of computer science publications in different ranking categories and subfields. Scientometrics, 110(3), 1351–1374. https://doi.org/10.1007/s11192-016-2235-4
Rosenberg, N., & Nelson, R. R. (1994). American universities and technical advance in industry. Research Policy, 23(3), 323–348. https://doi.org/10.1016/0048-7333(94)90042-6
Salimi, N., Bekkers, R., & Frenken, K. (2015). Does working with industry come at a price? A study of doctoral candidates’ performance in collaborative vs. non-collaborative Ph.D. projects. Technovation, 41–42, 51–61. https://doi.org/10.1016/j.technovation.2015.03.007
Sanberg, P. R., Gharib, M., Harker, P. T., Kaler, E. W., Marchase, R. B., Sands, T. D., & Sarkar, S. (2014). Changing the academic culture: Valuing patents and commercialization toward tenure and career advancement. Proceedings of the National Academy of Sciences of the United States of America, 111(18), 6542–6547. https://doi.org/10.1073/pnas.1404094111
Schmoch, U., Beckert, B., & Schaper-Rinkel, P. (2019). Impact assessment of a support programme of science-based emerging technologies. Scientometrics, 118(3), 1141–1161. https://doi.org/10.1007/s11192-018-03002-x
Shaw, J. (2022). There and back again: Revisiting Vannevar Bush, the linear model, and the freedom of science. Research Policy, 51(10), 104610. https://doi.org/10.1016/j.respol.2022.104610
Shneiderman, B. (2018). Twin-Win Model: A human-centered approach to research success. Proceedings of the National Academy of Sciences of the United States of America, 115(50), 12590–12594. https://doi.org/10.1073/pnas.1802918115
Spence, M. (1973). Job Market Signaling. The Quarterly Journal of Economics, 87(3), 355–374. https://doi.org/10.2307/1882010
Stokes, D. (1997). Pasteur’s quadrant: Basic science and technological innovation. Brookings Institution Press.
Tahamtan, I., Afshar, A. S., & Ahamdzadeh, K. (2016). Factors affecting number of citations: A comprehensive review of the literature. Scientometrics, 107(3), 1195–1225. https://doi.org/10.1007/s11192-016-1889-2
Tijssen, R. J. W. (2018). Anatomy of use-inspired researchers: From Pasteur’s quadrant to Pasteur’s cube model. Research Policy, 47(9), 1626–1638. https://doi.org/10.1016/j.respol.2018.05.010
Tijssen, R. J. W., Buter, R. K., & van Leeuwen, T. N. (2000). Technological relevance of science: An assessment of citation linkages between patents and research papers. Scientometrics, 47(2), 389–412. https://doi.org/10.1023/A:1005603513439
Toole, A., Pairolero, N., Giczy, A., Forman, J., Pulliam, C., Such, M., Frumkin, J. (2020). Inventing AI: Tracing the diffusion of artificial intelligence with US patents. In: October). Alexandria, VA: United States Patent and Trademark Office.
van Raan, A. F. J. (2005). Measuring Science. In H. F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of Quantitative Science and Technology Research: The Use of Publication and Patent Statistics in Studies of S&T Systems (pp. 19–50). Springer. https://doi.org/10.1007/1-4020-2755-9_2
van Raan, A. F. J. (2017). Sleeping beauties cited in patents: Is there also a dormitory of inventions? Scientometrics, 110(3), 1123–1156. https://doi.org/10.1007/s11192-016-2215-8
van Raan, A. F. J., & Winnink, J. J. (2018). Do younger sleeping beauties prefer a technological prince? Scientometrics, 114(2), 701–717. https://doi.org/10.1007/s11192-017-2603-8
van Looy, B., Ranga, M., Callaert, J., Debackere, K., & Zimmermann, E. (2004). Combining entrepreneurial and scientific performance in academia: Towards a compounded and reciprocal Matthew-effect? Research Policy, 33(3), 425–441. https://doi.org/10.1016/j.respol.2003.09.004
van Looy, B., Callaert, J., & Debackere, K. (2006). Publication and patent behavior of academic researchers: Conflicting, reinforcing or merely co-existing? Research Policy, 35(4), 596–608. https://doi.org/10.1016/j.respol.2006.02.003
Veugelers, R., & Wang, J. (2019). Scientific novelty and technological impact. Research Policy, 48(6), 1362–1372. https://doi.org/10.1016/j.respol.2019.01.019
Vrettas, G., & Sanderson, M. (2015). Conferences versus journals in computer science: Conferences vs Journals in Computer Science. Journal of the Association for Information Science and Technology, 55(9), 32–842. https://doi.org/10.1002/asi.23349
Wang, J. (2014). Unpacking the Matthew effect in citations. Journal of Informetrics, 8(2), 329–339. https://doi.org/10.1016/j.joi.2014.01.006
Wang, Y. Z., Xie, Y. D., Wang, D., Guo, L., & Zhou, R. T. (2022). Do cover papers get better citations and usage counts? An analysis of 42 journals in cell biology. Scientometrics, 127(7), 3793–3813. https://doi.org/10.1007/s11192-022-04444-0
Zhang, G. J., Feng, Y. Q., Yu, G., Liu, L. N., & Hao, Y. Q. Q. (2017). Analyzing the time delay between scientific research and technology patents based on the citation distribution model. Scientometrics, 111(3), 1287–1306. https://doi.org/10.1007/s11192-017-2357-3
Zhang, X., Wang, X. H., Zhao, H. K., Pablos, P., Sun, Y. Q., & Xiong, H. (2019). An effectiveness analysis of altmetrics indices for different levels of artificial intelligence publications. Scientometrics, 119(3), 1311–1344. https://doi.org/10.1007/s11192-019-03088-x
Zong, Q. J., Xie, Y. F., Tuo, R. C., Huang, J. S., & Yang, Y. (2019). The impact of video abstract on citation counts: Evidence from a retrospective cohort study of New Journal of Physics. Scientometrics, 119(3), 1715–1727. https://doi.org/10.1007/s11192-019-03108-w
Acknowledgements
This research was supported by the National Natural Science Foundation of China (Grant No. 71874173 and 72374188) and the Featured Social Science Fund of the University of Science and Technology of China (FSSF-A-230204). We would like to thank the editor and anonymous reviewers for their constructive comments and suggestions, which helped us to improve the paper.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflicts of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Gao, X., Wu, Q., Liu, Y. et al. Pasteur’s quadrant in AI: do patent-cited papers have higher scientific impact?. Scientometrics 129, 909–932 (2024). https://doi.org/10.1007/s11192-023-04925-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-023-04925-w