Abstract
Bibliometric studies often measure and compare scholarly performance, but they rarely investigate why universities, departments, and research groups do have different performance. In this paper we try to explain differences in scholarly performance of research groups in terms of organizational variables. In order to do this, we extensively review the relevant literature, and develop a model using two theoretical approaches. A multivariate analysis shows which of the independent variables do play a role in the various scholarly performance dimensions. The study shows what organizational strategies may help in optimizing performance in various dimensions. Implications are discussed.

Similar content being viewed by others
Notes
In this paper we restrict ourselves to the management and leadership factors that influence scholarly performance of research teams. We do not discuss other dimensions of performance, such as teaching performance, or societal outcomes. The general term ‘performance’ is used synonym for ‘scholarly performance’.
Non-response analysis shows that the respondents can be regarded as a representative sample of the Dutch biomedical and health research groups. The respondents were evenly distributed among the various research institutions and the sub-disciplines. Performance levels between respondents and non-respondents did not significantly differ.
Conference visit, stay abroad, courses. Cronbach’s Alpha = 0.65.
Meetings to discuss projects ideas; research proposals; new literature; draft papers) Cronbach’s Alpha = 0.7.
Relating to different roles in the team: intensively participating in research, source of knowledge, generating new ideas, more researcher than manager, etc. Cronbach’s Alpha = 0.8.
Up to date with the developments in the field; up to date with literature. Cronbach’s Alpha = 0.76.
For productivity and quality, the scores were multiplied by 100 to get count data (integers).
References
Adams, J. D., Black, G. C., Clemmons, J. R., & Stephan, P. E. (2005). Scientific teams and institutional collaborations: Evidence from US universities, 1981–1999. Research Policy, 34(3), 259–285.
Allen, T. J., & Sloan, A. P. (1970). Communication networks in R and D laboratories. R & D Management, 1(1), 14–21.
Allison, P. D. (1980). Inequality and scientific productivity. Social Studies of Science, 10, 163–179.
Allison, P. D., & Long, J. S. (1990). Departmental effects on scientific productivity. American Sociological Review, 55(4), 469–478.
Amabile, T. M., Schatzel, E. A., Moneta, G. B., & Kramer, S. J. (2004). Leader behaviors and the work environment for creativity: Perceived leader support. Leadership Quarterly, 15(1), 5–32.
Andrews, F. M. (1979). Motivation, diversity, and the performance of research units. In F. M. Andrews (Ed.), Scientific productivity The effectiveness of research groups in six countries. Cambridge: Cambridge University Press.
Andrews, F. M., & Farris, G. F. (1967). Supervisory practices and innovation in scientific teams. Personnel Psychology, 20(4), 497–515.
Auranen, O., & Nieminen, M. (2010). University research funding and publication performance—An international comparison. Research Policy, 39, 822–834.
Babu, A. R., & Sing, Y. P. (1998). Determinants of research productivity. Scientometrics, 43(3), 309–329.
Baird, L. L. (1986). What characterizes a productive research department? Research in Higher Education, 25(3), 211–225.
Blackburn, R. T., Behymer, C. E., & Hall, D. E. (1978). Research note - Correlates of faculty publications. Sociology of Education, 51(2), 132–141.
Bland, C. J., & Ruffin, M. T. (1992). Characteristics of a productive research environment: Literature review. Academic Medicine, 67(6), 385–397.
Bonaccorsi, A., & Daraio, C. (2005). Exploring size and agglomeration effects on public research productivity. Scientometrics, 63(1), 87–120.
Bornmann, L., Leydesdorff, L., & Van den Besselaar, P. (2010). A meta-evaluation of scientific research proposals: Different ways of comparing rejected to awarded applications. Journal of Informetrics, 4(3), 211–220.
Bornmann, L., Mutz, R., Neuhaus, C., & Daniel, H. (2008). Citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics in science and environmental politics, 8, 93–102.
Bozeman, B., & Corley, E. (2004). Scientists’ collaboration strategies: implications for scientific and technical human capital. Research Policy, 33(4), 599–616.
Bozeman B, & Boardman C. (2014). Research collaboration and team science: a state of the art review and agenda. Springer.
Braam, R., & Van den Besselaar, P. (2010). Life cycles of research groups: the case of CWTS. Research Evaluation, 19, 173–184.
Braam R. & Van den Besselaar P, (2014). Indicators for the dynamics of research organizations: a biomedical case study. Scientometrics, 99(3), 949–971.
Cameron, A. C., & Trivedi, P. K. (1998). Regression analysis of count data. Cambridge: Cambridge University Press.
Carayol, N. (2003). Objectives, agreements and matching in science-industry collaborations: Reassembling the pieces of the puzzle. Research Policy, 32(6), 887–908.
Carayol, N., & Matt, M. (2004). Does research organization influence academic production? Laboratory level evidence from a large European university. Research Policy, 33(8), 1081–1102.
Carayol, N., & Matt, M. (2006). Individual and collective determinants of academic scientists’ productivity. Information Economics and Policy, 18(1), 55–72.
Cardinal, L. B. (2001). Technological innovation in the pharmaceutical industry: the use of organizational control in managing research and development. Organization Science, 12(1), 19–36.
Cherchye, L., & Abeele, P. V. (2005). On research efficiency—A micro-analysis of dutch university research in economics and business management. Research Policy, 34(4), 495–516.
Crane, D. (1965). Scientists at major and minor universities: A study of productivity and recognition. American Sociological Review, 30(5), 699–714.
Crane, D. (1972). Invisible colleges: Diffusion of knowledge in scientific communities. Chicago: University of Chicago Press.
Cummings, J. N., & Cross, R. (2003). Structural properties of work groups and their consequences for performance. Social Networks, 25(3), 197–210.
De Jong, S. P. L., Van Arensbergen, P., Daemen, F., Van der Meulen, B., & Van den Besselaar, P. (2011). Evaluation of research in context: an approach and two cases. Research Evaluation, 20(2), 61–72.
D’Este, P., & Perkmann, M. (2011). Why do academics engage with industry? The entrepreneurial university and individual motivations. The Journal of Technology Transfer, 36, 316–339.
Dietz, J. S., & Bozeman, B. (2005). Academic careers, patents, and productivity: Industry experience as scientific and technical human capital. Research Policy, 34(3), 349–367.
Dill, D. D. (1982). The management of academic culture—Notes on the management of meaning and social integration. Higher Education, 11(3), 303–320.
Dillman, D.A., Smyth J.D. & Christian L.M. (2009) Internet, mail, and mixed-model surveys: the tailored design method. Hoboken: Wiley.
Dundar, H., & Lewis, D. R. (1998). Determinants of research productivity in higher education. Research in Higher Education, 39(6), 607–631.
Etzkowitz, H. (1998). The norms of entrepreneurial science: cognitive effects of the new university-industry linkages. Research Policy, 27(8), 823–833.
Falk-Krzesinski, H. F., Contractor, N., Fiore, S. M., Hall, K. L., Kane, C., Keyton, J., et al. (2011). Mapping a research agenda for the science of team science. Research Evaluation, 20(2), 145–158.
Fox, M. F. (1992). Researh, teaching, and publication productivity: Mutuality versus competition in academia. Sociology of Education, 65(4), 293–305.
Fox, M. F., & Mohapatra, S. (2007). Social-organizational characteristics of work and publication productivity among academic scientists in doctoral-granting departments. Journal of Higher Education, 78(5), 542–571.
Frederiksen, L. F., Hemlin, S., & Husted, K. (2004). The role of knowledge management in R&D: a survey of Danish R&D leaders’ perceptions and beliefs. International Journal of Technology Management, 28(7–8), 820–839.
Gardner, W., Mulvey, E. P., & Shaw, E. C. (1995). Regression analyses of counts and rates: poisson, overdispersed poisson, and negative binomial models. Psychological Bulletin, 118(3), 392–404.
Geuna, A. (2001). The changing rationale for European university research funding: Are there negative unintended consequences? Journal of Economic Issues, 35(3), 607–632.
Gladstein, D. L. (1984). Groups in context: A model of task group effectiveness. Administrative Science Quarterly, 29(4), 499–517.
Gonzalez-Brambila, C., & Veloso, F. M. (2007). The determinants of research output and impact: A study of Mexican researchers. Research Policy, 36, 1035–1051.
Goodall, A. H. (2009). Highly cited leaders and the performance of research universities. Research Policy, 38(7), 1079–1092.
Gornitzka, A. (1999). Governmental policies and organisational change in higher education. Higher Education, 38, 5–31.
Gottlieb, E. E., & Keith, B. (1997). The academic research-teaching nexus in eight advanced-industrialized countries. Higher Education, 34(3), 397–419.
Groot, T., & García-Valderrama, T. (2006). Research quality and efficiency. An analysis of assessments and management issues in Dutch economics and business research programs. Research Policy, 35, 1362–1376.
Gulbrandsen, M., & Smeby, J. C. (2005). Industry funding and university professors’ research performance. Research Policy, 34(6), 932–950.
Gustad, J. W. (1960). The career decisions of college teachers. Washington, DC: Department of Health, Education and Welfare.
Hackett, E. J. (2005). Essential tensions: Identity, control, and risk in research. Social Studies of Science, 35(5), 787–826.
Haeussler, C., & Colyvas, J. A. (2011). Breaking the ivory tower: Academic entrepreneurship in the life sciences in UK and Germany. Research Policy, 40, 41–54.
Haraszthy, A., & Szántó, L. (1979). Some problems of research planning: data from Hungary compared to other Round 1 countries. In F. M. Andrews (Ed.), Scientific productivity. The effectiveness of research groups in six countries. Cambridge: Cambridge University Press.
Harris, G., & Kaine, G. (1994). The determinants of research performance—A study of australian university economists. Higher Education, 27(2), 191–201.
Harvey, J., Pettigrew, A., & Ferlie, E. (2002). The determinants of research group performance: towards mode 2? Journal of Management Studies, 39(6), 746–774.
He, Z. L., Geng, X. S., & Campbell-Hunt, C. (2009). Research collaboration and research output: A longitudinal study of 65 biomedical scientists in a New Zealand university. Research Policy, 38(2), 306–317.
Heinze, T., Shapira, P., Rogers, J. D., & Senker, J. M. (2009). Organizational and institutional influences on creativity in scientific research. Research Policy, 38, 610–623.
Hornbostel, S., Böhmer, S., Klingsporn, B., Neufeld, J., & Von Ins, M. (2009). Funding of young scientist and scientific excellence. Scientometrics, 79(1), 171–190.
Jansen, D., Wald, A., Franke, K., Schmoch, U., & Schubert, T. (2007). Third party research funding and performance in research. On the effects of institutional conditions on research performance of teams. Kolner Zeitschrift fur Soziologie und Sozialpsychologie, 59(1), pp. 125-149.
Johnes, G. (1988). Determinants of research output in economics departments in British-universities. Research Policy, 17(3), 171–178.
Jordan, J. M., Meador, M., & Walters, S. J. K. (1988). Effects of department size and organization ont the research productivity of academic economists. Economics of Education Review, 7(2), 251–255.
Jordan, J. M., Meador, M., & Walters, S. J. K. (1989). Academic research productivity, department size and organization: Further results. Economics of Education Review, 8(4), 345–352.
Katz, R. (1978a). Influence of job longevity on employee reactions to task characteristics. Human Relations, 31(8), 703–725.
Katz, R. (1978b). Job longevity as a situational factor in job satisfaction. Administrative Science Quarterly, 23(2), 204–223.
Katz, J. S. & Martin, B. R. (1997). What is research collaboration? Research Policy, 26(1), 1–18.
Katz, R., & Tushman, M. L. (1979). Communication patterns, project performance, and task characteristics—Empirical-evaluation and integration in an R and D setting. Organizational Behavior and Human Performance, 23(2), 139–162.
Keith, B., & Babchuk, N. (1998). The quest for institutional recognition: a longitudinal analysis of scholarly productivity and academic prestige among sociology departments. Social Forces, 76(4), 1495–1533.
King, J. (1987). A review of bibliometric and other science indicators and their role in research evaluation. Journal of Information Science, 13, 261–276.
Knorr, K. D., & Mittermeir, R. (1980). Publication productivity and professional position: Cross-national evidence on the role of organizations. Scientometrics, 2(2), 95–120.
Kretschmer, H. (1985). Cooperation structure, group-size and productivity in research groups. Scientometrics, 7(1–2), 39–53.
Kyvik, S. (1995). Are big university departments better than small ones. Higher Education, 30(3), 295–304.
Lam, A. (2011). What motivates academic scientists to engage in research commercialization: ‘Gold’, ‘ribbon’ or ‘puzzle’? Research Policy, 40(10), 1354–1368.
Laredo, P., & Mustar, P. (2000). Laboratory activity profiles: An exploratory approach. Scientometrics, 47(3), 515–539.
Leisyte, L., Enders, J., & De Boer, H. (2008). The freedom to set research agendas—illusion and reality of the research units in the Dutch universities. Higher Education Policy, 21, 377–391.
Lepori, B., Van den Besselaar, P., Dinges, M., Potì, B., Reale, E., Slipersaeter, S., et al (2007). Indicators for comparative analysis of public project funding: concepts, implementation and evaluation. Research Evaluation, 16(4), 243-256.
Levin, S. G., & Stephan, P. E. (1991). Research productivity over the life-cycle: Evidence for academic scientists. American Economic Review, 81(1), 114–132.
Long, J. S. (1978). Productivity and academic position in scientific career. American Sociological Review, 43(6), 889–908.
Long, J. S., Allison, P. D., & McGinnis, R. (1979). Entrance into the academic career. American Sociological Review, 44(5), 816–830.
Long, J. S., & McGinnis, R. (1981). Organizational context and scientific productivity. American Sociological Review, 46(4), 422–442.
Louis, K. S., Holdsworth, J. M., Anderson, M. S., & Campbell, E. G. (2007). Becoming a scientist: The effects of work-group size and organizational climate. Journal of Higher Education, 78(3), 311–336.
Manjarrés-Henríquez, L., Gutiérrez-Gracia, A., & Vega-Jurado, J. (2008). Coexistence of university-industry relations and academic research: Barrier to or incentive for scientific productivity. Scientometrics, 76(3), 561–576.
Martin, B. R. (1996). The use of multiple indicators in the assessment of basic research. Scientometrics, 36(3), 343–362.
Martin, B. R., & Irvine, J. (1983). Assessing basic research. Some partial indicators of scientific progress in radio astronomy. Research Policy, 12, 61–90.
Martin-Sempere, M. J., Garzon-Garcia, B., & Rey-Rocha, J. (2008). Team consolidation, social integration and scientists’ research performance: An empirical study in the Biology and Biomedicine field. Scientometrics, 76(3), 457–482.
McKeachie, W. (1979). Perspectives from psychology. In D. Lewis & W. Becker (Eds.), Academic rewards in higher education. Massachusetts: Ballinger.
Mehra, A., Dixon, A. L., Brass, D. J., & Robertson, B. (2006). The social network ties of group leaders: Implications for group performance and leader reputation. Organization Science, 17(1), 64–79.
Melin, G. (2000). Pragmatism and self-organization. Research collaboration on the individual level. Research Policy, 29, 31–40.
Merton, R. K. (1988). The Matthew effect in science, II Cumulative advantage and the symbolism of intellectual property. ISIS, 79, 606–623.
Mets, B., & Galford, J. A. (2009). Leadership and management of academic anesthesiology departments in the United States. Journal of Clinical Anesthesia, 21(2), 83–93.
Moed, H. F. (2000). Bibliometric indicators reflect publication and management strategies. Scientometrics, 47(2), 323–346.
Mumford, M. D., Scott, G. M., Gaddis, B., & Strange, J. M. (2002). Leading creative people: Orchestrating expertise and relationships. Leadership Quarterly, 13(6), 705–750.
Neuhaus, C., & Daniel, H. D. (2009). A new reference standard for citation analysis in chemistry and related fields based on the sections of Chemical Abstracts. Scientometrics, 78(2), 219–229.
Oh, H. S., Chung, M. H., & Labianca, G. (2004). Group social capital and group effectiveness: The role of informal socializing ties. Academy of Management Journal, 47(6), 860–875.
Oh, H. S., Labianca, G., & Chung, M. H. (2006). A multilevel model of group social capital. Academy of Management Review, 31(3), 569–582.
Omta, S. W. F. (1995). Critical success factors in biomedical research and pharmaceutical innovation. Groningen: Rijks Universiteit Groningen.
Omta, S. W. F., & De Leeuw, A. C. J. (1997). Management control, ucertainty, and performance in biomedical research in universities, institutes and companies. Journal of Engineering and Technology Management, 14, 223–257.
Oshagbemi, T. (2004). Age influences on the leadership styles and behaviour of managers. Employee Relations, 26(1), 14–29.
Pelz, D. C. (1956). Some social factors related to performance in a research organization. Administrative Science Quarterly, 1(3), 310–325.
Pelz, D. C., & Andrews, F. M. (1966). Scientists in organizations. Productive Climates for Research and Development. New York–London–Sydney: Wiley.
Perkoff, G. T. (1985). The research environment in family-practice. Journal of Family Practice, 21(5), 389–393.
Pfeffer, J., & Salancik, G. R. (1978). The external control of organizations: A resource dependence perspective. New York: Harper & Row.
Pineau, C., & Levy-Leboyer, C. (1983). Managerial and organizational determinants of efficiency in biomedical research teams. In S. R. Epton, R. L. Payne, & A. W. Pearson (Eds.), Managing interdisciplinary research (pp. 141–163). New York: Wiley.
Porter, S. R., & Umbach, P. D. (2001). Analyzing faculty workload data using multilevel modeling. Research in Higher Education, 42(2), 171–196.
Prins, A. A. M. (1990). Behind the scenes of performance: Performance, practice and management in medical research. Research Policy, 19, 517–534.
Prpic, K. (1996). Characteristics and determinants of eminent scientists’ productivity. Scientometrics, 36(2), 185–206.
Ramsden, P. (1994). Describing and explaining research productivity. Higher Education, 28(2), 207–226.
Reagans, R., & Zuckerman, E. W. (2001). Networks, diversity, and productivity: The social capital of corporate R&D teams. Organization Science, 12(4), 502–517.
Reagans, R., Zuckerman, E., & McEvily, B. (2004). How to make the team: Social networks vs. demography as criteria for designing effective teams. Administrative Science Quarterly, 49(1), 101–133.
Reale, E., & Seeber, M. (2011). Organisation response to institutional pressures in Higher Education: the important role of the disciplines. Higher Education, 61, 1–22.
Reskin, B. F. (1979). Academic Sponsorship and Scientists Careers. Sociology of Education, 52(3), 129–146.
Sanz-Menéndez, L., & Cruz-Castro, L. (2003). Coping with environmental pressures: public research organisations responses to funding crises. Research Policy, 32, 1293–1308.
Schubert, A., & Braun, T. (1996). Cross-field normalization of scientometric indicators. Scientometrics, 36(3), 311–324.
Shin, J. C., & Cummings, W. K. (2010). Multilevel analysis of academic publishing across disciplines: research preference, collaboration, and time on research. Scientometrics, 85, 581–594.
Sindermann, C. J. (1985). The joy of science. New York: Plenum.
Smeby, J. C., & Try, S. (2005). Departmental contexts and faculty research activity in Norway. Research in Higher Education, 46(6), 593–619.
Spangenberg, J. F. A., Starmans, R., Bally, Y. W., Breemhaar, B., Nijhuis, F. J. N., & Vandorp, C. A. F. (1990). Prediction of Scientific Performance in Clinical Medicine. Research Policy, 19(3), 239–255.
Stankiewicz, R. (1976). Research groups and the academic research organization. Sociologisk Forskning, 13(2), 20–32.
Stankiewicz, R. (1979). The size and age of Swedish academic research groups and their scientific performance. In F. M. Andrews (Ed.), Scientific productivity. The effectiveness of research groups in six countries (pp. 191–222). Cambridge: Cambridge University Press.
Strandholm, K., Kumar, K., & Subramanian, R. (2004). Examining the interrelationships among perceived environmental change, strategic response, managerial characteristics, and organizational performance. Journal of Business Research, 57, 58–68.
Stvilia, B., Hinnant, C., Schindler, K., Worrall, A., Burnett, G., Burnett, K., et al. (2011). Team diversity and publication patterns in a scientific laboratory. Journal of American Society for Information Science and Technology, 62(2), 270–283.
Tijssen, R. J. W. (2003). Scoreboards of research excellence. Research Evaluation, 12(2), 91–103.
Tushman, M. L., & Katz, R. (1980). External Communication and Project Performance—An Investigation into the Role of Gatekeepers. Management Science, 26(11), 1071–1085.
Urwick, L. F. (1956). The manager’s span of control. Harvard Business Review, 34, 39–47.
Van den Besselaar, P. (2012). Grant committee membership: service or self-service? Journal of Informetrics, 6, 580–585.
Van den Besselaar, P., & Leydesdorff, L. (2009). Past performance, peer review, and project selection: A case study in the social and behavioral sciences. Research Evaluation, 18(4), 273–288.
Van der Weijden, I., De Gilder, D., Groenewegen, P., & Geerling, M. (2008a). Organizational culture, performance and career choices of Ph.D.s: A case study of Dutch Medical Researchers. Higher Education Policy, 21, 305–321.
Van der Weijden, I., De Gilder, D., Groenewegen, P., & Klasen, E. (2008b). Implications of managerial control on performance of Dutch academic (bio)medical and health research groups. Research Policy, 37, 1616–1629.
Van der Weijden, I., Verbree, M., & Van den Besselaar, P. (2012). From bench to bedside: an exploratory study of the societal orientation in Dutch biomedical and health research groups. Science and Public Policy, 39(3), 285–303.
Van Raan, A. F. J. (2004). Measuring science. Capita selecta of current main issues. In H. F. Moed, W. Glanzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research (pp. 19–50). Dordrecht: Kluwer Academic Publishers.
Van Rijnsoever, F. J., Hessels, L. K., & Vandeberg, R. L. J. (2008). A resource-based view on the interactions of university researchers. Research Policy, 37, 1255–1266.
Verbree, M., Van der Weijden, I., & Van den Besselaar, P. (2013a). Academic leadership of high-performing research groups. In S. Hemlin, C. M. Allwood, B. Martin, & M. Mumford (Eds.), Creativity and leadership in science, technology, and innovation. London: Routledge.
Verbree, M., Van der Weijden, I., & Van den Besselaar, P. (2013b). Age, generation and life cycle effects on academic leadership. In S. Hemlin, C. M. Allwood, B. Martin, & M. Mumford (Eds.), Creativity and leadership in science, technology, and innovation. London: Routledge.
Visart, N. (1979). Communication between and within research units. In F. M. Andrews (Ed.), Scientific productivity The effectiveness of research groups in six countries (pp. 223–252). Cambridge: Cambridge University Press/UNESCO.
Von Tunzelmann, N., Ranga, M., Martin, B., & Geuna, A. (2003). The effects of size on research performance: A SPRU review: SPRU Science and Technology Policy Research. Document Number).
Wagner, C. S. (2005). Six case studies of international collaboration in science. Scientometrics, 62(1), 3–26.
Whitley, R. (2000). The intellectual and social organization of the sciences (2nd ed.). New York: Oxford University Press.
Wouters, P. (1999). The citation culture. PhD thesis University of Amsterdam.
Zaleznik, A. (1977). Managers and leaders. Are they different? Harvard Business Review, 55(3), 67–78.
Acknowledgments
The authors acknowledge the very useful comments of two anonymous reviewers.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Verbree, M., Horlings, E., Groenewegen, P. et al. Organizational factors influencing scholarly performance: a multivariate study of biomedical research groups. Scientometrics 102, 25–49 (2015). https://doi.org/10.1007/s11192-014-1437-x
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-014-1437-x