{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,6,4]],"date-time":"2024-06-04T13:18:19Z","timestamp":1717507099980},"reference-count":34,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2020,3,14]],"date-time":"2020-03-14T00:00:00Z","timestamp":1584144000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,3,14]],"date-time":"2020-03-14T00:00:00Z","timestamp":1584144000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Scientometrics"],"published-print":{"date-parts":[[2020,5]]},"abstract":"Abstract<\/jats:title>Since the h<\/jats:italic>-index has been invented, it is the most frequently discussed bibliometric value and one of the most commonly used metrics to quantify a researcher\u2019s scientific output. The more it is increasingly gaining popularity to use the metric as an indication of the quality of a job applicant or an employee the more important it is to assure its correctitude. Many platforms offer the h<\/jats:italic>-index of a scientist as a service, sometimes without the explicit knowledge of the respective person. In this article we show that looking up the h<\/jats:italic>-index for a researcher on the five most commonly used platforms, namely AMiner, Google Scholar, ResearchGate, Scopus and Web of Science, results in a variance that is in many cases as large as the average value. This is due to the varying definitions of what a scientific article is, the underlying data basis, and different qualities of the entity recognition problem. To perform our study, we crawled the h<\/jats:italic>-index of the worlds top researchers according to two different rankings, all the Nobel Prize laureates except Literature and Peace, and the teaching staff of the computer science department of the TU Kaiserslautern Germany with whom we additionally computed their h<\/jats:italic>-index manually. Thus we showed that the individual h<\/jats:italic>-indices differ to an alarming extent between the platforms. We observed that researchers with an extraordinary high h<\/jats:italic>-index and researchers with an index appropriate to the scientific career path and the respective scientific field are affected alike by these problems.<\/jats:p>","DOI":"10.1007\/s11192-020-03407-7","type":"journal-article","created":{"date-parts":[[2020,3,14]],"date-time":"2020-03-14T08:02:49Z","timestamp":1584172969000},"page":"735-751","update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Quantitative analysis of automatic performance evaluation systems based on the h-index"],"prefix":"10.1007","volume":"123","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-1598-1812","authenticated-orcid":false,"given":"Marc P.","family":"Hauer","sequence":"first","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-7845-813X","authenticated-orcid":false,"given":"Xavier C. R.","family":"Hofmann","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-3527-1092","authenticated-orcid":false,"given":"Tobias D.","family":"Krafft","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-4294-9017","authenticated-orcid":false,"given":"Katharina A.","family":"Zweig","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,3,14]]},"reference":[{"key":"3407_CR1","unstructured":"AMiner (2017a) AMiner author search facility. Retrieved on May 15, 2017 from https:\/\/aminer.org\/."},{"key":"3407_CR2","unstructured":"AMiner (2017b) AMiner researcher with highest h-index. Retrieved on May 7, 2017 from https:\/\/aminer.org\/ranks\/experts\/h_index."},{"key":"3407_CR3","doi-asserted-by":"publisher","first-page":"737","DOI":"10.1038\/448737a","volume":"448","author":"P Ball","year":"2007","unstructured":"Ball, P. (2007). Achievement index climbs the ranks. Nature, 448, 737.","journal-title":"Nature"},{"issue":"2","key":"3407_CR4","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1007\/s11192-008-0216-y","volume":"74","author":"J Bar-Ilan","year":"2008","unstructured":"Bar-Ilan, J. (2008). Which h-index?\u2014A comparison of WoS, Scopus and Google Scholar. Scientometrics, 74(2), 257\u2013271.","journal-title":"Scientometrics"},{"issue":"1","key":"3407_CR5","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1007\/s11192-006-0090-4","volume":"68","author":"PD Batista","year":"2006","unstructured":"Batista, P. D., Campiteli, M. G., & Kinouchi, O. (2006). Is it possible to compare researchers with different scientific interests? Scientometrics, 68(1), 179\u2013189.","journal-title":"Scientometrics"},{"key":"3407_CR6","doi-asserted-by":"publisher","unstructured":"Diakopoulos, N. (2014). Algorithmic-accountability: On the investigation of black boxes. Tow Center for Digital Journalism. https:\/\/doi.org\/10.7916\/D8ZK5TW2.","DOI":"10.7916\/D8ZK5TW2"},{"issue":"1","key":"3407_CR7","first-page":"105","volume":"38","author":"EA Drost","year":"2011","unstructured":"Drost, E. A. (2011). Validity and reliability in social science research. Education Research and perspectives, 38(1), 105.","journal-title":"Education Research and perspectives"},{"issue":"5","key":"3407_CR8","doi-asserted-by":"publisher","first-page":"250","DOI":"10.1016\/j.tree.2008.01.009","volume":"23","author":"L Engqvist","year":"2008","unstructured":"Engqvist, L., & Frommen, J. G. (2008). The h-index and self-citations. Trends in Ecology & Evolution, 23(5), 250\u2013252.","journal-title":"Trends in Ecology & Evolution"},{"issue":"2","key":"3407_CR9","doi-asserted-by":"publisher","first-page":"338","DOI":"10.1096\/fj.07-9492lsf","volume":"22","author":"ME Falagas","year":"2008","unstructured":"Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., & Pappas, G. (2008). Comparison of PubMed, scopus, web of science, and google scholar: strengths and weaknesses. The FASEB Journal, 22(2), 338\u2013342. https:\/\/doi.org\/10.1096\/fj.07-9492lsf.","journal-title":"The FASEB Journal"},{"key":"3407_CR10","unstructured":"Gapski, H. (2015). Big data und medienbildung. Zwischen Kontrollverlust, Selbstverteidigung und Souver\u00e4nit\u00e4t in der digitalen Welt, D\u00fcsseldorf-M\u00fcnchen."},{"key":"3407_CR11","unstructured":"Google Scholar. (2017). Helping menue on Google Scholar. Retrieved on May 6, 2017 from https:\/\/scholar.google.de\/intl\/de\/scholar\/help.html#coverage."},{"key":"3407_CR12","unstructured":"Harzing, A. W., et\u00a0al. (2007). Publish or perish. https:\/\/harzing.com\/resources\/publish-or-perish."},{"issue":"46","key":"3407_CR13","doi-asserted-by":"publisher","first-page":"16569","DOI":"10.1073\/pnas.0507655102","volume":"102","author":"JE Hirsch","year":"2005","unstructured":"Hirsch, J. E. (2005). An index to quantify an individual\u2019s scientific research output. Proceedings of the National academy of Sciences of the United States of America, 102(46), 16569.","journal-title":"Proceedings of the National academy of Sciences of the United States of America"},{"issue":"3","key":"3407_CR14","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/s11192-007-1805-x","volume":"73","author":"JE Iglesias","year":"2007","unstructured":"Iglesias, J. E., & Pecharrom\u00e1n, C. (2007). Scaling the h-index for different scientific isi fields. Scientometrics, 73(3), 303\u2013320.","journal-title":"Scientometrics"},{"key":"3407_CR15","doi-asserted-by":"publisher","unstructured":"Jordan, K. (2014). Academics\u2019 awareness, perceptions and uses of social networking sites: Analysis of a social networking sites survey dataset. https:\/\/doi.org\/10.2139\/ssrn.2507318.","DOI":"10.2139\/ssrn.2507318"},{"key":"3407_CR16","doi-asserted-by":"publisher","unstructured":"Lischka, K., & Klingel, A. (2017). Wenn Maschinen Menschen bewerten. Bertelsmann-Stiftung: Arbeitspapier. https:\/\/doi.org\/10.11586\/2017025.","DOI":"10.11586\/2017025"},{"issue":"5","key":"3407_CR17","doi-asserted-by":"publisher","first-page":"342","DOI":"10.1002\/(SICI)1097-4571(198909)40:5<342::AID-ASI7>3.0.CO;2-U","volume":"40","author":"MH MacRoberts","year":"1989","unstructured":"MacRoberts, M. H., & MacRoberts, B. R. (1989). Problems of citation analysis: A critical review. Journal of the American Society for Information Science, 40(5), 342.","journal-title":"Journal of the American Society for Information Science"},{"issue":"4970","key":"3407_CR18","doi-asserted-by":"publisher","first-page":"450","DOI":"10.1038\/205450d0","volume":"205","author":"Nature","year":"1965","unstructured":"Nature. (1965). Science citation index. Nature, 205(4970), 450\u2013450. https:\/\/doi.org\/10.1038\/205450d0.","journal-title":"Nature"},{"key":"3407_CR19","doi-asserted-by":"publisher","DOI":"10.1038\/462825a","author":"Nature","year":"2017","unstructured":"Nature. (2017). Credit where credit is due. Nature,. https:\/\/doi.org\/10.1038\/462825a.","journal-title":"Nature"},{"key":"3407_CR20","unstructured":"Nobel. (2017). List of all Nobel laureates. Retrieved May 7, 2017 from https:\/\/www.nobelprize.org\/nobel_prizes\/."},{"key":"3407_CR21","volume-title":"Weapons of math destruction: How big data increases inequality and threatens democracy","author":"C O\u2019Neil","year":"2016","unstructured":"O\u2019Neil, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. New York: Crown Publishing Group."},{"issue":"43","key":"3407_CR22","doi-asserted-by":"publisher","first-page":"15316","DOI":"10.1073\/pnas.1323111111","volume":"111","author":"AM Petersen","year":"2014","unstructured":"Petersen, A. M., Fortunato, S., Pan, R. K., Kaski, K., Penner, O., Rungi, A., et al. (2014). Reputation and impact in academic careers. Proceedings of the National Academy of Sciences, 111(43), 15316\u201315321.","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"7431","key":"3407_CR23","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1038\/493159a","volume":"493","author":"H Piwowar","year":"2013","unstructured":"Piwowar, H. (2013). Altmetrics: Value all research products. Nature, 493(7431), 159.","journal-title":"Nature"},{"key":"3407_CR24","unstructured":"Razak, R. A., & Fahrurazi, F. R. (2011). Agile testing with selenium. In: 2011 5th Malaysian conference in software engineering (MySEC) (pp. 217\u2013219). IEEE."},{"key":"3407_CR25","unstructured":"ReasearchGate. (2017). ResearchGate author search facility. Retrievedon May 07, 2017 from https:\/\/www.researchgate.net\/."},{"key":"3407_CR26","unstructured":"Reuters, T. (2008). Web of science: The definitive resource for global research. l\u00ednea. Disponible en:\/scientific.thomsonreuters.com."},{"issue":"1","key":"3407_CR27","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1186\/1755-7682-4-25","volume":"4","author":"T Saleem","year":"2011","unstructured":"Saleem, T. (2011). The Hirsch index-a play on numbers or a true appraisal of academic output? International Archives of Medicine, 4(1), 25.","journal-title":"International Archives of Medicine"},{"key":"3407_CR28","first-page":"33","volume-title":"Das demokratische Unternehmen","author":"T Sattelberger","year":"2015","unstructured":"Sattelberger, T. (2015). Abh\u00e4ngiger oder souver\u00e4ner Unternehmensb\u00fcrger \u2013 der Mensch in der Aera der Digitalisierung. In T. Sattelberger, I. Welpe, & A. Boes (Eds.), Das demokratische Unternehmen (pp. 33\u201353). Freiburg, M\u00fcnchen: Neue Arbeits-und F\u00fchrungskulturen im Zeitalter digitaler Wirtschaft."},{"issue":"3","key":"3407_CR29","doi-asserted-by":"publisher","first-page":"30002","DOI":"10.1209\/0295-5075\/78\/30002","volume":"78","author":"M Schreiber","year":"2007","unstructured":"Schreiber, M. (2007). Self-citation corrections for the Hirsch index. Europhysics Letters, 78(3), 30002.","journal-title":"Europhysics Letters"},{"key":"3407_CR30","unstructured":"Scopus. (2017). Scopus author search facility. Retrieved May 07, 2017 from https:\/\/www.scopus.com\/freelookup\/form\/author.uri."},{"key":"3407_CR31","doi-asserted-by":"crossref","unstructured":"Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., & Su, Z. (2008). Arnetminer: extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 990\u2013998). ACM.","DOI":"10.1145\/1401890.1402008"},{"key":"3407_CR32","unstructured":"Web of Science. (2017). Web Of Science author search facility. Retrieved May 7, 2017 from https:\/\/apps.webofknowledge.com\/WOS_GeneralSearch_input.do?product=WOS&search_mode=GeneralSearch."},{"key":"3407_CR33","unstructured":"Webometrics. (2017). Webometrics Ranking of World Universities. Retrieved on May 30, 2017 from www.webometrics.info\/en\/About_Us."},{"key":"3407_CR34","doi-asserted-by":"publisher","unstructured":"Zweig, K. A. (2018). Wie Maschinen irren k\u00f6nnen. Verantwortlichkeiten und Fehlerquellen in Prozessen algorithmischer Entscheidungsfindung. Bertelsmann-Stiftung Arbeitspapier. https:\/\/doi.org\/10.11586\/2018006.","DOI":"10.11586\/2018006"}],"container-title":["Scientometrics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-020-03407-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s11192-020-03407-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-020-03407-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,14]],"date-time":"2021-03-14T00:21:42Z","timestamp":1615681302000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s11192-020-03407-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,14]]},"references-count":34,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2020,5]]}},"alternative-id":["3407"],"URL":"https:\/\/doi.org\/10.1007\/s11192-020-03407-7","relation":{},"ISSN":["0138-9130","1588-2861"],"issn-type":[{"value":"0138-9130","type":"print"},{"value":"1588-2861","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3,14]]},"assertion":[{"value":"18 April 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 March 2020","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}