{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,25]],"date-time":"2024-08-25T20:24:21Z","timestamp":1724617461425},"reference-count":42,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2023,2,14]],"date-time":"2023-02-14T00:00:00Z","timestamp":1676332800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,2,14]],"date-time":"2023-02-14T00:00:00Z","timestamp":1676332800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1548562","1053575"],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1445604","2005506"],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1700765"],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Scientometrics"],"published-print":{"date-parts":[[2023,6]]},"abstract":"Abstract<\/jats:title>This paper uses accounting concepts\u2014particularly the concept of Return on Investment (ROI)\u2014to reveal the quantitative value of scientific research pertaining to a major US cyberinfrastructure project (XSEDE\u2014the eXtreme Science and Engineering Discovery Environment). XSEDE provides operational and support services for advanced information technology systems, cloud systems, and supercomputers supporting non-classified US research, with an average budget for XSEDE of US$20M+ per year over the period studied (2014\u20132021). To assess the financial effectiveness of these services, we calculated a proxy for ROI, and converted quantitative measures of XSEDE service delivery into financial values using costs for service from the US marketplace. We calculated two estimates of ROI: a Conservative Estimate, functioning as a lower bound and using publicly available data for a lower valuation of XSEDE services; and a Best Available Estimate, functioning as a more accurate estimate, but using some unpublished valuation data. Using the largest dataset assembled for analysis of ROI for a cyberinfrastructure project, we found a Conservative Estimate of ROI of 1.87, and a Best Available Estimate of ROI of 3.24. Through accounting methods, we show that XSEDE services offer excellent value to the US government, that the services offered uniquely by XSEDE (that is, not otherwise available for purchase) were the most valuable to the facilitation of US research activities, and that accounting-based concepts hold great value for understanding the mechanisms of scientific research generally.<\/jats:p>","DOI":"10.1007\/s11192-022-04539-8","type":"journal-article","created":{"date-parts":[[2023,2,17]],"date-time":"2023-02-17T05:52:31Z","timestamp":1676613151000},"page":"3225-3255","update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Use of accounting concepts to study research: return on investment in XSEDE, a US cyberinfrastructure service"],"prefix":"10.1007","volume":"128","author":[{"ORCID":"http:\/\/orcid.org\/0000-0003-2423-9019","authenticated-orcid":false,"given":"Craig A.","family":"Stewart","sequence":"first","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-9869-1780","authenticated-orcid":false,"given":"Claudia M.","family":"Costa","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-5705-9527","authenticated-orcid":false,"given":"Julie A.","family":"Wernert","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0003-4354-7092","authenticated-orcid":false,"given":"Winona","family":"Snapp-Childs","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-9491-238X","authenticated-orcid":false,"given":"Marques","family":"Bland","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-9129-1223","authenticated-orcid":false,"given":"Philip","family":"Blood","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0001-6010-669X","authenticated-orcid":false,"given":"Terry","family":"Campbell","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-2823-3127","authenticated-orcid":false,"given":"Peter","family":"Couvares","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0001-7078-6609","authenticated-orcid":false,"given":"Jeremy","family":"Fischer","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0001-8082-8980","authenticated-orcid":false,"given":"David Y.","family":"Hancock","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0001-6393-0967","authenticated-orcid":false,"given":"David L.","family":"Hart","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-7774-1435","authenticated-orcid":false,"given":"Harmony","family":"Jankowski","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-4296-9421","authenticated-orcid":false,"given":"Richard","family":"Knepper","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0001-9996-1888","authenticated-orcid":false,"given":"Donald F.","family":"McMullen","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-2437-5494","authenticated-orcid":false,"given":"Susan","family":"Mehringer","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-9582-5712","authenticated-orcid":false,"given":"Marlon","family":"Pierce","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-9224-3661","authenticated-orcid":false,"given":"Gary","family":"Rogers","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0001-7377-9762","authenticated-orcid":false,"given":"Robert S.","family":"Sinkovits","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0001-7961-2277","authenticated-orcid":false,"given":"John","family":"Towns","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,2,14]]},"reference":[{"issue":"1","key":"4539_CR13","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1007\/s00181-014-0833-7","volume":"48","author":"AW Apon","year":"2015","unstructured":"Apon, A. W., Ngo, L. B., Payne, M. E., & Wilson, P. W. (2015). Assessing the effect of high performance computing capabilities on academic research output. Empirical Economics, 48(1), 283\u2013312. https:\/\/doi.org\/10.1007\/s00181-014-0833-7","journal-title":"Empirical Economics"},{"issue":"1","key":"4539_CR38","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1109\/MCSE.2022.3145608","volume":"24","author":"J Brase","year":"2022","unstructured":"Brase, J., Campbell, N., Helland, B., Hoang, T., Parashar, M., Rosenfield, M., Sexton, J., & Towns, J. (2022). The COVID-19 High-Performance Computing Consortium. Computing in Science & Engineering, 24(1), 78\u201385. https:\/\/doi.org\/10.1109\/MCSE.2022.3145608","journal-title":"Computing in Science & Engineering"},{"key":"4539_CR36","doi-asserted-by":"publisher","unstructured":"Brazil, M., Brunson, D., Culich, A., DeStefano, L., Jennewein, D., Jolley, T., Middelkoop, T., Neeman, H., Rivera, L., Smith, J., & Wernert, J. (2019). Campus champions: Building and sustaining a thriving community of practice around research computing and data. In Proceedings of the Practice and Experience in Advanced Research Computing on Rise of the Machines (learning) (pp. 1\u20137). https:\/\/doi.org\/10.1145\/3332186.3332200","DOI":"10.1145\/3332186.3332200"},{"key":"4539_CR30","doi-asserted-by":"publisher","unstructured":"Chityala, P., Costa, C. M., Wernert, J. A., & Stewart, C. A. (2022a). Cyberinfrastructure value: A survey on perceived importance and usage. In Practice and experience in advanced research computing (PEARC \u201922) (pp. 1\u20134). Association for Computing Machinery. https:\/\/doi.org\/10.1145\/3491418.3535173","DOI":"10.1145\/3491418.3535173"},{"key":"4539_CR31","doi-asserted-by":"crossref","unstructured":"Chityala, P., Costa, C., Wernert, J., & Stewart, C. (2022b). Datasets: Cyberinfrastructure value: Perceived importance and usage. Indiana University. https:\/\/hdl.handle.net\/2022\/27600","DOI":"10.1145\/3491418.3535173"},{"key":"4539_CR21","unstructured":"Costa, C. M., Wernert, J. A., McMullen, D., Stewart, C. A., Blood, P. D., Sinkovits, R., Mehringer, S., Knepper, R., Bland, M., & Rogers, G. (2022). Dataset: XSEDE return on investment data and analysis - July 2014 to August 2021. https:\/\/hdl.handle.net\/2022\/27622"},{"key":"4539_CR25","doi-asserted-by":"publisher","unstructured":"Coulter, E., Fischer, J., Hallock, B., Knepper, R., & Stewart, C. (2016). Implementation of simple XSEDE-like clusters: Science enabled and lessons learned. In Proceedings of the XSEDE16 conference on diversity, big data, and science at scale (pp. 1\u20138). https:\/\/doi.org\/10.1145\/2949550.2949570","DOI":"10.1145\/2949550.2949570"},{"key":"4539_CR40","unstructured":"Frontier Economics Ltd. (2014). Rates of return to investment in science and innovation: A report prepared for the Department for Business, Innovation and Skills (BIS). https:\/\/assets.publishing.service.gov.uk\/government\/uploads\/system\/uploads\/attachment_data\/file\/333006\/bis-14-990-rates-of-return-to-investment-in-science-and-innovation-revised-final-report.pdf"},{"key":"4539_CR22","unstructured":"Globus. (2022). https:\/\/www.globus.org"},{"issue":"16","key":"4539_CR37","doi-asserted-by":"publisher","first-page":"4683","DOI":"10.1002\/cpe.4683","volume":"31","author":"DY Hancock","year":"2019","unstructured":"Hancock, D. Y., Stewart, C. A., Vaughn, M., Fischer, J., Lowe, J. M., Turner, G., Swetnam, T. L., Chafin, T. K., Afgan, E., Pierce, M. E., & Snapp-Childs, W. (2019). Jetstream-Early operations performance, adoption, and impacts. Concurrency and Computation: Practice and Experience, 31(16), 4683. https:\/\/doi.org\/10.1002\/cpe.4683","journal-title":"Concurrency and Computation: Practice and Experience"},{"key":"4539_CR39","doi-asserted-by":"publisher","unstructured":"Hart, D. (2022). Measuring XSEDE: Usage metrics for the XSEDE federation of resources. In Practice and Experience in Advanced Research Computing (PEARC \u201922), (pp. 1\u201312). Association for Computing Machinery, Boston, MA. https:\/\/doi.org\/10.1145\/3491418.3530765.","DOI":"10.1145\/3491418.3530765"},{"key":"4539_CR26","doi-asserted-by":"crossref","unstructured":"IBM. (2021). Cost of a Data Breach Report 2021. https:\/\/www.ibm.com\/security\/data-breach","DOI":"10.1016\/S1361-3723(21)00082-8"},{"issue":"3","key":"4539_CR11","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1007\/s10606-010-9116-9","volume":"19","author":"KF Kee","year":"2010","unstructured":"Kee, K. F., & Browning, L. D. (2010). The dialectical tensions in the funding infrastructure of cyberinfrastructure. Computer Supported Cooperative Work (CSCW), 19(3), 283\u2013308. https:\/\/doi.org\/10.1007\/s10606-010-9116-9","journal-title":"Computer Supported Cooperative Work (CSCW)"},{"key":"4539_CR18","volume-title":"Cost Accounting: Foundations and Evolutions","author":"MR Kinney","year":"2011","unstructured":"Kinney, M. R., & Raiborn, C. A. (2011). Cost accounting: Foundations and evolutions. Cengage Learning.\u00a0ISBN-13: 978-1111971724"},{"issue":"6","key":"4539_CR10","doi-asserted-by":"publisher","first-page":"0157628","DOI":"10.1371\/journal.pone.0157628","volume":"11","author":"R Knepper","year":"2016","unstructured":"Knepper, R., & B\u00f6rner, K. (2016). Comparing the consumption of CPU hours with scientific output for the eXtreme Science and Engineering Discovery Environment (XSEDE). PLoS ONE, 11(6), 0157628. https:\/\/doi.org\/10.1371\/journal.pone.0157628","journal-title":"PloS one"},{"key":"4539_CR41","doi-asserted-by":"publisher","first-page":"1273","DOI":"10.1126\/science.1175335","volume":"324","author":"J Lane","year":"2009","unstructured":"Lane, J. (2009). Assessing the impact of science funding. Science, 324, 1273\u20131275. https:\/\/doi.org\/10.1126\/science.1175335","journal-title":"Science"},{"key":"4539_CR27","unstructured":"LIGO Caltech. (2022). Laser Interferometer Gravitational-Wave Observatory. https:\/\/www.ligo.caltech.edu\/"},{"key":"4539_CR29","doi-asserted-by":"publisher","unstructured":"Mahendran, S., Costa, C. M., Wernert, J. A., & Stewart, C. A. (2022). Institutional value of a Nobel Prize. In Practice and experience in advanced research computing (PEARC \u201922) (pp. 1\u20134). Association for Computing Machinery. https:\/\/doi.org\/10.1145\/3491418.3535157","DOI":"10.1145\/3491418.3535157"},{"key":"4539_CR2","doi-asserted-by":"publisher","unstructured":"National Academies of Sciences. (2020). Information technology innovation: Resurgence, confluence, and continuing impact. National Academies Press. https:\/\/doi.org\/10.17226\/25961","DOI":"10.17226\/25961"},{"key":"4539_CR5","unstructured":"National Science Foundation. (2011). NSF Award#: 1053575 XSEDE: eXtreme Science and Engineering Discovery Environment. https:\/\/www.nsf.gov\/awardsearch\/showAward?AWD_ID=1053575"},{"key":"4539_CR6","unstructured":"National Science Foundation. (2016). NSF Award#: 1548562 XSEDE 2.0: Integrating, enabling and enhancing national cyberinfrastructure with expanding community involvement. https:\/\/www.nsf.gov\/awardsearch\/showAward?AWD_ID=1548562"},{"key":"4539_CR33","unstructured":"National Science Foundation. (2021). Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS). https:\/\/www.nsf.gov\/pubs\/2021\/nsf21555\/nsf21555.htm"},{"key":"4539_CR23","doi-asserted-by":"publisher","unstructured":"Navarro, J.-P., Stewart, C. A., Knepper, R., Liming, L., Lifka, D., & Dahan, M. (2017). The community software repository from XSEDE: A resource for the national research community. In Proceedings of the practice and experience in advanced research computing 2017 on sustainability, success and impact (pp. 1\u20138). https:\/\/doi.org\/10.1145\/3093338.3093373","DOI":"10.1145\/3093338.3093373"},{"issue":"5","key":"4539_CR28","doi-asserted-by":"publisher","first-page":"2199","DOI":"10.1111\/jofi.12185","volume":"69","author":"M O\u2019Hara","year":"2014","unstructured":"O\u2019Hara, M., Yao, C., & Ye, M. (2014). What\u2019s not there: Odd lots and market data. The Journal of Finance, 69(5), 2199\u20132236. https:\/\/doi.org\/10.1111\/jofi.12185","journal-title":"The Journal of Finance"},{"issue":"4","key":"4539_CR12","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1080\/07421222.1992.11517939","volume":"8","author":"CS Saunders","year":"1992","unstructured":"Saunders, C. S., & Jones, J. W. (1992). Measuring performance of the information systems function. Journal of Management Information Systems, 8(4), 63\u201382. https:\/\/doi.org\/10.1080\/07421222.1992.11517939","journal-title":"Journal of Management Information Systems"},{"key":"4539_CR1","doi-asserted-by":"publisher","unstructured":"Schumpeter, J. (2003). Theorie der Wirtschaftlichen Entwicklung (reprint of 1912 book). Springer. https:\/\/doi.org\/10.1007\/0-306-48082-4_2","DOI":"10.1007\/0-306-48082-4_2"},{"key":"4539_CR14","doi-asserted-by":"publisher","unstructured":"Smith, P., & Lien\u00a0Harrell, S. (2021). Research computing on Campus\u2014Application of a production function to the value of academic High-Performance computing. In Practice and experience in advanced research computing (PEARC\u201921) (pp. 1\u20138). https:\/\/doi.org\/10.1145\/3437359.3465564","DOI":"10.1145\/3437359.3465564"},{"key":"4539_CR32","doi-asserted-by":"publisher","unstructured":"Stewart, C. A., Costa, C. M., Wernert, J. A., Hancock, D. Y., McMullen, D. F., Blood, P., Sinkovits, R., Mehringer, S., Knepper, R., Fischer, J., Bland, M., Rogers, G., Couvares, P., Campbell, T., Jankowski, H., Snapp-Childs, W., & Towns, J. (2022). Metrics of financial effectiveness: Return on investment in XSEDE, a national cyberinfrastructure coordination and support organization. In Practice and experience in advanced research computing (PEARC \u201922) (pp. 1\u201312). Association for Computing Machinery. https:\/\/doi.org\/10.1145\/3491418.3530287","DOI":"10.1145\/3491418.3530287"},{"key":"4539_CR17","doi-asserted-by":"publisher","unstructured":"Stewart, C. A., Hancock, D. Y., Wernert, J., Furlani, T., Lifka, D., Sill, A., Berente, N., McMullen, D. F., Cheatham, T., Apon, A., Payne, R., & Slavin, S. D. (2019). Assessment of financial returns on investments in cyberinfrastructure facilities: A survey of current methods. In Proceedings of the practice and experience in advanced research computing on rise of the machines (learning) (pp. 1\u20138). https:\/\/doi.org\/10.1145\/3332186.3332228","DOI":"10.1145\/3332186.3332228"},{"key":"4539_CR4","doi-asserted-by":"publisher","unstructured":"Stewart, C. A., Knepper, R., Link, M. R., Pierce, M., Wernert, E., & Wilkins-Diehr, N. (2019). Cyberinfrastructure, cloud computing, science gateways, visualization, and cyberinfrastructure ease of use. In Advanced methodologies and technologies in network architecture, mobile computing, and data analytics (pp. 157\u2013170). IGI Global. https:\/\/doi.org\/10.4018\/978-1-5225-7598-6.ch012","DOI":"10.4018\/978-1-5225-7598-6.ch012"},{"key":"4539_CR15","doi-asserted-by":"publisher","unstructured":"Stewart, C. A., Roskies, R., Knepper, R., Moore, R. L., Whitt, J., & Cockerill, T. M. (2015). XSEDE value added, cost avoidance, and return on investment. In Proceedings of the 2015 XSEDE conference: Scientific advancements enabled by enhanced cyberinfrastructure (pp. 1\u20138). St. Louis, MO. https:\/\/doi.org\/10.1145\/2792745.2792768","DOI":"10.1145\/2792745.2792768"},{"key":"4539_CR16","doi-asserted-by":"publisher","unstructured":"Stewart, C. A., Wernert, J. A., Costa, C. M., Hancock, D. Y., Knepper, R., & Snapp-Childs, W. (2022). Return on investment in research cyberinfrastructure: State of the art. In Practice and experience in advanced research computing (PEARC \u201922) (pp. 1\u20134). Association for Computing Machinery. https:\/\/doi.org\/10.1145\/3491418.3535131","DOI":"10.1145\/3491418.3535131"},{"key":"4539_CR42","unstructured":"The Value Reporting Foundation. (2021). 2021 International Framework. https:\/\/www.integratedreporting.org\/wp-content\/uploads\/2021\/01\/InternationalIntegratedReportingFramework.pdf"},{"issue":"5","key":"4539_CR3","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1109\/MCSE.2014.80","volume":"16","author":"J Towns","year":"2014","unstructured":"Towns, J., Cockerill, T., Dahan, M., Foster, I., Gaither, K., Grimshaw, A., Hazlewood, V., Lathrop, S., Lifka, D., Peterson, G. D., et al. (2014). XSEDE: Accelerating scientific discovery. Computing in Science & Engineering, 16(5), 62\u201374. https:\/\/doi.org\/10.1109\/MCSE.2014.80","journal-title":"Computing in Science & Engineering"},{"key":"4539_CR9","doi-asserted-by":"publisher","unstructured":"von Laszewski, G., Wang, F., Fox, G. C., Hart, D. L., Furlani, T. R., DeLeon, R. L., & Gallo, S. M. (2015). Peer comparison of XSEDE and NCAR publication data. In 2015 IEEE international conference on cluster computing (pp. 531\u2013532). https:\/\/doi.org\/10.1109\/CLUSTER.2015.98","DOI":"10.1109\/CLUSTER.2015.98"},{"key":"4539_CR20","unstructured":"Wernert, J. A., Costa, C. M., McMullen, D. F., Stewart, C. A., Blood, P. D., Sinkovits, R., Mehringer, S. H., Knepper, R., Bland, M., & Rogers, G. (2022). Technical report: XSEDE return on investment (proxy) data and analysis methods, July 2014 to August 2021. Technical report. https:\/\/hdl.handle.net\/2022\/27623"},{"key":"4539_CR24","doi-asserted-by":"publisher","unstructured":"Wilkins-Diehr, N., Sanielevici, S., Alameda, J., Cazes, J., Crosby, L., Pierce, M., Roskies, R. (2015). An overview of the XSEDE extended collaborative support program. pp. 3-13 In: Gitler, I. and J. Klapp (eds). High Performance Computer Applications - 6th International Conference, ISUM 2015, Revised Selected Papers. Springer Communications in Computer and Information Science, vol. 595. ISSN 1865-0929. https:\/\/doi.org\/10.1007\/978-3-319-32243-8_1","DOI":"10.1007\/978-3-319-32243-8_1"},{"key":"4539_CR19","unstructured":"WK Kellogg Foundation. (2004). WK Kellogg Foundation logic model development guide. WK Kellogg Foundation. https:\/\/wkkf.issuelab.org\/resource\/logic-model-development-guide.html"},{"key":"4539_CR34","unstructured":"XSEDE. (2021). Campus champions. https:\/\/hdl.handle.net\/2142\/114451"},{"key":"4539_CR7","unstructured":"XSEDE. (2022a): URLs for XSEDE annual or project year reports. https:\/\/hdl.handle.net\/2142\/114456"},{"key":"4539_CR8","unstructured":"XSEDE. (2022b): XSEDE Federation. https:\/\/hdl.handle.net\/2142\/114454"},{"key":"4539_CR35","unstructured":"XSEDE. (2020). XSEDE Service Provider Checklist Version 4.3. http:\/\/hdl.handle.net\/2142\/108886"}],"container-title":["Scientometrics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-022-04539-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11192-022-04539-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-022-04539-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,23]],"date-time":"2023-05-23T17:24:13Z","timestamp":1684862653000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11192-022-04539-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,14]]},"references-count":42,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2023,6]]}},"alternative-id":["4539"],"URL":"https:\/\/doi.org\/10.1007\/s11192-022-04539-8","relation":{},"ISSN":["0138-9130","1588-2861"],"issn-type":[{"value":"0138-9130","type":"print"},{"value":"1588-2861","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,14]]},"assertion":[{"value":"28 December 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 September 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 February 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"M. Bland, P. Blood, C. M. Costa, Hart, H. Jankowski, R. Knepper, D. F. McMullen, S. Mehringer, M. Pierce, G. Rogers, R. S. Sinkovits, W. Snapp-Childs, C. A. Stewart, J. Towns, and J. A. Wernert were funded in part by NSF awards in support of XSEDE and XSEDE2. J. Fischer, D. Y. Hancock, and M. Pierce were funded in part by NSF grant awards in support of Jetstream and Jetstream2. P. Couvares was funded in part by an NSF award in support of the LIGO project.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Surveys employed in the course of this research were approved by the Indiana University Institutional Review Board.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"Not applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}]}}