{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,16]],"date-time":"2024-09-16T11:08:18Z","timestamp":1726484898252},"reference-count":119,"publisher":"Springer Science and Business Media LLC","issue":"11","license":[{"start":{"date-parts":[[2023,3,24]],"date-time":"2023-03-24T00:00:00Z","timestamp":1679616000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,24]],"date-time":"2023-03-24T00:00:00Z","timestamp":1679616000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010685","name":"H2020 Science with and for Society","doi-asserted-by":"publisher","award":["786641"],"id":[{"id":"10.13039\/100010685","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010661","name":"Horizon 2020 Framework Programme","doi-asserted-by":"publisher","award":["945539"],"id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Artif Intell Rev"],"published-print":{"date-parts":[[2023,11]]},"abstract":"Abstract<\/jats:title>Artificial intelligence (AI) is producing highly beneficial impacts in many domains, from transport to healthcare, from energy distribution to marketing, but it also raises concerns about undesirable ethical and social consequences. AI impact assessments (AI-IAs) are a way of identifying positive and negative impacts early on to safeguard AI\u2019s benefits and avoid its downsides. This article describes the first systematic review of these AI-IAs. Working with a population of 181 documents, the authors identified 38 actual AI-IAs and subjected them to a rigorous qualitative analysis with regard to their purpose, scope, organisational context, expected issues, timeframe, process and methods, transparency and challenges. The review demonstrates some convergence between AI-IAs. It also shows that the field is not yet at the point of full agreement on content, structure and implementation. The article suggests that AI-IAs are best understood as means to stimulate reflection and discussion concerning the social and ethical consequences of AI ecosystems. Based on the analysis of existing AI-IAs, the authors describe a baseline process of implementing AI-IAs that can be implemented by AI developers and vendors and that can be used as a critical yardstick by regulators and external observers to evaluate organisations\u2019 approaches to AI.<\/jats:p>","DOI":"10.1007\/s10462-023-10420-8","type":"journal-article","created":{"date-parts":[[2023,3,24]],"date-time":"2023-03-24T09:02:57Z","timestamp":1679648577000},"page":"12799-12831","update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":33,"title":["A systematic review of artificial intelligence impact assessments"],"prefix":"10.1007","volume":"56","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-4058-4456","authenticated-orcid":false,"given":"Bernd Carsten","family":"Stahl","sequence":"first","affiliation":[]},{"given":"Josephina","family":"Antoniou","sequence":"additional","affiliation":[]},{"given":"Nitika","family":"Bhalla","sequence":"additional","affiliation":[]},{"given":"Laurence","family":"Brooks","sequence":"additional","affiliation":[]},{"given":"Philip","family":"Jansen","sequence":"additional","affiliation":[]},{"given":"Blerta","family":"Lindqvist","sequence":"additional","affiliation":[]},{"given":"Alexey","family":"Kirichenko","sequence":"additional","affiliation":[]},{"given":"Samuel","family":"Marchal","sequence":"additional","affiliation":[]},{"given":"Rowena","family":"Rodrigues","sequence":"additional","affiliation":[]},{"given":"Nicole","family":"Santiago","sequence":"additional","affiliation":[]},{"given":"Zuzanna","family":"Warso","sequence":"additional","affiliation":[]},{"given":"David","family":"Wright","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,3,24]]},"reference":[{"key":"10420_CR1","volume-title":"Human rights in the age of artificial intelligence","author":"Access Now","year":"2018","unstructured":"Access Now (2018) Human rights in the age of artificial intelligence. Access Now"},{"key":"10420_CR2","volume-title":"The Toronto declaration: protecting the right to equality and non-discrimination in machine learning systems","author":"Access Now Policy Team","year":"2018","unstructured":"Access Now Policy Team (2018) The Toronto declaration: protecting the right to equality and non-discrimination in machine learning systems. Access Now Policy Team, Toronto"},{"key":"10420_CR3","volume-title":"Examining tools for assessing algorithmic systems the Black Box","author":"Ada Lovelace Institute","year":"2020","unstructured":"Ada Lovelace Institute (2020) Examining tools for assessing algorithmic systems the Black Box. Ada Lovelace Institute, London"},{"key":"10420_CR4","volume-title":"Algorithmic impact assessment: a case study in healthcare","author":"Ada Lovelace Institute","year":"2022","unstructured":"Ada Lovelace Institute (2022) Algorithmic impact assessment: a case study in healthcare. Ada Lovelace Institute, London"},{"key":"10420_CR5","first-page":"98","volume":"84","author":"R Adner","year":"2006","unstructured":"Adner R (2006) Match your innovation strategy to your innovation ecosystem. Harv Bus Rev 84:98\u2013107","journal-title":"Harv Bus Rev"},{"key":"10420_CR6","first-page":"39","volume":"43","author":"R Adner","year":"2017","unstructured":"Adner R (2017) Ecosystem as structure: an actionable construct for strategy. J Manag 43:39\u201358","journal-title":"J Manag"},{"key":"10420_CR7","volume-title":"Algorithmic impact assessments: a practical framework for public agency accountability","author":"AI Now Institute","year":"2018","unstructured":"AI Now Institute (2018a) Algorithmic impact assessments: a practical framework for public agency accountability. AI Now Institute, New York"},{"key":"10420_CR8","unstructured":"AI Now Institute (2018b) Algorithmic impact assessments: toward accountable automation in public agencies. https:\/\/medium.com\/@AINowInstitute\/algorithmic-impact-assessments-toward-accountable-automation-in-public-agencies-bd9856e6fdde. Accessed 26 Apr 2021"},{"key":"10420_CR9","unstructured":"AIEI Group (2020) From principles to practice\u2014an interdisciplinary framework to operationalise AI ethics. VDE\/Bertelsmann Stiftung"},{"key":"10420_CR10","doi-asserted-by":"crossref","unstructured":"Andrade NNG, Kontschieder V (2021) AI impact assessment: a policy prototyping experiment. OpenLoop","DOI":"10.2139\/ssrn.3772500"},{"key":"10420_CR11","first-page":"1","volume":"2","author":"J Aronson","year":"1995","unstructured":"Aronson J (1995) A pragmatic view of thematic analysis. Qual Rep 2:1\u20133","journal-title":"Qual Rep"},{"key":"10420_CR12","doi-asserted-by":"publisher","first-page":"311","DOI":"10.1016\/S0377-2217(00)00074-6","volume":"128","author":"HA Becker","year":"2001","unstructured":"Becker HA (2001) Social impact assessment. Eur J Oper Res 128:311\u2013321. https:\/\/doi.org\/10.1016\/S0377-2217(00)00074-6","journal-title":"Eur J Oper Res"},{"key":"10420_CR13","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1007\/s11292-017-9286-2","volume":"13","author":"R Berk","year":"2017","unstructured":"Berk R (2017) An impact assessment of machine learning risk forecasts on parole board decisions and recidivism. J Exp Criminol 13:193\u2013216. https:\/\/doi.org\/10.1007\/s11292-017-9286-2","journal-title":"J Exp Criminol"},{"key":"10420_CR14","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1057\/jit.2014.26","volume":"30","author":"SK Boell","year":"2015","unstructured":"Boell SK, Cecez-Kecmanovic D (2015) On being \u2018systematic\u2019 in literature reviews in IS. J Inf Technol 30:161\u2013173. https:\/\/doi.org\/10.1057\/jit.2014.26","journal-title":"J Inf Technol"},{"key":"10420_CR15","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1191\/1478088706qp063oa","volume":"3","author":"V Braun","year":"2006","unstructured":"Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3:77\u2013101","journal-title":"Qual Res Psychol"},{"key":"10420_CR16","doi-asserted-by":"publisher","unstructured":"Brey P, King O, Jansen P, Dainow B, Erden YJ, Rodrigues R, Resseguier A et al (2022) SIENNA D6.1: Generalised methodology for ethical assessment of emerging technologies, Zenodo. https:\/\/doi.org\/10.5281\/zenodo.7266895","DOI":"10.5281\/zenodo.7266895"},{"key":"10420_CR17","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1145\/3072528","volume":"60","author":"B Brinkman","year":"2017","unstructured":"Brinkman B, Flick C, Gotterbarn D et al (2017) Listening to professional voices: draft 2 of the ACM code of ethics and professional conduct. Commun ACM 60:105\u2013111. https:\/\/doi.org\/10.1145\/3072528","journal-title":"Commun ACM"},{"key":"10420_CR18","volume-title":"Possible elements of a legal framework on artificial intelligence, based on the Council of Europe\u2019s standards on human rights, democracy and the rule of law","author":"CAHAI Ad Hoc Committee on Artificial Intelligence","year":"2022","unstructured":"CAHAI Ad Hoc Committee on Artificial Intelligence (2022) Possible elements of a legal framework on artificial intelligence, based on the Council of Europe\u2019s standards on human rights, democracy and the rule of law. CAHAI Ad Hoc Committee on Artificial Intelligence, Strasbourg"},{"key":"10420_CR19","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1038\/s42256-020-0151-z","volume":"2","author":"RA Calvo","year":"2020","unstructured":"Calvo RA, Peters D, Cave S (2020) Advancing impact assessment for intelligent systems. Nat Mach Intell 2:89\u201391","journal-title":"Nat Mach Intell"},{"key":"10420_CR20","volume-title":"Ethics assessment for research and innovation\u2014part 2: ethical impact assessment framework","author":"CEN-CENELEC","year":"2017","unstructured":"CEN-CENELEC (2017) Ethics assessment for research and innovation\u2014part 2: ethical impact assessment framework. CEN-CENELEC, Brussels"},{"key":"10420_CR21","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1016\/j.clsr.2009.02.002","volume":"25","author":"R Clarke","year":"2009","unstructured":"Clarke R (2009) Privacy impact assessment: its origins and development. Comput Law Secur Rev 25:123\u2013135. https:\/\/doi.org\/10.1016\/j.clsr.2009.02.002","journal-title":"Comput Law Secur Rev"},{"key":"10420_CR22","doi-asserted-by":"crossref","first-page":"410","DOI":"10.1016\/j.clsr.2019.04.007","volume":"35","author":"R Clarke","year":"2019","unstructured":"Clarke R (2019a) Principles and business processes for responsible AI. Comput Law Secur Rev 35:410\u2013422","journal-title":"Comput Law Secur Rev"},{"key":"10420_CR23","doi-asserted-by":"crossref","first-page":"398","DOI":"10.1016\/j.clsr.2019.04.008","volume":"35","author":"R Clarke","year":"2019","unstructured":"Clarke R (2019b) Regulatory alternatives for AI. Comput Law Secur Rev 35:398\u2013409","journal-title":"Comput Law Secur Rev"},{"key":"10420_CR24","unstructured":"CNIL (2015) Privacy impact assessment (PIA) methodology. CNIL"},{"key":"10420_CR25","doi-asserted-by":"publisher","first-page":"31","DOI":"10.26116\/techreg.2019.003","volume":"1","author":"M Coeckelbergh","year":"2019","unstructured":"Coeckelbergh M (2019) Artificial intelligence: some ethical issues and regulatory challenges. Technol Regul 1:31\u201334. https:\/\/doi.org\/10.26116\/techreg.2019.003","journal-title":"Technol Regul"},{"key":"10420_CR28","unstructured":"Council of Europe (2019) Unboxing artificial intelligence: 10 steps to protect human rights. https:\/\/www.coe.int\/en\/web\/commissioner\/view\/-\/asset_publisher\/ugj3i6qSEkhZ\/content\/unboxing-artificial-intelligence-10-steps-to-protect-human-rights. Accessed 11 April 2022"},{"key":"10420_CR29","unstructured":"Deloitte Australia (2020) A moral license for AI - Ethics as a dialogue between firms and communities. https:\/\/www2.deloitte.com\/us\/en\/insights\/focus\/cognitive-technologies\/artificial-intelligence-impact-onsociety.html. Accessed 21 April 2021"},{"key":"10420_CR30","volume-title":"A method for ethical AI in defence","author":"K Devitt","year":"2020","unstructured":"Devitt K, Gan M, Scholz J, Bolia R (2020) A method for ethical AI in defence. Australian Government - Department of Defence, Canberra"},{"key":"10420_CR31","volume-title":"Lessons in practical AI ethics: taking the UK\u2019s AI ecosystem from \u2018what\u2019 to \u2018how\u2019","author":"Digital Catapult","year":"2020","unstructured":"Digital Catapult (2020) Lessons in practical AI ethics: taking the UK\u2019s AI ecosystem from \u2018what\u2019 to \u2018how.\u2019 Digital Catapult, London"},{"key":"10420_CR32","volume-title":"A survey of artificial intelligence risk assessment methodologies: The global state of play and leading practices identified","author":"G Eceani","year":"2021","unstructured":"Eceani G, Koene A, Kumar R et al (2021) A survey of artificial intelligence risk assessment methodologies: The global state of play and leading practices identified. Ernest & Young LLP and Trilateral Research, London"},{"key":"10420_CR33","unstructured":"ECP Platform for the Information Provision (2019) Artificial intelligence impact assessment. https:\/\/ecp.nl\/wp-content\/uploads\/2019\/01\/Artificial-Intelligence-Impact-Assessment-English.pdf. Accessed 18 May 2021"},{"key":"10420_CR34","unstructured":"EDPS (2020) A preliminary opinion on data protection and scientific research"},{"key":"10420_CR35","volume-title":"Artificial intelligence: how knowledge is created, transferred, and used\u2014trends in China, Europe, and the United States","author":"Elsevier","year":"2018","unstructured":"Elsevier (2018) Artificial intelligence: how knowledge is created, transferred, and used\u2014trends in China, Europe, and the United States. Elsevier, Amsterdam"},{"key":"10420_CR36","doi-asserted-by":"crossref","unstructured":"Erd\u00e9lyi OJ, Goldsmith J (2018) Regulating artificial intelligence: proposal for a global solution. In: Proceedings of the 2018 AAAI\/ACM conference on AI, ethics, and society. Association for Computing Machinery, New York, pp 95\u2013101","DOI":"10.1145\/3278721.3278731"},{"key":"10420_CR26","volume-title":"Proposal for a regulation on a European approach for artificial intelligence","author":"European Commission","year":"2021","unstructured":"European Commission (2021) Proposal for a regulation on a european approach for artificial intelligence (COM (2021) 206 final). European Commission. https:\/\/digital-strategy.ec.europa.eu\/en\/library\/proposal-regulation-european-approach-artificialintelligence. Accessed 21 April 2021"},{"key":"10420_CR37","volume-title":"Proposal for a Directive on adapting non contractual civil liability rules to artificial intelligence","author":"European Commission","year":"2022","unstructured":"European Commission (2022) Proposal for a Directive on adapting non contractual civil liability rules to artificial intelligence. European Commission, Brussels"},{"key":"10420_CR38","volume-title":"Charter of fundamental rights of the European Union","author":"European Union","year":"2010","unstructured":"European Union (2010) Charter of fundamental rights of the European Union. European Union, Brussels"},{"key":"10420_CR39","volume-title":"Getting the future right artificial intelligence and fundamental rights; report","author":"European Union Agency for Fundamental Rights","year":"2020","unstructured":"European Union Agency for Fundamental Rights (2020) Getting the future right artificial intelligence and fundamental rights; report. European Union Agency for Fundamental Rights, Vienna"},{"key":"10420_CR40","volume-title":"Liability for artificial intelligence and other emerging digital technologies","author":"Expert Group on Liability and New Technologies","year":"2019","unstructured":"Expert Group on Liability and New Technologies (2019) Liability for artificial intelligence and other emerging digital technologies. European Commission, Luxembourg"},{"key":"10420_CR41","doi-asserted-by":"crossref","unstructured":"Findlay M, Seah J (2020) An ecosystem approach to ethical AI and data use: experimental reflections. In: 2020 IEEE\/ITU international conference on artificial intelligence for good (AI4G). pp 192\u2013197","DOI":"10.1109\/AI4G50087.2020.9311069"},{"key":"10420_CR42","doi-asserted-by":"crossref","unstructured":"Fjeld J, Achten N, Hilligoss H, Nagy A, Srikumar M (2020) Principled artificial intelligence: Mapping consensus in ethical and Rights-based approaches to principles for AI. https:\/\/dash.harvard.edu\/handle\/1\/42160420. Accessed 22 Nov 2020","DOI":"10.2139\/ssrn.3518482"},{"key":"10420_CR43","volume-title":"Getting the future right\u2014artificial intelligence and fundamental rights","author":"FRA","year":"2020","unstructured":"FRA (2020) Getting the future right\u2014artificial intelligence and fundamental rights. European Union Agency for Fundamental Rights, Luxembourg"},{"key":"10420_CR44","doi-asserted-by":"publisher","DOI":"10.1007\/s43681-021-00069-w","author":"A Gardner","year":"2021","unstructured":"Gardner A, Smith AL, Steventon A et al (2021) Ethical funding for trustworthy AI: proposals to address the responsibilities of funders to ensure that projects adhere to trustworthy AI practice. AI Ethics. https:\/\/doi.org\/10.1007\/s43681-021-00069-w","journal-title":"AI Ethics"},{"key":"10420_CR45","unstructured":"GDPR (2016) Regulation (EU) 2016\/679 of the European parliament and of the council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95\/46\/EC (General Data Protection Regulation). Official Journal of the European Union L119\/1"},{"key":"10420_CR46","unstructured":"Gebru T, Morgenstern J, Vecchione B, et al (2020) Datasheets for datasets. https:\/\/arxiv.org\/abs\/1803.09010"},{"key":"10420_CR47","volume-title":"Technology Assessment, Emerging Opportunities, Challenges, and Implications","author":"Government Accountability Office","year":"2018","unstructured":"Government Accountability Office (2018) Technology Assessment, Emerging Opportunities, Challenges, and Implications. Government Accountability Office, Washington, D.C."},{"key":"10420_CR48","volume-title":"Algorithmic impact assessment tool","author":"Government of Canada","year":"2022","unstructured":"Government of Canada (2022) Algorithmic impact assessment tool. Government of Canada, Ottawa"},{"key":"10420_CR27","unstructured":"Government of Canada (2022) Algorithmic impact assessment tool, Government of Canada. https:\/\/www.canada.ca\/en\/government\/system\/digital-government\/digital-government-innovations\/responsible-useai\/algorithmic-impact-assessment.html. Accessed 22 Sept 2022"},{"key":"10420_CR49","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aay7120","author":"D Gunning","year":"2019","unstructured":"Gunning D, Stefik M, Choi J et al (2019) XAI\u2014explainable artificial intelligence. Sci Robot. https:\/\/doi.org\/10.1126\/scirobotics.aay7120","journal-title":"Sci Robot"},{"key":"10420_CR50","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1016\/j.eiar.2004.12.002","volume":"25","author":"N Hartley","year":"2005","unstructured":"Hartley N, Wood C (2005) Public participation in environmental impact assessment\u2014implementing the Aarhus Convention. Environ Impact Assess Rev 25:319\u2013340. https:\/\/doi.org\/10.1016\/j.eiar.2004.12.002","journal-title":"Environ Impact Assess Rev"},{"key":"10420_CR51","volume-title":"Ethics guidelines for trustworthy AI","author":"AI Hleg","year":"2019","unstructured":"Hleg AI (2019) Ethics guidelines for trustworthy AI. European Commission - Directorate-General for Communication, Brussels"},{"key":"10420_CR52","volume-title":"Assessment list for trustworthy AI (ALTAI)","author":"AI Hleg","year":"2020","unstructured":"Hleg AI (2020) Assessment list for trustworthy AI (ALTAI). European Commission, Brussels"},{"key":"10420_CR53","unstructured":"IAIA Impact Assessment. https:\/\/www.iaia.org\/wiki-details.php?ID=4. Accessed 12 Sep 2021"},{"key":"10420_CR54","unstructured":"ICO (2020) Guidance on the AI auditing framework\u2014draft guidance for consultation. Information Commissioner\u2019s Office"},{"key":"10420_CR55","unstructured":"IEEE (2020a) IEEE 7010-2020a\u2014IEEE recommended practice for assessing the impact of autonomous and intelligent systems on human well-being. IEEE"},{"key":"10420_CR56","unstructured":"IEEE Computer Society (2021) IEEE standard model process for addressing ethical concerns during system design\u20147000-2021"},{"key":"10420_CR57","unstructured":"Information Commissioner\u2019s Office (2009) Privacy impact assessment handbook, v. 2.0"},{"key":"10420_CR58","unstructured":"Institute for the future of work (2020) Artificial intelligence in hiring\u2014assessing impacts on equality"},{"key":"10420_CR59","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1007\/s10462-016-9457-y","volume":"46","author":"S Iqbal","year":"2016","unstructured":"Iqbal S, Altaf W, Aslam M et al (2016) Application of intelligent agents in health-care: review. Artif Intell Rev 46:83\u2013112. https:\/\/doi.org\/10.1007\/s10462-016-9457-y","journal-title":"Artif Intell Rev"},{"key":"10420_CR60","doi-asserted-by":"publisher","unstructured":"Ivanova Y (2020) The data protection impact assessment as a tool to enforce Non-discriminatory AI. In: Antunes L, Naldi M, Italiano GF, Rannenberg K, Drogkaris P (eds) Privacy technologies and policy. Springer International Publishing, pp 3\u201324. https:\/\/doi.org\/10.1007\/978-3-030-55196-4_1","DOI":"10.1007\/978-3-030-55196-4_1"},{"key":"10420_CR61","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1038\/s42256-019-0088-2","volume":"1","author":"A Jobin","year":"2019","unstructured":"Jobin A, Ienca M, Vayena E (2019) The global landscape of AI ethics guidelines. Nat Mach Intell 1:389\u2013399. https:\/\/doi.org\/10.1038\/s42256-019-0088-2","journal-title":"Nat Mach Intell"},{"key":"10420_CR62","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3456224","author":"ME Kaminski","year":"2019","unstructured":"Kaminski ME, Malgieri G (2019) Algorithmic impact assessments under the GDPR: producing multi-layered explanations. SSRN Electron J. https:\/\/doi.org\/10.2139\/ssrn.3456224","journal-title":"SSRN Electron J"},{"key":"10420_CR63","unstructured":"Leslie D (2019) Understanding artificial intelligence ethics and safety. https:\/\/www.gov.uk\/guidance\/understanding-artificial-intelligence-ethics-and-safety. Accessed 18 May 2021"},{"key":"10420_CR64","doi-asserted-by":"crossref","first-page":"181","DOI":"10.28945\/479","volume":"9","author":"Y Levy","year":"2006","unstructured":"Levy Y, Ellis TJ (2006) A systems approach to conduct an effective literature review in support of information systems research. Informing Sci 9:181\u2013212","journal-title":"Informing Sci"},{"key":"10420_CR65","doi-asserted-by":"publisher","first-page":"e1000100","DOI":"10.1371\/journal.pmed.1000100","volume":"6","author":"A Liberati","year":"2009","unstructured":"Liberati A, Altman DG, Tetzlaff J et al (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLOS Med 6:e1000100. https:\/\/doi.org\/10.1371\/journal.pmed.1000100","journal-title":"PLOS Med"},{"key":"10420_CR66","unstructured":"Lindblad Kernell E, Bloch Veiberg C, Jacquot C (2020) Human rights impact assessment of digital activities. The Danish Institute for Human Rights"},{"key":"10420_CR67","doi-asserted-by":"publisher","DOI":"10.1016\/j.eiar.2020.106495","author":"W Liu","year":"2021","unstructured":"Liu W, Zhao J, Du L et al (2021) Intelligent comprehensive evaluation system using artificial intelligence for environmental evaluation. Environ Impact Assess Rev. https:\/\/doi.org\/10.1016\/j.eiar.2020.106495","journal-title":"Environ Impact Assess Rev"},{"key":"10420_CR68","doi-asserted-by":"publisher","first-page":"52:1","DOI":"10.1145\/3512899","volume":"6","author":"M Madaio","year":"2022","unstructured":"Madaio M, Egede L, Subramonyam H et al (2022) Assessing the fairness of AI systems: AI practitioners\u2019 processes, challenges, and needs for support. Proc ACM Hum Comput Interact 6:52:1-52:26. https:\/\/doi.org\/10.1145\/3512899","journal-title":"Proc ACM Hum Comput Interact"},{"key":"10420_CR69","doi-asserted-by":"publisher","first-page":"754","DOI":"10.1016\/j.clsr.2018.05.017","volume":"34","author":"A Mantelero","year":"2018","unstructured":"Mantelero A (2018) AI and Big Data: a blueprint for a human rights, social and ethical impact assessment. Comput Law Secur Rev 34:754\u2013772. https:\/\/doi.org\/10.1016\/j.clsr.2018.05.017","journal-title":"Comput Law Secur Rev"},{"key":"10420_CR70","doi-asserted-by":"publisher","first-page":"105561","DOI":"10.1016\/j.clsr.2021.105561","volume":"41","author":"A Mantelero","year":"2021","unstructured":"Mantelero A, Esposito MS (2021) An evidence-based methodology for human rights impact assessment (HRIA) in the development of AI data-intensive systems. Comput Law Secur Rev 41:105561. https:\/\/doi.org\/10.1016\/j.clsr.2021.105561","journal-title":"Comput Law Secur Rev"},{"key":"10420_CR71","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1145\/3148541","volume":"8","author":"CD Martin","year":"2017","unstructured":"Martin CD, Makoundou TT (2017) Taking the high road ethics by design in AI. ACM Inroads 8:35\u201337","journal-title":"ACM Inroads"},{"key":"10420_CR72","doi-asserted-by":"publisher","unstructured":"Metcali J, Moss E, Watkins EA, et al (2021) Algorithmic impact assessments and accountability: ACM. https:\/\/doi.org\/10.1145\/3442188","DOI":"10.1145\/3442188"},{"key":"10420_CR73","unstructured":"Microsoft and Article One (2018) Human Rights Impact Assessment (HRIA) of the human rights risks and opportunities related to artificial intelligence (AI)"},{"key":"10420_CR74","volume-title":"Qualitative data analysis: an expanded sourcebook","author":"MB Miles","year":"1994","unstructured":"Miles MB, Huberman AM (1994) Qualitative data analysis: an expanded sourcebook. Sage, Thousand Oaks"},{"key":"10420_CR75","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-021-10088-y","author":"D Minh","year":"2021","unstructured":"Minh D, Wang HX, Li YF, Nguyen TN (2021) Explainable artificial intelligence: a comprehensive review. Artif Intell Rev. https:\/\/doi.org\/10.1007\/s10462-021-10088-y","journal-title":"Artif Intell Rev"},{"key":"10420_CR76","unstructured":"Ministerie van Algemene Zaken (2021) Fundamental Rights and Algorithms Impact Assessment (FRAIA). Government of the Netherlands"},{"key":"10420_CR77","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0114-4","author":"B Mittelstadt","year":"2019","unstructured":"Mittelstadt B (2019) Principles alone cannot guarantee ethical AI. Nat Mach Intell. https:\/\/doi.org\/10.1038\/s42256-019-0114-4","journal-title":"Nat Mach Intell"},{"key":"10420_CR78","first-page":"75","volume":"71","author":"JF Moore","year":"1993","unstructured":"Moore JF (1993) Predators and prey: a new ecology of competition. Harv Bus Rev 71:75\u201386","journal-title":"Harv Bus Rev"},{"key":"10420_CR79","volume-title":"The impact of artificial intelligence on human rights, democracy and the rule of law","author":"C Muller","year":"2020","unstructured":"Muller C (2020) The impact of artificial intelligence on human rights, democracy and the rule of law. Council of Europe, Ad Hoc Committee on Artificial Intelligence (CAHAI), Strasbourg"},{"key":"10420_CR80","doi-asserted-by":"publisher","first-page":"20180089","DOI":"10.1098\/rsta.2018.0089","volume":"376","author":"P Nemitz","year":"2018","unstructured":"Nemitz P (2018) Constitutional democracy and technology in the age of artificial intelligence. Philos Trans R Soc A 376:20180089. https:\/\/doi.org\/10.1098\/rsta.2018.0089","journal-title":"Philos Trans R Soc A"},{"key":"10420_CR81","volume-title":"AI risk management framework: second draft","author":"NIST","year":"2022","unstructured":"NIST (2022) AI risk management framework: second draft. NIST, Gaithersburg"},{"key":"10420_CR82","volume-title":"Recommendation of the council on artificial intelligence","author":"OECD","year":"2019","unstructured":"OECD (2019) Recommendation of the council on artificial intelligence. OECD, Paris"},{"key":"10420_CR83","unstructured":"Office of Science and Technology Policy (2022) Blueprint for an AI bill of rights. In: The White House. https:\/\/www.whitehouse.gov\/ostp\/ai-bill-of-rights\/. Accessed 4 Oct 2022"},{"key":"10420_CR84","doi-asserted-by":"publisher","unstructured":"Oswald M (2018) Algorithmic risk assessment policing models: lessons from the Durham HART model and \u2018Experimental\u2019 proportionality. https:\/\/doi.org\/10.1080\/13600834.2018.1458455","DOI":"10.1080\/13600834.2018.1458455"},{"key":"10420_CR85","doi-asserted-by":"publisher","first-page":"4953","DOI":"10.1007\/s11269-018-2066-6","volume":"32","author":"D Park","year":"2018","unstructured":"Park D, Um M-J (2018) Robust decision-making technique for strategic environment assessment with deficient information. Water Resour Manag 32:4953\u20134970. https:\/\/doi.org\/10.1007\/s11269-018-2066-6","journal-title":"Water Resour Manag"},{"key":"10420_CR86","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-021-10106-z","author":"Y Peng","year":"2022","unstructured":"Peng Y, Liu E, Peng S et al (2022) Using artificial intelligence technology to fight COVID-19: a review. Artif Intell Rev. https:\/\/doi.org\/10.1007\/s10462-021-10106-z","journal-title":"Artif Intell Rev"},{"key":"10420_CR87","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1007\/978-3-319-55783-0_15","volume-title":"Privacy and identity management. Facing up to next steps","author":"A Persson","year":"2016","unstructured":"Persson A (2016) Implicit bias in predictive data profiling within recruitments. In: Lehmann A, Whitehouse D, Fischer-H\u00fcbner S et al (eds) Privacy and identity management. Facing up to next steps. Springer International Publishing, Cham, pp 212\u2013230"},{"key":"10420_CR88","unstructured":"PricewaterhouseCoopers (2019) Responsible AI Toolkit. PwC. https:\/\/www.pwc.com\/gx\/en\/issues\/data-and-analytics\/artificial-intelligence\/what-is-responsible-ai.html. Accessed 2 Jul 2021"},{"key":"10420_CR89","volume-title":"A practical guide to responsible artificial intelligence (AI)","author":"PWC","year":"2019","unstructured":"PWC (2019) A practical guide to responsible artificial intelligence (AI). PWC, London"},{"key":"10420_CR90","unstructured":"QSR NVivo 11 for Windows Help\u2014Run a coding comparison query. http:\/\/help-nv11.qsrinternational.com\/desktop\/procedures\/run_a_coding_comparison_query.htm. Accessed 12 Sep 2021"},{"key":"10420_CR91","doi-asserted-by":"crossref","unstructured":"Raji ID, Smart A, White RN, et al (2020) Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing. In: Proceedings of the 2020 conference on fairness, accountability, and transparency. ACM, pp 33\u201344","DOI":"10.1145\/3351095.3372873"},{"key":"10420_CR92","volume-title":"Artificial intelligence & human rights: opportunities & risks","author":"FA Raso","year":"2018","unstructured":"Raso FA, Hilligoss H, Krishnamurthy V et al (2018) Artificial intelligence & human rights: opportunities & risks. Social Science Research Network, Rochester"},{"key":"10420_CR93","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1057\/ejis.2014.7","volume":"23","author":"F Rowe","year":"2014","unstructured":"Rowe F (2014) What literature review is not: diversity, boundaries and recommendations. Eur J Inf Syst 23:241\u2013255. https:\/\/doi.org\/10.1057\/ejis.2014.7","journal-title":"Eur J Inf Syst"},{"key":"10420_CR94","unstructured":"Schmitt CE (2018) Evaluating the impact of artificial intelligence on human rights"},{"key":"10420_CR95","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1145\/1278253.1278259","volume":"38","author":"A Schwarz","year":"2007","unstructured":"Schwarz A, Mehta M, Johnson N, Chin WW (2007) Understanding frameworks and reviews: a commentary to assist us in moving our field forward by analyzing our past. SIGMIS Database 38:29\u201350. https:\/\/doi.org\/10.1145\/1278253.1278259","journal-title":"SIGMIS Database"},{"key":"10420_CR96","doi-asserted-by":"publisher","first-page":"102170","DOI":"10.1016\/j.ijinfomgt.2020.102170","volume":"55","author":"JC Sipior","year":"2020","unstructured":"Sipior JC (2020) Considerations for development and use of AI in response to COVID-19. Int J Inf Manag 55:102170. https:\/\/doi.org\/10.1016\/j.ijinfomgt.2020.102170","journal-title":"Int J Inf Manag"},{"key":"10420_CR97","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-69978-9","volume-title":"Artificial intelligence for a better future: an ecosystem perspective on the ethics of AI and emerging digital technologies","author":"BC Stahl","year":"2021","unstructured":"Stahl BC (2021) Artificial intelligence for a better future: an ecosystem perspective on the ethics of AI and emerging digital technologies. Springer International Publishing, Cham"},{"key":"10420_CR98","doi-asserted-by":"publisher","first-page":"55:1","DOI":"10.1145\/2871196","volume":"48","author":"BC Stahl","year":"2016","unstructured":"Stahl BC, Timmermans J, Mittelstadt BD (2016) The ethics of computing: a survey of the computing-oriented literature. ACM Comput Surv 48:55:1-55:38. https:\/\/doi.org\/10.1145\/2871196","journal-title":"ACM Comput Surv"},{"key":"10420_CR99","doi-asserted-by":"publisher","first-page":"374","DOI":"10.1016\/j.jbusres.2020.11.030","volume":"124","author":"BC Stahl","year":"2021","unstructured":"Stahl BC, Andreou A, Brey P et al (2021) Artificial intelligence for human flourishing\u2014beyond principles for machine learning. J Bus Res 124:374\u2013388. https:\/\/doi.org\/10.1016\/j.jbusres.2020.11.030","journal-title":"J Bus Res"},{"key":"10420_CR100","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1007\/s11948-020-00277-3","volume":"27","author":"C Stix","year":"2021","unstructured":"Stix C (2021) Actionable principles for artificial intelligence policy: three pathways. Sci Eng Ethics 27:15. https:\/\/doi.org\/10.1007\/s11948-020-00277-3","journal-title":"Sci Eng Ethics"},{"key":"10420_CR101","unstructured":"Stone P, Brooks R, Brynjolfsson E, et al (2016) Artificial intelligence and life in 2030. One hundred year study on artificial intelligence: report of the 2015\u20132016 study panel. Stanford University, Stanford. https:\/\/ai100.stanford.edu\/2016-report. Accessed 6 Sept 2016"},{"key":"10420_CR102","doi-asserted-by":"publisher","DOI":"10.17705\/1CAIS.03705","author":"M Tate","year":"2015","unstructured":"Tate M, Furtmueller E, Evermann J, Bandara W (2015) Introduction to the special issue: the literature review in information systems. Commun Assoc Inf Syst. https:\/\/doi.org\/10.17705\/1CAIS.03705","journal-title":"Commun Assoc Inf Syst"},{"key":"10420_CR103","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","volume":"25","author":"EJ Topol","year":"2019","unstructured":"Topol EJ (2019) High-performance medicine: the convergence of human and artificial intelligence. Nat Med 25:44\u201356. https:\/\/doi.org\/10.1038\/s41591-018-0300-7","journal-title":"Nat Med"},{"key":"10420_CR104","unstructured":"UK AI Council (2021) AI roadmap"},{"key":"10420_CR105","unstructured":"UK Government (2021) National AI strategy"},{"key":"10420_CR106","unstructured":"UK Governmental Digital Service (2020) Data ethics framework"},{"key":"10420_CR107","unstructured":"UN General Assembly (1966) International covenant on economic, social and cultural rights"},{"key":"10420_CR108","unstructured":"UnBias (2018) Fairness toolkit. In: UnBias. https:\/\/unbias.wp.horizon.ac.uk\/fairness-toolkit\/. Accessed 2 Jul 2021"},{"key":"10420_CR109","volume-title":"First version of a draft text of a recommendation on the Ethics of Artificial Intelligence","author":"UNESCO","year":"2020","unstructured":"UNESCO (2020) First version of a draft text of a recommendation on the Ethics of Artificial Intelligence. UNESCO, Paris"},{"key":"10420_CR110","doi-asserted-by":"crossref","first-page":"84","DOI":"10.2307\/j.ctvhrd092.18","volume-title":"Being profiled: cogitas ergo sum","author":"B Wagner","year":"2018","unstructured":"Wagner B (2018) Ethics as an escape from regulation: from ethics-washing to ethics-shopping. In: Bayamlioglu E, Baraliuc I, Janssens LAW, Hildebrandt M (eds) Being profiled: cogitas ergo sum. Amsterdam University Press, Amsterdam, pp 84\u201390"},{"key":"10420_CR111","doi-asserted-by":"publisher","first-page":"505","DOI":"10.1109\/JPROC.2019.2899422","volume":"107","author":"W Wallach","year":"2019","unstructured":"Wallach W, Marchant G (2019) Toward the agile and comprehensive international governance of AI and robotics [point of view]. Proc IEEE 107:505\u2013508. https:\/\/doi.org\/10.1109\/JPROC.2019.2899422","journal-title":"Proc IEEE"},{"key":"10420_CR112","doi-asserted-by":"publisher","first-page":"201","DOI":"10.1007\/BF00143762","volume":"5","author":"B Whitby","year":"1991","unstructured":"Whitby B (1991) Ethical AI. Artif Intell Rev 5:201\u2013204. https:\/\/doi.org\/10.1007\/BF00143762","journal-title":"Artif Intell Rev"},{"key":"10420_CR113","first-page":"55","volume":"22","author":"C Williams","year":"2020","unstructured":"Williams C (2020) A health rights impact assessment guide for artificial intelligence projects\u2014abstract\u2014Europe PMC. Health Hum Rights J 22:55\u201362","journal-title":"Health Hum Rights J"},{"key":"10420_CR114","unstructured":"Winter P, Eder S, Weissenb\u00f6ck J, et al (2021) White paper\u2014trusted artificial intelligence: towards certification of machine learning applications. T\u00dcV Austria"},{"key":"10420_CR115","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1007\/s10676-010-9242-6","volume":"13","author":"D Wright","year":"2011","unstructured":"Wright D (2011) A framework for the ethical impact assessment of information technology. Ethics Inf Technol 13:199\u2013226. https:\/\/doi.org\/10.1007\/s10676-010-9242-6","journal-title":"Ethics Inf Technol"},{"key":"10420_CR116","doi-asserted-by":"publisher","first-page":"755","DOI":"10.1093\/scipol\/sct083","volume":"40","author":"D Wright","year":"2013","unstructured":"Wright D, Friedewald M (2013) Integrating privacy and ethical impact assessments. Sci Public Policy 40:755\u2013766. https:\/\/doi.org\/10.1093\/scipol\/sct083","journal-title":"Sci Public Policy"},{"key":"10420_CR117","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1177\/0739456X17723971","volume":"39","author":"Y Xiao","year":"2019","unstructured":"Xiao Y, Watson M (2019) Guidance on conducting a systematic literature review. J Plan Educ Res 39:93\u2013112. https:\/\/doi.org\/10.1177\/0739456X17723971","journal-title":"J Plan Educ Res"},{"key":"10420_CR118","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1109\/TTS.2021.3066209","volume":"2","author":"RV Zicari","year":"2021","unstructured":"Zicari RV, Brodersen J, Brusseau J et al (2021) Z-Inspection\u00ae: a process to assess trustworthy AI. IEEE Trans Technol Soc 2:83\u201397. https:\/\/doi.org\/10.1109\/TTS.2021.3066209","journal-title":"IEEE Trans Technol Soc"},{"key":"10420_CR119","unstructured":"Zuboff PS (2019) The age of surveillance capitalism: the fight for a human future at the new frontier of power, 01 edn. Profile Books"}],"container-title":["Artificial Intelligence Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-023-10420-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10462-023-10420-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-023-10420-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,23]],"date-time":"2023-09-23T03:35:23Z","timestamp":1695440123000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10462-023-10420-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,24]]},"references-count":119,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2023,11]]}},"alternative-id":["10420"],"URL":"https:\/\/doi.org\/10.1007\/s10462-023-10420-8","relation":{},"ISSN":["0269-2821","1573-7462"],"issn-type":[{"value":"0269-2821","type":"print"},{"value":"1573-7462","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,24]]},"assertion":[{"value":"1 February 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 March 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}