{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,3,19]],"date-time":"2025-03-19T13:25:11Z","timestamp":1742390711420,"version":"3.37.3"},"reference-count":79,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,12,17]],"date-time":"2024-12-17T00:00:00Z","timestamp":1734393600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,12,17]],"date-time":"2024-12-17T00:00:00Z","timestamp":1734393600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"publisher","award":["grant no.: 16DII131 \u2013 \u201cWeizenbaum-Institut\u201d"],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Weizenbaum-Institut e.V."}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Comput Soc Sc"],"published-print":{"date-parts":[[2025,2]]},"abstract":"Abstract<\/jats:title>\n This article presents a comparative analysis of the potential of two large language model (LLM)-based chatbots\u2014ChatGPT and Bing Chat (recently rebranded to Microsoft Copilot)\u2014to detect veracity of political information. We use AI auditing methodology to investigate how chatbots evaluate true, false, and borderline statements on five topics: COVID-19, Russian aggression against Ukraine, the Holocaust, climate change, and LGBTQ\u2009+\u2009-related debates. We compare how the chatbots respond in high- and low-resource languages by using prompts in English, Russian, and Ukrainian. Furthermore, we explore chatbots\u2019 ability to evaluate statements according to political communication concepts of disinformation, misinformation, and conspiracy theory, using definition-oriented prompts. We also systematically test how such evaluations are influenced by source attribution. The results show high potential of ChatGPT for the baseline veracity evaluation task, with 72% of the cases evaluated in accordance with the baseline on average across languages without pre-training. Bing Chat evaluated 67% of the cases in accordance with the baseline. We observe significant disparities in how chatbots evaluate prompts in high- and low-resource languages and how they adapt their evaluations to political communication concepts with ChatGPT providing more nuanced outputs than Bing Chat. These findings highlight the potential of LLM-based chatbots in tackling different forms of false information in online environments, but also point to the substantial variation in terms of how such potential is realized due to specific factors (e.g. language of the prompt or the topic).<\/jats:p>","DOI":"10.1007\/s42001-024-00338-8","type":"journal-article","created":{"date-parts":[[2024,12,17]],"date-time":"2024-12-17T06:45:05Z","timestamp":1734417905000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["In generative AI we trust: can chatbots effectively verify political information?"],"prefix":"10.1007","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3614-1804","authenticated-orcid":false,"given":"Elizaveta","family":"Kuznetsova","sequence":"first","affiliation":[]},{"given":"Mykola","family":"Makhortykh","sequence":"additional","affiliation":[]},{"given":"Victoria","family":"Vziatysheva","sequence":"additional","affiliation":[]},{"given":"Martha","family":"Stolze","sequence":"additional","affiliation":[]},{"given":"Ani","family":"Baghumyan","sequence":"additional","affiliation":[]},{"given":"Aleksandra","family":"Urman","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,12,17]]},"reference":[{"issue":"3","key":"338_CR1","doi-asserted-by":"publisher","first-page":"299","DOI":"10.1332\/CQNF2293","volume":"5","author":"JC Aguerri","year":"2022","unstructured":"Aguerri, J. C., & Santisteban, M. (2022). The algorithmic responses to disinformation: A suitable pathway? Justice, Power and Resistance, 5(3), 299\u2013306. https:\/\/doi.org\/10.1332\/CQNF2293","journal-title":"Justice, Power and Resistance"},{"issue":"5","key":"338_CR2","doi-asserted-by":"publisher","first-page":"e19458","DOI":"10.2196\/19458","volume":"22","author":"W Ahmed","year":"2020","unstructured":"Ahmed, W., Vidal-Alaball, J., Downing, J., & Segu\u00ed, F. L. (2020). COVID-19 and the 5G conspiracy theory: Social network analysis of twitter data. Journal of Medical Internet Research, 22(5), e19458. https:\/\/doi.org\/10.2196\/19458","journal-title":"Journal of Medical Internet Research"},{"issue":"1","key":"338_CR3","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1007\/s13278-023-01028-5","volume":"13","author":"E A\u00efmeur","year":"2023","unstructured":"A\u00efmeur, E., Amri, S., & Brassard, G. (2023). Fake news, disinformation and misinformation in social media: A review. Social Network Analysis and Mining, 13(1), 30. https:\/\/doi.org\/10.1007\/s13278-023-01028-5","journal-title":"Social Network Analysis and Mining"},{"key":"338_CR4","doi-asserted-by":"publisher","DOI":"10.2861\/368879","volume-title":"Automated tackling of disinformation: Major challenges ahead","author":"A Alaphilippe","year":"2019","unstructured":"Alaphilippe, A., Gizikis, A., Hanot, C., & Bontcheva, K. (2019). Automated tackling of disinformation: Major challenges ahead. European Parliament: Directorate General for Parliamentary Research Services. https:\/\/doi.org\/10.2861\/368879"},{"key":"338_CR5","doi-asserted-by":"publisher","first-page":"1770","DOI":"10.1109\/BigData55660.2022.10020223","volume":"2022","author":"I Alieva","year":"2022","unstructured":"Alieva, I., Ng, L. H. X., & Carley, K. M. (2022). Investigating the spread of Russian disinformation about Biolabs in Ukraine on twitter using social network analysis. IEEE International Conference on Big Data (Big Data), 2022, 1770\u20131775. https:\/\/doi.org\/10.1109\/BigData55660.2022.10020223","journal-title":"IEEE International Conference on Big Data (Big Data)"},{"issue":"36","key":"338_CR6","doi-asserted-by":"publisher","first-page":"abf4393","DOI":"10.1126\/sciadv.abf4393","volume":"7","author":"J Allen","year":"2021","unstructured":"Allen, J., Arechar, A. A., Pennycook, G., & Rand, D. G. (2021). Scaling up fact-checking using the wisdom of crowds. Science Advances, 7(36), abf4393. https:\/\/doi.org\/10.1126\/sciadv.abf4393","journal-title":"Science Advances"},{"issue":"9","key":"338_CR7","doi-asserted-by":"publisher","first-page":"1502","DOI":"10.1038\/s41562-023-01641-6","volume":"7","author":"AA Arechar","year":"2023","unstructured":"Arechar, A. A., Allen, J., Berinsky, A. J., Cole, R., Epstein, Z., Garimella, K., Gully, A., Lu, J. G., Ross, R. M., Stagnaro, M. N., Zhang, Y., Pennycook, G., & Rand, D. G. (2023). Understanding and combatting misinformation across 16 countries on six continents. Nature Human Behaviour, 7(9), 1502\u20131513. https:\/\/doi.org\/10.1038\/s41562-023-01641-6","journal-title":"Nature Human Behaviour"},{"key":"338_CR8","doi-asserted-by":"publisher","first-page":"1331","DOI":"10.1007\/s10796-021-10133-9","volume":"24","author":"CH Au","year":"2022","unstructured":"Au, C. H., Ho, K. K. W., & Chiu, D. K. (2022). The role of online misinformation and fake news in ideological polarization: Barriers, catalysts, and implications. Information Systems Frontiers, 24, 1331\u20131354. https:\/\/doi.org\/10.1007\/s10796-021-10133-9","journal-title":"Information Systems Frontiers"},{"issue":"1","key":"338_CR9","doi-asserted-by":"publisher","first-page":"52","DOI":"10.61969\/jai.1337500","volume":"7","author":"D Baidoo-anu","year":"2023","unstructured":"Baidoo-anu, D., & Owusu Ansah, L. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI, 7(1), 52\u201356.","journal-title":"Journal of AI"},{"key":"338_CR10","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1109\/CogSIMA51574.2021.9475929","volume":"2021","author":"W Bailer","year":"2021","unstructured":"Bailer, W., Thallinger, G., Backfried, G., & Thomas-Aniola, D. (2021). Challenges for automatic detection of fake news related to migration: Invited paper. IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA), 2021, 133\u2013138. https:\/\/doi.org\/10.1109\/CogSIMA51574.2021.9475929","journal-title":"IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA)"},{"key":"338_CR11","unstructured":"Bandy, J. (2021). Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits (arXiv:2102.04256). arXiv. http:\/\/arxiv.org\/abs\/2102.04256"},{"issue":"11","key":"338_CR12","doi-asserted-by":"publisher","first-page":"6943","DOI":"10.3390\/su14116943","volume":"14","author":"A Barchetti","year":"2022","unstructured":"Barchetti, A., Neybert, E., Mantel, S. P., & Kardes, F. R. (2022). The half-truth effect and its implications for sustainability. Sustainability, 14(11), 6943. https:\/\/doi.org\/10.3390\/su14116943","journal-title":"Sustainability"},{"issue":"3","key":"338_CR13","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1108\/IJCMA-02-2019-0032","volume":"30","author":"M Bastian","year":"2019","unstructured":"Bastian, M., Makhortykh, M., & Dobber, T. (2019). News personalization for peace: How algorithmic recommendations can impact conflict coverage. International Journal of Conflict Management, 30(3), 309\u2013328. https:\/\/doi.org\/10.1108\/IJCMA-02-2019-0032","journal-title":"International Journal of Conflict Management"},{"key":"338_CR14","doi-asserted-by":"publisher","first-page":"612","DOI":"10.1109\/SaTML59370.2024.00037","volume":"2024","author":"A Birhane","year":"2024","unstructured":"Birhane, A., Steed, R., Ojewale, V., Vecchione, B., & Raji, I. D. (2024). AI auditing: The broken bus on the road to AI accountability. IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 2024, 612\u2013643.","journal-title":"IEEE Conference on Secure and Trustworthy Machine Learning (SaTML)"},{"issue":"1","key":"338_CR15","doi-asserted-by":"publisher","first-page":"19","DOI":"10.3917\/pdc.006.0019","volume":"6","author":"JG Blumler","year":"2016","unstructured":"Blumler, J. G. (2016). The fourth age of political communication: Politiques de. Communication N\u00b0, 6(1), 19\u201330. https:\/\/doi.org\/10.3917\/pdc.006.0019","journal-title":"Communication N\u00b0"},{"issue":"2","key":"338_CR16","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1108\/OIR-11-2018-0341","volume":"44","author":"M Bonart","year":"2019","unstructured":"Bonart, M., Samokhina, A., Heisenberg, G., & Schaer, P. (2019). An investigation of biases in web search engine query suggestions. Online Information Review, 44(2), 365\u2013381. https:\/\/doi.org\/10.1108\/OIR-11-2018-0341","journal-title":"Online Information Review"},{"issue":"4","key":"338_CR17","doi-asserted-by":"publisher","first-page":"660","DOI":"10.1080\/10584609.2019.1670899","volume":"36","author":"S Boulianne","year":"2019","unstructured":"Boulianne, S. (2019). US dominance of research on political communication: A meta-view. Political Communication, 36(4), 660\u2013665. https:\/\/doi.org\/10.1080\/10584609.2019.1670899","journal-title":"Political Communication"},{"key":"338_CR18","unstructured":"Bountouridis, D., Makhortykh, M., Sullivan, E., Harambam, J., Tintarev, N., & Hauff, C. (2019, July). Annotating credibility: Identifying and mitigating bias in credibility datasets. ROME 2019: Workshop on Reducing Online Misinformation Exposure, Paris France. https:\/\/rome2019.github.io\/papers\/Bountouridis_etal_ROME2019.pdf"},{"issue":"3","key":"338_CR19","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1007\/s10676-013-9321-6","volume":"15","author":"E Bozdag","year":"2013","unstructured":"Bozdag, E. (2013). Bias in algorithmic filtering and personalization. Ethics and Information Technology, 15(3), 209\u2013227. https:\/\/doi.org\/10.1007\/s10676-013-9321-6","journal-title":"Ethics and Information Technology"},{"issue":"4","key":"338_CR20","doi-asserted-by":"publisher","first-page":"1442","DOI":"10.14763\/2019.4.1442","volume":"8","author":"S Bradshaw","year":"2019","unstructured":"Bradshaw, S. (2019). Disinformation optimised: Gaming search engine algorithms to amplify junk news. Internet Policy Review, 8(4), 1442. https:\/\/doi.org\/10.14763\/2019.4.1442","journal-title":"Internet Policy Review"},{"key":"338_CR21","doi-asserted-by":"publisher","unstructured":"Caramancion, K. M. (2023). News Verifiers Showdown: A Comparative Performance Evaluation of ChatGPT 3.5, ChatGPT 4.0, Bing AI, and Bard in News Fact-Checking (arXiv:2306.17176). arXiv. https:\/\/doi.org\/10.48550\/arXiv.2306.17176","DOI":"10.48550\/arXiv.2306.17176"},{"key":"338_CR22","unstructured":"Conspiracy Theory. (2023). In Cambridge Dictionary. Cambridge University Press & Assessment. https:\/\/dictionary.cambridge.org\/dictionary\/english\/conspiracy-theory"},{"issue":"7","key":"338_CR23","doi-asserted-by":"publisher","first-page":"1217","DOI":"10.3390\/vaccines11071217","volume":"11","author":"G Deiana","year":"2023","unstructured":"Deiana, G., Dettori, M., Arghittu, A., Azara, A., Gabutti, G., & Castiglia, P. (2023). Artificial intelligence and public health: Evaluating ChatGPT responses to vaccination myths and misconceptions. Vaccines, 11(7), 1217. https:\/\/doi.org\/10.3390\/vaccines11071217","journal-title":"Vaccines"},{"key":"338_CR24","doi-asserted-by":"publisher","first-page":"102642","DOI":"10.1016\/j.ijinfomgt.2023.102642","volume":"71","author":"YK Dwivedi","year":"2023","unstructured":"Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., & Wright, R. (2023). Opinion Paper: \u201cSo what if ChatGPT wrote it?\u201d Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. https:\/\/doi.org\/10.1016\/j.ijinfomgt.2023.102642","journal-title":"International Journal of Information Management"},{"issue":"7","key":"338_CR25","doi-asserted-by":"publisher","first-page":"566","DOI":"10.1038\/s42256-021-00370-7","volume":"3","author":"G Falco","year":"2021","unstructured":"Falco, G., Shneiderman, B., Badger, J., Carrier, R., Dahbura, A., Danks, D., Eling, M., Goodloe, A., Gupta, J., Hart, C., Jirotka, M., Johnson, H., LaPointe, C., Llorens, A. J., Mackworth, A. K., Maple, C., P\u00e1lsson, S. E., Pasquale, F., Winfield, A., & Yeong, Z. K. (2021). Governing AI safety through independent audits. Nature Machine Intelligence, 3(7), 566\u2013571. https:\/\/doi.org\/10.1038\/s42256-021-00370-7","journal-title":"Nature Machine Intelligence"},{"issue":"3","key":"338_CR26","doi-asserted-by":"publisher","first-page":"298","DOI":"10.1080\/13183222.2018.1463047","volume":"25","author":"J Farkas","year":"2018","unstructured":"Farkas, J., & Schou, J. (2018). Fake news as a floating signifier: hegemony, antagonism and the politics of falsehood. Javnost - The Public, 25(3), 298\u2013314. https:\/\/doi.org\/10.1080\/13183222.2018.1463047","journal-title":"Javnost - The Public"},{"key":"338_CR27","doi-asserted-by":"publisher","unstructured":"Freelon, D., & Lokot, T. (2020). Russian disinformation campaigns on Twitter target political communities across the spectrum. Collaboration between opposed political groups might be the most effective way to counter it. Misinformation Review. https:\/\/doi.org\/10.37016\/mr-2020-003","DOI":"10.37016\/mr-2020-003"},{"key":"338_CR28","first-page":"181","volume":"49","author":"JM Garon","year":"2022","unstructured":"Garon, J. M. (2022). When AI goes to war: corporate accountability for virtual mass disinformation, algorithmic atrocities, and synthetic propaganda. N Ky L Rev, 49, 181.","journal-title":"N Ky L Rev"},{"key":"338_CR29","doi-asserted-by":"crossref","unstructured":"Ghosh, S., & Caliskan, A. (2023). ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languages. arXiv preprint arXiv:2305.10510","DOI":"10.1145\/3600211.3604672"},{"issue":"30","key":"338_CR30","doi-asserted-by":"publisher","first-page":"e2305016120","DOI":"10.1073\/pnas.2305016120","volume":"120","author":"F Gilardi","year":"2023","unstructured":"Gilardi, F., Alizadeh, M., & Kubil, M. (2023). ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences of the United States of America, 120(30), e2305016120. https:\/\/doi.org\/10.1073\/pnas.2305016120","journal-title":"Proceedings of the National Academy of Sciences of the United States of America"},{"key":"338_CR31","unstructured":"Google. (2023, March 14). Generative AI Prohibited Use Policy. https:\/\/policies.google.com\/terms\/generative-ai\/use-policy"},{"key":"338_CR32","doi-asserted-by":"publisher","first-page":"6","DOI":"10.37307\/j.2196-9817.2019.01.06","volume":"1","author":"H-C Gr\u00e4fe","year":"2018","unstructured":"Gr\u00e4fe, H.-C. (2018). Webtracking und Microtargeting als Gefahr f\u00fcr Demokratie und Medien. PinG Privacy in Germany, 1, 6. https:\/\/doi.org\/10.37307\/j.2196-9817.2019.01.06","journal-title":"PinG Privacy in Germany"},{"key":"338_CR33","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1017\/9781108890960.003","volume-title":"Social Media and Democracy: The State of the Field, Prospects for Reform","author":"AM Guess","year":"2020","unstructured":"Guess, A. M., & Lyons, B. A. (2020). Misinformation, Disinformation, and Online Propaganda. In J. A. Tucker & N. Persily (Eds.), Social Media and Democracy: The State of the Field, Prospects for Reform (pp. 10\u201333). Cambridge University Press."},{"key":"338_CR34","unstructured":"Guhl, J., & Davey, J. (2020). Hosting the \u2018holohoax\u2019: A snapshot of holocaust denial across social media. The Institute for Strategic Dialogue. https:\/\/www.isdglobal.org\/wp-content\/uploads\/2020\/08\/Hosting-the-Holohoax.pdf"},{"issue":"11","key":"338_CR35","doi-asserted-by":"publisher","first-page":"1596","DOI":"10.1080\/1369118X.2021.1874038","volume":"25","author":"M Hameleers","year":"2022","unstructured":"Hameleers, M., van der Meer, T., & Vliegenhart, R. (2022). Civilized truths, hateful lies? Incivility and hate speech in false information \u2013 evidence from fact-checked statements in the US. Information, Communication & Society, 25(11), 1596\u20131613. https:\/\/doi.org\/10.1080\/1369118X.2021.1874038","journal-title":"Information, Communication & Society"},{"issue":"4","key":"338_CR36","doi-asserted-by":"publisher","first-page":"466","DOI":"10.1177\/0963662514559891","volume":"24","author":"J Harambam","year":"2015","unstructured":"Harambam, J., & Aupers, S. (2015). Contesting epistemic authority: Conspiracy theories on the boundaries of science. Public Understanding of Science, 24(4), 466\u2013480. https:\/\/doi.org\/10.1177\/0963662514559891","journal-title":"Public Understanding of Science"},{"key":"338_CR37","first-page":"20","volume":"15","author":"A Hinsley","year":"2021","unstructured":"Hinsley, A., & Holton, A. (2021). Fake news cues: examining the impact of content, source, and typology of news cues on People\u2019s confidence in identifying Mis-and disinformation. International Journal of Communication, 15, 20.","journal-title":"International Journal of Communication"},{"key":"338_CR38","doi-asserted-by":"publisher","unstructured":"Hoes, E., Altay, S., & Bermeo, J. (2023). Leveraging ChatGPT for Efficient Fact-Checking. PsyArXiv. https:\/\/doi.org\/10.31234\/osf.io\/qnjkf","DOI":"10.31234\/osf.io\/qnjkf"},{"key":"338_CR39","unstructured":"Holan, A. (2024). The Principles of the Truth-O-Meter: PolitiFact\u2019s methodology for independent fact-checking. https:\/\/ www.politifact.com\/truth-o-meter\/article\/2018\/feb\/12\/principles-truth-o-meter-politifactsmethodology-i\/"},{"issue":"S3","key":"338_CR40","doi-asserted-by":"publisher","first-page":"S331","DOI":"10.2105\/AJPH.2020.305940","volume":"110","author":"A Jamison","year":"2020","unstructured":"Jamison, A., Broniatowski, D. A., Smith, M. C., Parikh, K. S., Malik, A., Dredze, M., & Quinn, S. C. (2020). Adapting and extending a typology to identify vaccine misinformation on Twitter. American Journal of Public Health, 110(S3), S331\u2013S339. https:\/\/doi.org\/10.2105\/AJPH.2020.305940","journal-title":"American Journal of Public Health"},{"issue":"12","key":"338_CR41","doi-asserted-by":"publisher","first-page":"85","DOI":"10.6918\/IJOSSER.202012_3(12).0011","volume":"3","author":"F Jia","year":"2020","unstructured":"Jia, F. (2020). Misinformation literature review: definitions, taxonomy, and models. International Journal of Social Science and Education Research, 3(12), 85\u201390. https:\/\/doi.org\/10.6918\/IJOSSER.202012_3(12).0011","journal-title":"International Journal of Social Science and Education Research"},{"issue":"5","key":"338_CR42","doi-asserted-by":"publisher","first-page":"1301","DOI":"10.1177\/1461444820959296","volume":"23","author":"E Kapantai","year":"2021","unstructured":"Kapantai, E., Christopoulou, A., Berberidis, C., & Peristeras, V. (2021). A systematic literature review on disinformation: Toward a unified taxonomical framework. New Media & Society, 23(5), 1301\u20131326. https:\/\/doi.org\/10.1177\/1461444820959296","journal-title":"New Media & Society"},{"key":"338_CR43","doi-asserted-by":"publisher","first-page":"102274","DOI":"10.1016\/j.lindif.2023.102274","volume":"103","author":"E Kasneci","year":"2023","unstructured":"Kasneci, E., Sessler, K., K\u00fcchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., G\u00fcnnemann, S., H\u00fcllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https:\/\/doi.org\/10.1016\/j.lindif.2023.102274","journal-title":"Learning and Individual Differences"},{"key":"338_CR44","doi-asserted-by":"publisher","unstructured":"Kotek, H., Dockum, R., & Sun, D. Q. (2023). Gender bias and stereotypes in Large Language Models. In Proceedings of The ACM Collective Intelligence Conference, 12\u201324. https:\/\/doi.org\/10.1145\/3582269.3615599","DOI":"10.1145\/3582269.3615599"},{"key":"338_CR45","first-page":"971","volume":"17","author":"E Kuznetsova","year":"2023","unstructured":"Kuznetsova, E., & Makhortykh, M. (2023). Blame it on the algorithm? Russian government-sponsored media and algorithmic curation of political information on facebook. International Journal of Communication, 17, 971\u2013992.","journal-title":"International Journal of Communication"},{"issue":"2021","key":"338_CR46","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1146\/annurev-publhealth-090419-102409","volume":"42","author":"S Lewandowsky","year":"2021","unstructured":"Lewandowsky, S. (2021). Climate change disinformation and how to combat it. Annual Review of Public Health, 42(2021), 1\u201321. https:\/\/doi.org\/10.1146\/annurev-publhealth-090419-102409","journal-title":"Annual Review of Public Health"},{"key":"338_CR47","unstructured":"Lewandowsky, S., & Cook, J. (2020). The Conspiracy Theory Handbook. Copyright, Fair Use, Scholarly Communication. https:\/\/skepticalscience.com\/docs\/ConspiracyTheoryHandbook.pdf"},{"key":"338_CR48","doi-asserted-by":"publisher","unstructured":"Litvinenko, A. (2023). Propaganda on demand: Russia\u2019s media environment during the war in Ukraine. Global Media Journal - German Edition, 12, no. 2. https:\/\/doi.org\/10.22032\/DBT.55518","DOI":"10.22032\/DBT.55518"},{"key":"338_CR49","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2021.103654","volume":"304","author":"R Liu","year":"2022","unstructured":"Liu, R., Jia, C., Wei, J., Xu, G., & Vosoughi, S. (2022). Quantifying and alleviating political bias in language models. Artificial Intelligence, 304, 103654. https:\/\/doi.org\/10.1016\/j.artint.2021.103654","journal-title":"Artificial Intelligence"},{"issue":"3","key":"338_CR50","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1108\/LHTN-01-2023-0009","volume":"40","author":"BD Lund","year":"2023","unstructured":"Lund, B. D., & Wang, T. (2023). Chatting about ChatGPT: How may AI and GPT impact academia and libraries? Library Hi Tech News, 40(3), 26\u201329. https:\/\/doi.org\/10.1108\/LHTN-01-2023-0009","journal-title":"Library Hi Tech News"},{"issue":"1","key":"338_CR51","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1037\/xap0000315","volume":"27","author":"R Maertens","year":"2021","unstructured":"Maertens, R., Roozenbeek, J., Basol, M., & van der Linden, S. (2021). Long-term effectiveness of inoculation against misinformation: Three longitudinal experiments. Journal of Experimental Psychology: Applied, 27(1), 1\u201316. https:\/\/doi.org\/10.1037\/xap0000315","journal-title":"Journal of Experimental Psychology: Applied"},{"issue":"1","key":"338_CR52","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1007\/s44163-023-00072-6","volume":"3","author":"M Makhortykh","year":"2023","unstructured":"Makhortykh, M., Zucker, E. M., Simon, D. J., Bultmann, D., & Ulloa, R. (2023). Shall androids dream of genocides? How generative AI can change the future of memorialization of mass atrocities. Discover Artificial Intelligence, 3(1), 28. https:\/\/doi.org\/10.1007\/s44163-023-00072-6","journal-title":"Discover Artificial Intelligence"},{"issue":"1","key":"338_CR53","doi-asserted-by":"publisher","first-page":"120","DOI":"10.1038\/s41746-023-00873-0","volume":"6","author":"B Mesk\u00f3","year":"2023","unstructured":"Mesk\u00f3, B., & Topol, E. J. (2023). The imperative for regulatory oversight of large language models (or generative AI) in healthcare. Npj Digital Medicine, 6(1), 120. https:\/\/doi.org\/10.1038\/s41746-023-00873-0","journal-title":"Npj Digital Medicine"},{"issue":"2016","key":"338_CR54","first-page":"4991","volume":"10","author":"B Mittelstadt","year":"2016","unstructured":"Mittelstadt, B. (2016). Automation, algorithms, and politics| auditing for transparency in content personalization systems. International Journal of Communication, 10(2016), 4991\u20135002.","journal-title":"International Journal of Communication"},{"key":"338_CR55","unstructured":"Moskvichev, A., Odouard, V. V., & Mitchell, M. (2023). The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domain. Arxiv.org. https:\/\/arxiv.org\/abs\/2305.07141"},{"issue":"1","key":"338_CR56","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/s11127-023-01097-2","volume":"198","author":"F Motoki","year":"2024","unstructured":"Motoki, F., Pinho Neto, V., & Rodrigues, V. (2023). More human than human: Measuring ChatGPT political bias. Public Choice, 198(1), 3\u201323. https:\/\/doi.org\/10.1007\/s11127-023-01097-2","journal-title":"Public Choice"},{"key":"338_CR57","unstructured":"Mumtarin, M., Chowdhury, M. S., & Wood, J. (2023). Large Language Models in Analyzing Crash Narratives\u2014A Comparative Study of ChatGPT, BARD and GPT-4. arXiv:2308.13563"},{"key":"338_CR58","unstructured":"Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., & Mian, A. (2023). A comprehensive overview of large language models. arXiv preprint arXiv:2307.06435"},{"key":"338_CR59","unstructured":"Nilsson, N. J. (1998). Artificial intelligence: A new synthesis. Morgan Kaufman."},{"key":"338_CR60","doi-asserted-by":"publisher","unstructured":"Rader, E., & Gray, R. (2015). Understanding User Beliefs About Algorithmic Curation in the Facebook News Feed. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 173\u2013182. https:\/\/doi.org\/10.1145\/2702123.2702174","DOI":"10.1145\/2702123.2702174"},{"issue":"10","key":"338_CR61","doi-asserted-by":"publisher","DOI":"10.1098\/rsos.201199","volume":"7","author":"J Roozenbeek","year":"2020","unstructured":"Roozenbeek, J., Schneider, C. R., Dryhurst, S., Kerr, J., Freeman, A. L. J., Recchia, G., van der Bles, A. M., & van der Linden, S. (2020). Susceptibility to misinformation about COVID-19 around the world. Royal Society Open Science, 7(10), 201199. https:\/\/doi.org\/10.1098\/rsos.201199","journal-title":"Royal Society Open Science"},{"key":"338_CR62","doi-asserted-by":"publisher","unstructured":"R\u00f6ttger, P., Hofmann, V., Pyatkin, V., Hinck, M., Kirk, H. R., Sch\u00fctze, H., & Hovy, D. (2024). Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models (arXiv:2402.16786). arXiv. https:\/\/doi.org\/10.48550\/arXiv.2402.16786","DOI":"10.48550\/arXiv.2402.16786"},{"key":"338_CR63","doi-asserted-by":"publisher","unstructured":"Rozado, D. (2024). The Political Preferences of LLMs (arXiv:2402.01789). arXiv. https:\/\/doi.org\/10.48550\/arXiv.2402.01789","DOI":"10.48550\/arXiv.2402.01789"},{"issue":"2","key":"338_CR64","doi-asserted-by":"publisher","DOI":"10.1016\/j.lisr.2023.101237","volume":"45","author":"H Ruokolainen","year":"2023","unstructured":"Ruokolainen, H., Wid\u00e9n, G., & Eskola, E.-L. (2023). How and why does official information become misinformation? A typology of official misinformation. Library & Information Science Research, 45(2), 101237. https:\/\/doi.org\/10.1016\/j.lisr.2023.101237","journal-title":"Library & Information Science Research"},{"issue":"1","key":"338_CR65","first-page":"7115633","volume":"2024","author":"J Rutinowski","year":"2024","unstructured":"Rutinowski, J., Franke, S., Endendyk, J., Dormuth, I., Roidl, M., & Pauly, M. (2024). The self-perception and political biases of ChatGPT. Human Behavior and Emerging Technologies, 2024(1), 7115633.","journal-title":"Human Behavior and Emerging Technologies"},{"issue":"1","key":"338_CR66","doi-asserted-by":"publisher","first-page":"138","DOI":"10.1080\/14693062.2023.2245792","volume":"24","author":"SA Samoilenko","year":"2024","unstructured":"Samoilenko, S. A., & Cook, J. (2024). Developing an Ad Hominem typology for classifying climate misinformation. Climate Policy, 24(1), 138\u2013151. https:\/\/doi.org\/10.1080\/14693062.2023.2245792","journal-title":"Climate Policy"},{"issue":"6","key":"338_CR67","doi-asserted-by":"publisher","first-page":"820","DOI":"10.1080\/21670811.2020.1765401","volume":"8","author":"F Saurwein","year":"2020","unstructured":"Saurwein, F., & Spencer-Smith, C. (2020). Combating disinformation on social media: multilevel governance and distributed accountability in Europe. Digital Journalism, 8(6), 820\u2013841. https:\/\/doi.org\/10.1080\/21670811.2020.1765401","journal-title":"Digital Journalism"},{"issue":"6","key":"338_CR68","doi-asserted-by":"publisher","first-page":"5929","DOI":"10.1007\/s11229-019-02444-x","volume":"198","author":"SO S\u00f8e","year":"2021","unstructured":"S\u00f8e, S. O. (2021). A unified account of information, misinformation, and disinformation. Synthese, 198(6), 5929\u20135949. https:\/\/doi.org\/10.1007\/s11229-019-02444-x","journal-title":"Synthese"},{"issue":"26","key":"338_CR69","doi-asserted-by":"publisher","first-page":"eadh1850","DOI":"10.1126\/sciadv.adh1850","volume":"9","author":"G Spitale","year":"2023","unstructured":"Spitale, G., Biller-Andorno, N., & Germani, F. (2023). AI model GPT-3 (dis) informs us better than humans. Science Advances, 9(26), eadh1850. https:\/\/doi.org\/10.1126\/sciadv.adh1850","journal-title":"Science Advances"},{"key":"338_CR70","unstructured":"Strand, C., & Svensson, J. (2021). Disinformation campaigns about LGBTI+ people in the EU and foreign influence (pp. 1\u201328) [Briefing]. European Parlament, Policy Department for External Relations. https:\/\/dspace.ceid.org.tr\/xmlui\/bitstream\/handle\/1\/1805\/QA0921283ENN.en.pdf?sequence=1&isAllowed=y"},{"key":"338_CR71","unstructured":"Tao, Y., Agrawal, A., Dombi, J., Sydorenko, T., & Lee, J. I. (2024). ChatGPT Role-play Dataset: Analysis of User Motives and Model Naturalness. https:\/\/arxiv.org\/abs\/2403.18121"},{"key":"338_CR72","unstructured":"T\u00f6rnberg, P., Valeeva, D., Uitermark, J., & Bail, C. (2023). Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms. arXiv preprint arXiv:2310.05984"},{"issue":"8","key":"338_CR73","doi-asserted-by":"publisher","first-page":"1850","DOI":"10.1002\/asi.23820","volume":"68","author":"J Unkel","year":"2017","unstructured":"Unkel, J., & Haas, A. (2017). The effects of credibility cues on the selection of search engine results. Journal of the Association for Information Science and Technology, 68(8), 1850\u20131862. https:\/\/doi.org\/10.1002\/asi.23820","journal-title":"Journal of the Association for Information Science and Technology"},{"key":"338_CR74","doi-asserted-by":"publisher","unstructured":"Urman, A., & Makhortykh, M. (2023). The Silence of the LLMs: Cross-Lingual Analysis of Political Bias and False Information Prevalence in ChatGPT, Google Bard, and Bing Chat. https:\/\/doi.org\/10.31219\/osf.io\/q9v8f","DOI":"10.31219\/osf.io\/q9v8f"},{"key":"338_CR75","doi-asserted-by":"publisher","unstructured":"Vidgen, B., Scherrer, N., Kirk, H. R., Qian, R., Kannappan, A., Hale, S. A., & R\u00f6ttger, P. (2023). SimpleSafetyTests: A Test Suite for Identifying Critical Safety Risks in Large Language Models. https:\/\/doi.org\/10.48550\/ARXIV.2311.08370","DOI":"10.48550\/ARXIV.2311.08370"},{"key":"338_CR76","doi-asserted-by":"publisher","unstructured":"Wan, Y., Pu, G., Sun, J., & Garimella, A. (2023). \u201cKelly is a Warm Person, Joseph is a Role Model\u201d: Gender Biases in LLM-Generated Reference Letters. https:\/\/doi.org\/10.48550\/arXiv.2310.09219","DOI":"10.48550\/arXiv.2310.09219"},{"key":"338_CR77","unstructured":"Wardle, C., & Derakshan, H. (2017). Information disorder: Toward an interdisciplinary framework for research and policy making. Council of Europe. https:\/\/rm.coe.int\/information-disorder-toward-an-interdisciplinary-framewor k-for-researc\/168076277c"},{"issue":"4","key":"338_CR78","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1353\/jod.2023.a907695","volume":"34","author":"E Yang","year":"2023","unstructured":"Yang, E., & Roberts, M. E. (2023). The authoritarian data problem. Journal of Democracy, 34(4), 141\u2013150. https:\/\/doi.org\/10.1353\/jod.2023.a907695","journal-title":"Journal of Democracy"},{"key":"338_CR79","unstructured":"Zheng, S. (2023). China\u2019s Answers to ChatGPT Have a Censorship Problem. Bloomberg. https:\/\/www.bloomberg.com\/news\/newsletters\/2023-05-02\/china-s-chatgpt-answers-raise-questions-about-censoring-generative-ai"}],"container-title":["Journal of Computational Social Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s42001-024-00338-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s42001-024-00338-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s42001-024-00338-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,15]],"date-time":"2025-02-15T05:29:25Z","timestamp":1739597365000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s42001-024-00338-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,17]]},"references-count":79,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,2]]}},"alternative-id":["338"],"URL":"https:\/\/doi.org\/10.1007\/s42001-024-00338-8","relation":{},"ISSN":["2432-2717","2432-2725"],"issn-type":[{"type":"print","value":"2432-2717"},{"type":"electronic","value":"2432-2725"}],"subject":[],"published":{"date-parts":[[2024,12,17]]},"assertion":[{"value":"21 April 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 September 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 December 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"On behalf of all authors, the corresponding author states that there is no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"15"}}