{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,5,25]],"date-time":"2024-05-25T00:30:21Z","timestamp":1716597021070},"reference-count":128,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,5,24]],"date-time":"2024-05-24T00:00:00Z","timestamp":1716508800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"Asking annotators to explain \u201cwhy\u201d they labeled an instance yields annotator rationales: natural language explanations that provide reasons for classifications. In this work, we survey the collection and use of annotator rationales. Human-annotated rationales can improve data quality and form a valuable resource for improving machine learning models. Moreover, human-annotated rationales can inspire the construction and evaluation of model-annotated rationales, which can play an important role in explainable artificial intelligence.<\/jats:p>","DOI":"10.3389\/frai.2024.1260952","type":"journal-article","created":{"date-parts":[[2024,5,24]],"date-time":"2024-05-24T04:55:38Z","timestamp":1716526538000},"update-policy":"http:\/\/dx.doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Human-annotated rationales and explainable text classification: a survey"],"prefix":"10.3389","volume":"7","author":[{"given":"Elize","family":"Herrewijnen","sequence":"first","affiliation":[]},{"given":"Dong","family":"Nguyen","sequence":"additional","affiliation":[]},{"given":"Floris","family":"Bex","sequence":"additional","affiliation":[]},{"given":"Kees","family":"van Deemter","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2024,5,24]]},"reference":[{"key":"B1","first-page":"1758","article-title":"\u201cLearning cause identifiers from annotator rationales,\u201d","volume-title":"Twenty-Second International Joint Conference on Artificial Intelligence","author":"Abedin","year":"2011"},{"key":"B2","first-page":"55","article-title":"\u201cInteractive annotation learning with indirect feature voting,\u201d","volume-title":"Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Student Research Workshop and Doctoral Consortium","author":"Arora","year":"2009"},{"key":"B3","first-page":"5868","article-title":"\u201cMARTA: leveraging human rationales for explainable text classification,\u201d","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Arous","year":"2021"},{"key":"B4","doi-asserted-by":"crossref","first-page":"2911","DOI":"10.18653\/v1\/D19-1289","article-title":"\u201cWhat gets echoed? Understanding the \u201cpointers\u201d in explanations of persuasive arguments,\u201d","author":"Atkinson","year":"2019","journal-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)"},{"key":"B5","doi-asserted-by":"crossref","first-page":"1903","DOI":"10.18653\/v1\/D18-1216","article-title":"\u201cDeriving machine attention from human rationales,\u201d","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Bao","year":"2018"},{"key":"B6","doi-asserted-by":"crossref","first-page":"2963","DOI":"10.18653\/v1\/P19-1284","article-title":"\u201cInterpretable neural predictions with differentiable binary variables,\u201d","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Bastings","year":"2019"},{"key":"B7","doi-asserted-by":"crossref","first-page":"3889","DOI":"10.18653\/v1\/2022.acl-long.269","article-title":"\u201cIs attention explanation? An introduction to the debate,\u201d","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Bibal","year":"2022"},{"key":"B8","author":"Bujel","year":"2023","journal-title":"Finding the Needle in a Haystack: Unsupervised Rationale Extraction from Long Text Classifiers. arXiv [Preprint]. arXiv"},{"key":"B9","doi-asserted-by":"publisher","first-page":"103111","DOI":"10.1016\/j.ipm.2022.103111","article-title":"A survey on XAI and natural language explanations","volume":"60","author":"Cambria","year":"2023","journal-title":"Inform. Proc. Manage"},{"key":"B10","first-page":"9539","article-title":"\u201ce-SNLI: natural language inference with natural language explanations,\u201d","volume-title":"Advances in Neural Information Processing Systems 31","author":"Camburu","year":"2018"},{"key":"B11","first-page":"1075","article-title":"\u201cWhat to learn, and how: toward effective learning from rationales,\u201d","author":"Carton","year":"2022","journal-title":"of the Association for Computational Linguistics: ACL 2022"},{"key":"B12","doi-asserted-by":"crossref","first-page":"3497","DOI":"10.18653\/v1\/D18-1386","article-title":"\u201cExtractive adversarial networks: high-recall explanations for identifying personal attacks in social media posts,\u201d","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Carton","year":"2018"},{"key":"B13","doi-asserted-by":"crossref","first-page":"9294","DOI":"10.18653\/v1\/2020.emnlp-main.747","article-title":"\u201cEvaluating and characterizing human rationales,\u201d","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Carton","year":"2020"},{"key":"B14","doi-asserted-by":"publisher","first-page":"832","DOI":"10.3390\/electronics8080832","article-title":"Machine learning interpretability: a survey on methods and metrics","volume":"8","author":"Carvalho","year":"2019","journal-title":"Electronics"},{"key":"B15","first-page":"226","article-title":"\u201cParagraph-level rationale extraction through regularization: a case study on european court of human rights cases,\u201d","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Chalkidis","year":"2021"},{"key":"B16","first-page":"1448","article-title":"\u201cInvariant rationalization,\u201d","volume-title":"International Conference on Machine Learning","author":"Chang","year":"2020"},{"key":"B17","first-page":"3792","article-title":"\u201cCan rationalization improve robustness?,\u201d","volume-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Chen","year":"2022"},{"key":"B18","first-page":"1905","article-title":"\u201cExplainable text classification in legal document review a case study of explainable predictive coding,\u201d","volume-title":"IEEE International Conference on Big Data (IEEE BigData 2018)","author":"Chhatwal","year":"2018"},{"key":"B19","first-page":"25","article-title":"\u201cRe-examining human annotations for interpretable NLP,\u201d","author":"Chiang","year":"2022","journal-title":"Explainable Agency in Artificial Intelligence Workshop"},{"key":"B20","doi-asserted-by":"crossref","first-page":"6920","DOI":"10.18653\/v1\/2022.acl-long.477","article-title":"\u201cAn empirical study on explanations in out-of-domain settings,\u201d","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Chrysostomou","year":"2022"},{"key":"B21","first-page":"2376","article-title":"\u201cA study of automatic metrics for the evaluation of natural language explanations,\u201d","volume-title":"Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume","author":"Clinciu","year":"2021"},{"key":"B22","first-page":"19","article-title":"\u201cTowards explainable text classification by jointly learning lexicon and modifier terms,\u201d","volume-title":"IJCAI-17 Workshop on Explainable AI (XAI)","author":"Clos","year":"2017"},{"key":"B23","first-page":"1","article-title":"\u201cAutomatic generation of natural language explanations,\u201d","volume-title":"Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion, IUI '18 Companion","author":"Costa","year":"2018"},{"key":"B24","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1007\/978-3-642-35142-6_14","article-title":"\u201cAmazon mechanical turk: a research tool for organizations and information systems scholars,\u201d","volume-title":"Shaping the Future of ICT Research. Methods and Approaches","author":"Crowston","year":"2012"},{"key":"B25","first-page":"447","article-title":"\u201cA survey of the state of explainable AI for natural language processing,\u201d","volume-title":"Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing","author":"Danilevsky","year":"2020"},{"key":"B26","doi-asserted-by":"crossref","first-page":"4443","DOI":"10.18653\/v1\/2020.acl-main.408","article-title":"\u201cERASER: a benchmark to evaluate rationalized NLP models,\u201d","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"DeYoung","year":"2020"},{"key":"B27","first-page":"32","article-title":"\u201cMicroTalk: using argumentation to improve crowdsourcing accuracy,\u201d","volume-title":"Proceedings of the AAAI Conference on Human Computation and Crowdsourcing","author":"Drapeau","year":"2016"},{"key":"B28","first-page":"11","article-title":"\u201cCrowdTruth 2.0: quality metrics for crowdsourcing with disagreement,\u201d","volume-title":"1st Workshop on Subjectivity, Ambiguity and Disagreement in Crowdsourcing, and Short Paper 1st Workshop on Disentangling the Relation Between Crowdsourcing and Bias Management, SAD+ CrowdBias 2018","author":"Dumitrache","year":"2018"},{"key":"B29","first-page":"4295","article-title":"Do transformer models show similar attention patterns to task-specific human gaze?","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics","author":"Eberle","year":"2022"},{"key":"B30","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1145\/3278721.3278736","article-title":"\u201cRationalization: a neural machine translation approach to generating natural language explanations,\u201d","volume-title":"Proceedings of the 2018 AAAI\/ACM Conference on AI, Ethics, and Society, AIES '18","author":"Ehsan","year":"2018"},{"key":"B31","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1145\/3301275.3302316","article-title":"\u201cAutomated rationale generation: a technique for explainable AI and its effects on human perceptions,\u201d","volume-title":"Proceedings of the 24th International Conference on Intelligent User Interfaces","author":"Ehsan","year":"2019"},{"key":"B32","doi-asserted-by":"crossref","first-page":"338","DOI":"10.1007\/978-3-031-08337-2_28","article-title":"\u201cOn the evaluation of the plausibility and faithfulness of sentiment analysis explanations,\u201d","author":"El Zini","year":"2022","journal-title":"Artificial Intelligence Applications and Innovations"},{"key":"B33","doi-asserted-by":"crossref","first-page":"165","DOI":"10.18653\/v1\/2021.eval4nlp-1.17","article-title":"\u201cThe Eval4NLP shared task on explainable quality estimation: overview and results,\u201d","volume-title":"Proceedings of the 2nd Workshop on Evaluation and Comparison of NLP Systems","author":"Fomicheva","year":"2021"},{"key":"B34","doi-asserted-by":"crossref","first-page":"1080","DOI":"10.18653\/v1\/2020.findings-emnlp.97","article-title":"\u201cWhy do you think that? Exploring faithful sentence-level rationales without supervision,\u201d","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020","author":"Glockner","year":"2020"},{"key":"B35","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1093\/llc\/fqm020","article-title":"Quantitative authorship attribution: an evaluation of techniques","volume":"22","author":"Grieve","year":"2007","journal-title":"Liter. Lingu. Comp"},{"key":"B36","doi-asserted-by":"crossref","first-page":"6534","DOI":"10.18653\/v1\/2021.emnlp-main.525","article-title":"\u201cSPECTRA: sparse structured text rationalization,\u201d","author":"Guerreiro","year":"2021","journal-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing"},{"key":"B37","doi-asserted-by":"publisher","first-page":"1225093","DOI":"10.3389\/frai.2023.1225093","article-title":"Rationalization for explainable NLP: a survey","volume":"6","author":"Gurrapu","year":"2023","journal-title":"Front. Artif. Intellig"},{"key":"B38","first-page":"3610","article-title":"\u201cRaFoLa: a rationale-annotated corpus for detecting indicators of forced labour,\u201d","volume-title":"Proceedings of the Thirteenth Language Resources and Evaluation Conference","author":"Guzman","year":"2022"},{"key":"B39","doi-asserted-by":"crossref","first-page":"1884","DOI":"10.18653\/v1\/P18-1175","article-title":"\u201cTraining classifiers with natural language explanations,\u201d","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Hancock","year":"2018"},{"key":"B40","doi-asserted-by":"crossref","first-page":"40","DOI":"10.18653\/v1\/2022.lnls-1.5","article-title":"\u201cA survey on improving NLP models with human explanations,\u201d","volume-title":"Proceedings of the First Workshop on Learning with Natural Language Supervision","author":"Hartmann","year":"2022"},{"key":"B41","first-page":"2017","article-title":"\u201cArTest: the first test collection for arabic web search with relevance rationales,\u201d","volume-title":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '20","author":"Hasanain","year":"2020"},{"key":"B42","first-page":"3650","article-title":"\u201cThe out-of-distribution problem in explainability and search methods for feature importance explanations,\u201d","volume-title":"35th Conference on Neural Information Processing Systems (NeurIPS 2021)","author":"Hase","year":"2021"},{"key":"B43","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2021.emnlp-main.510","article-title":"\u201cDoes BERT learn as humans perceive? Understanding linguistic styles through Lexica,\u201d","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Hayati","year":"2021"},{"key":"B44","volume-title":"The Promise and Peril of Human Evaluation for Model Interpretability","author":"Herman","year":"2017"},{"key":"B45","volume-title":"Machine-Annotated Rationales: Faithfully Explaining Machine Learning Models for Text Classification","author":"Herrewijnen","year":"2020"},{"key":"B46","first-page":"11","article-title":"\u201cMachine-annotated rationales: faithfully explaining text classification,\u201d","volume-title":"Proceedings for the Explainable Agency in AI Workshop at the 35th AAAI Conference on Artificial Intelligence","author":"Herrewijnen","year":"2021"},{"key":"B47","doi-asserted-by":"crossref","first-page":"4198","DOI":"10.18653\/v1\/2020.acl-main.386","article-title":"\u201cTowards faithfully interpretable nlp systems: how should we define and evaluate faithfulness?,\u201d","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Jacovi","year":"2020"},{"key":"B48","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1162\/tacl_a_00367","article-title":"Aligning faithful interpretations with their social attribution","volume":"9","author":"Jacovi","year":"2021","journal-title":"Trans. Assoc. Comput. Linguist"},{"key":"B49","first-page":"3543","article-title":"\u201cAttention is not explanation,\u201d","author":"Jain","year":"2019","journal-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019"},{"key":"B50","first-page":"4459","article-title":"\u201cLearning to faithfully rationalize by construction,\u201d","volume-title":"Proceedings of the Association for Computational Linguistics (ACL)","author":"Jain","year":"2020"},{"key":"B51","doi-asserted-by":"crossref","first-page":"1033","DOI":"10.18653\/v1\/2023.acl-long.59","article-title":"\u201cBeing right for whose right reasons?,\u201d","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Jakobsen","year":"2023"},{"key":"B52","doi-asserted-by":"crossref","first-page":"5540","DOI":"10.18653\/v1\/2021.emnlp-main.450","article-title":"\u201cHuman rationales as attribution priors for explainable stance detection,\u201d","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Jayaram","year":"2021"},{"key":"B53","doi-asserted-by":"crossref","first-page":"131","DOI":"10.18653\/v1\/2022.blackboxnlp-1.11","article-title":"\u201cAre multilingual sentiment models equally right for the right reasons?,\u201d","volume-title":"Proceedings of the Fifth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP","author":"J\u00f8rgensen","year":"2022"},{"key":"B54","first-page":"93","article-title":"\u201cER-TEST evaluating explanation regularization methods for NLP models,\u201d","volume-title":"Proceedings of the 2nd Workshop on Trustworthy Natural Language Processing (TrustNLP 2022)","author":"Joshi","year":"2022"},{"key":"B55","first-page":"5911","article-title":"\u201cLogic traps in evaluating attribution scores,\u201d","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics","author":"Ju","year":"2022"},{"key":"B56","article-title":"\u201cRationale-based human-in-the-loop via supervised attention,\u201d","volume-title":"Proceedings of the 1st Workshop on Data Science with Human in the Loop (DaSH) at 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","author":"Kanchinadam","year":"2020"},{"key":"B57","doi-asserted-by":"crossref","first-page":"386","DOI":"10.18653\/v1\/2020.conll-1.31","article-title":"\u201cTrClaim-19: the first collection for turkish check-worthy claim detection with annotator rationales,\u201d","volume-title":"Proceedings of the 24th Conference on Computational Natural Language Learning","author":"Kartal","year":"2020"},{"key":"B58","volume-title":"Learning the Difference That Makes a Difference with Counterfactually-Augmented Data","author":"Kaushik","year":"2019"},{"key":"B59","first-page":"252","article-title":"\u201cLooking beyond the surface: A challenge set for reading comprehension over multiple sentences,\u201d","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Khashabi","year":"2018"},{"key":"B60","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.conll-1.5","article-title":"Modeling Subjective Assessments of Guilt in Newspaper Crime Narratives.","author":"Kreiss","year":"2020","journal-title":"Proceedings of the 24th Conference on Computational Natural Language Learning, pages 56-68. Association for Computational Linguistics."},{"key":"B61","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1109\/TCDS.2016.2628365","article-title":"Learning from explanations using sentiment and advice in RL","volume":"9","author":"Krening","year":"2016","journal-title":"IEEE Trans, Cognitive Dev. Syst"},{"key":"B62","doi-asserted-by":"crossref","first-page":"8730","DOI":"10.18653\/v1\/2020.acl-main.771","article-title":"\u201cNILE: natural language inference with faithful natural language explanations,\u201d","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Kumar","year":"2020"},{"key":"B63","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1613\/jair.1.12012","article-title":"Annotator rationales for labeling tasks in crowdsourcing","volume":"69","author":"Kutlu","year":"2020","journal-title":"J. Artif. Intellig. Res"},{"key":"B64","doi-asserted-by":"publisher","first-page":"790","DOI":"10.1162\/tacl_a_00398","article-title":"QED: A Framework and dataset for explanations in question answering","volume":"9","author":"Lamm","year":"2021","journal-title":"Trans. Assoc. Comp. Linguist"},{"key":"B65","doi-asserted-by":"crossref","first-page":"107","DOI":"10.18653\/v1\/D16-1011","article-title":"\u201cRationalizing neural predictions,\u201d","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","author":"Lei","year":"2016"},{"key":"B66","first-page":"8196","article-title":"\u201cKeywords-guided abstractive sentence summarization,\u201d","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Li","year":"2020"},{"key":"B67","doi-asserted-by":"crossref","first-page":"237","DOI":"10.18653\/v1\/K15-1024","article-title":"\u201cRecovering traceability links in requirements documents,\u201d","volume-title":"Proceedings of the Nineteenth Conference on Computational Natural Language Learning","author":"Li","year":"2015"},{"key":"B68","first-page":"9815","article-title":"\u201cAbstractive summarization: a survey of the state of the art,\u201d","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Lin","year":"2019"},{"key":"B69","doi-asserted-by":"crossref","first-page":"6986","DOI":"10.18653\/v1\/2022.acl-long.481","article-title":"\u201cA rationale-centric framework for human-in-the-loop machine learning,\u201d","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Lu","year":"2022"},{"key":"B70","first-page":"4765","article-title":"\u201cA unified approach to interpreting model predictions,\u201d","author":"Lundberg","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B71","first-page":"1","article-title":"\u201cTowards faithful model explanation in NLP: a survey,\u201d","volume-title":"Computational Linguistics","author":"Lyu","year":"2024"},{"key":"B72","first-page":"305","article-title":"\u201cFaithful chain-of-thought reasoning,\u201d","author":"Lyu","year":"2023","journal-title":"Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)"},{"key":"B73","first-page":"4046","article-title":"\u201cILDC for CJPE: indian legal documents corpus for court judgment prediction and explanation,\u201d","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Malik","year":"2021"},{"key":"B74","doi-asserted-by":"publisher","first-page":"1406","DOI":"10.1109\/JBHI.2015.2431314","article-title":"Automating risk of bias assessment for clinical trials","volume":"19","author":"Marshall","year":"2015","journal-title":"IEEE J. Biomed. Health Inform"},{"key":"B75","doi-asserted-by":"publisher","first-page":"73","DOI":"10.25300\/MISQ\/2014\/38.1.04","article-title":"Explaining data-driven document classifications","volume":"38","author":"Martens","year":"2014","journal-title":"MIS Q"},{"key":"B76","first-page":"14867","article-title":"\u201cHateXplain: a benchmark dataset for explainable hate speech detection,\u201d","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Mathew","year":"2021"},{"key":"B77","doi-asserted-by":"crossref","first-page":"1020","DOI":"10.1109\/ICDM.2012.110","article-title":"\u201cLearning attitudes and attributes from multi-aspect reviews,\u201d","volume-title":"2012 IEEE 12th International Conference on Data Mining","author":"McAuley","year":"2012"},{"key":"B78","first-page":"139","article-title":"\u201cWhy is that relevant? Collecting annotator rationales for relevance judgments,\u201d","volume-title":"Proceedings of the AAAI Conference on Human Computation and Crowdsourcing","author":"McDonnell","year":"2016"},{"key":"B79","doi-asserted-by":"publisher","first-page":"101952","DOI":"10.1016\/j.artmed.2020.101952","article-title":"The natural language explanation algorithms for the lung cancer computer-aided diagnosis system","volume":"108","author":"Meldo","year":"2020","journal-title":"Artif. Intell. Med"},{"key":"B80","article-title":"\u201cExplainable AI: beware of inmates running the asylum,\u201d","volume-title":"Proceedings of the Workshop on Explainable Artificial Intelligence (XAI) (IJCAI 2017)","author":"Miller","year":"2017"},{"key":"B81","first-page":"22","article-title":"\u201cQuantitative evaluation of machine learning explanations: a human-grounded benchmark,\u201d","volume-title":"26th International Conference on Intelligent User Interfaces, IUI '21","author":"Mohseni","year":"2021"},{"key":"B82","doi-asserted-by":"publisher","first-page":"111676","DOI":"10.1016\/j.jss.2023.111676","article-title":"Explaining software fault predictions to spreadsheet users","volume":"201","author":"Mukhtar","year":"2023","journal-title":"J. Syst. Softw"},{"key":"B83","doi-asserted-by":"crossref","first-page":"1938","DOI":"10.18653\/v1\/2020.emnlp-main.153","article-title":"\u201cAn information bottleneck approach for controlling conciseness in rationale extraction,\u201d","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Paranjape","year":"2020"},{"key":"B84","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1162\/tacl_a_00465","article-title":"Evaluating explanations: how much do explanations from the teacher aid students?","volume":"10","author":"Pruthi","year":"2022","journal-title":"Trans. Assoc. Comp. Linguist"},{"key":"B85","first-page":"144","article-title":"\u201cUnderstanding the impact of text highlighting in crowdsourcing tasks,\u201d","volume-title":"Proceedings of the AAAI Conference on Human Computation and Crowdsourcing","author":"Ram\u00edrez","year":"2019"},{"key":"B86","doi-asserted-by":"crossref","first-page":"12709","DOI":"10.18653\/v1\/2023.emnlp-main.783","article-title":"\u201cExplaining interactions between text spans,\u201d","author":"Ray Choudhury","year":"2023","journal-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing"},{"key":"B87","doi-asserted-by":"crossref","first-page":"1135","DOI":"10.1145\/2939672.2939778","article-title":"\u201c\u201cWhy should i trust you?\u201d: explaining the predictions of any classifier,\u201d","volume-title":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Ribeiro","year":"2016"},{"key":"B88","first-page":"38","article-title":"\u201cCan we do better explanations? A proposal of user-centered explainable AI,\u201d","volume-title":"IUI Workshops","author":"Ribera","year":"2019"},{"key":"B89","first-page":"2375","article-title":"\u201cAutomatic detection of psychological distress indicators and severity assessment from online forum posts,\u201d","volume-title":"Proceedings of COLING 2012","author":"Saleem","year":"2012"},{"key":"B90","doi-asserted-by":"crossref","first-page":"5477","DOI":"10.18653\/v1\/2020.acl-main.486","article-title":"\u201cSocial bias frames: reasoning about social and power implications of language,\u201d","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Sap","year":"2020"},{"key":"B91","article-title":"\u201cTowards human-centred explainability benchmarks for text classification,\u201d","author":"Schlegel","year":"2022","journal-title":"Proceedings of the 1st Workshop on Novel Evaluation Approaches for Text Classification Systems (NEATCLasS"},{"key":"B92","doi-asserted-by":"crossref","first-page":"4596","DOI":"10.18653\/v1\/2020.acl-main.419","article-title":"\u201cHuman attention maps for text classification: do humans and neural networks focus on the same words?,\u201d","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Sen","year":"2020"},{"key":"B93","article-title":"\u201cWhat human rationales tell us about machine explanations,\u201d","volume-title":"Non Archival Submission at the First Workshop on Bridging Human-Computer Interaction and Natural Language Processing","author":"Sen","year":"2021"},{"key":"B94","doi-asserted-by":"crossref","first-page":"5263","DOI":"10.18653\/v1\/2020.emnlp-main.425","article-title":"\u201cA computational approach to understanding empathy expressed in text-based mental health support,\u201d","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Sharma","year":"2020"},{"key":"B95","doi-asserted-by":"publisher","first-page":"797","DOI":"10.1007\/s10994-017-5671-3","article-title":"Learning with rationales for document classification","volume":"107","author":"Sharma","year":"2018","journal-title":"Mach. Learn"},{"key":"B96","first-page":"441","article-title":"\u201cActive learning with rationales for text classification,\u201d","volume-title":"Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Sharma","year":"2015"},{"key":"B97","doi-asserted-by":"crossref","first-page":"10","DOI":"10.18653\/v1\/2022.acl-short.2","article-title":"\u201cAre shortest rationales the best explanations for human understanding?,\u201d","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"Shen","year":"2022"},{"key":"B98","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1145\/3375627.3375830","article-title":"\u201cFooling LIME and SHAP: adversarial attacks on post hoc explanation methods,\u201d","volume-title":"Proceedings of the AAAI\/ACM Conference on AI, Ethics, and Society, AIES '20","author":"Slack","year":"2020"},{"key":"B99","first-page":"9109","article-title":"\u201cRobustness to spurious correlations via human annotations,\u201d","volume-title":"International Conference on Machine Learning","author":"Srivastava","year":"2020"},{"key":"B100","first-page":"1527","article-title":"\u201cJoint concept learning and semantic parsing from natural language explanations,\u201d","volume-title":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","author":"Srivastava","year":"2017"},{"key":"B101","doi-asserted-by":"crossref","first-page":"56","DOI":"10.18653\/v1\/W19-4807","article-title":"\u201cDo human rationales improve machine explanations?,\u201d","volume-title":"Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP","author":"Strout","year":"2019"},{"key":"B102","first-page":"521","article-title":"\u201cExplaining why: how instructions and user interfaces impact annotator rationales when labeling text data,\u201d","volume-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Sullivan","year":"2022"},{"key":"B103","article-title":"\u201cModeling annotator rationales with application to pneumonia classification,\u201d","volume-title":"Proceedings of Expanding the Boundaries of Health Informatics Using AI Workshop of AAAI","author":"Tepper","year":"2013"},{"key":"B104","first-page":"308","article-title":"\u201cA joint model of text and aspect ratings for sentiment summarization,\u201d","volume-title":"Proceedings of ACL-08: HLT","author":"Titov","year":"2008"},{"key":"B105","article-title":"\u201cLanguage models don't always say what they think: unfaithful explanations in chain-of-thought prompting,\u201d","author":"Turpin","year":"2024","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B106","doi-asserted-by":"publisher","first-page":"1385","DOI":"10.1613\/jair.1.12752","article-title":"Learning from disagreement: a survey","volume":"72","author":"Uma","year":"2022","journal-title":"J. Artif. Int. Res"},{"key":"B107","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2021.emnlp-main.807","article-title":"\u201cRationales for sequential predictions,\u201d","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021","author":"Vafa","year":"2021"},{"key":"B108","first-page":"2289","article-title":"\u201cIntroducing CAD: the contextual abuse dataset,\u201d","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Vidgen","year":"2021"},{"key":"B109","article-title":"\u201cImproving gender prediction of social media users via weighted annotator rationales,\u201d","volume-title":"NeurIPS 2014 Workshop on Personalization","author":"Volkova","year":"2014"},{"key":"B110","doi-asserted-by":"crossref","first-page":"2034","DOI":"10.18653\/v1\/2022.findings-acl.161","article-title":"\u201cRanking-constrained learning with rationales for text classification,\u201d","volume-title":"Findings of the Association for Computational Linguistics: ACL 2022","author":"Wang","year":"2022"},{"key":"B111","article-title":"\u201cLearning from explanations with neural execution tree,\u201d","volume-title":"8th International Conference on Learning Representations","author":"Wang","year":"2020"},{"key":"B112","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/1852102.1852106","article-title":"A similarity measure for indefinite rankings","volume":"28","author":"Webber","year":"2010","journal-title":"ACM Trans. Inform. Syst. (TOIS)"},{"key":"B113","first-page":"24824","article-title":"\u201cChain of-thought prompting elicits reasoning in large language models,\u201d","volume-title":"36th Conference on Neural Information Processing Systems (NeurIPS 2022)","author":"Wei","year":"2022"},{"key":"B114","article-title":"\u201cTeach me to explain: a review of datasets for explainable natural language processing,\u201d","volume-title":"Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1)","author":"Wiegreffe","year":"2021"},{"key":"B115","doi-asserted-by":"crossref","first-page":"10266","DOI":"10.18653\/v1\/2021.emnlp-main.804","article-title":"\u201cMeasuring association between labels and free-text rationales,\u201d","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Wiegreffe","year":"2021"},{"key":"B116","doi-asserted-by":"crossref","first-page":"412","DOI":"10.18653\/v1\/2022.bionlp-1.41","article-title":"\u201cModel distillation for faithful explanations of medical code predictions,\u201d","volume-title":"Proceedings of the 21st Workshop on Biomedical Language Processing","author":"Wood-Doughty","year":"2022"},{"key":"B117","first-page":"21","article-title":"\u201cClinical corpus annotation: challenges and strategies,\u201d","volume-title":"Proceedings of the third workshop on building and evaluating resources for biomedical text mining (BioTxtM2012) in conjunction with the international conference on language resources and evaluation (LREC)","author":"Xia","year":"2012"},{"key":"B118","doi-asserted-by":"crossref","first-page":"2369","DOI":"10.18653\/v1\/D18-1259","article-title":"\u201cHotpotQA: a dataset for diverse, explainable multi-hop question answering,\u201d","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Yang","year":"2018"},{"key":"B119","first-page":"152","article-title":"\u201cShedding (a thousand points of) light on biased language,\u201d","volume-title":"Mturk@HLT-NAACL","author":"Yano","year":"2010"},{"key":"B120","first-page":"11629","article-title":"\u201cBeyond labels: empowering human annotators with natural language explanations through a novel active-learning architecture,\u201d","author":"Yao","year":"2023","journal-title":"Findings of the Association for Computational Linguistics: EMNLP 2023"},{"key":"B121","first-page":"336","article-title":"\u201cAutomatically generating annotator rationales to improve sentiment classification,\u201d","volume-title":"Proceedings of the ACL 2010 Conference Short Papers","author":"Yessenalina","year":"2010"},{"key":"B122","doi-asserted-by":"crossref","first-page":"2631","DOI":"10.18653\/v1\/2022.acl-long.188","article-title":"\u201cOn the sensitivity and stability of model interpretations,\u201d","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Yin","year":"2022"},{"key":"B123","first-page":"31","article-title":"\u201cModeling annotators: a generative approach to learning from annotator rationales,\u201d","volume-title":"Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing","author":"Zaidan","year":"2008"},{"key":"B124","first-page":"260","article-title":"\u201cUsing \u201cAnno tator Rationales\u201d to improve machine learning for text categorization,\u201d","author":"Zaidan","year":"2007","journal-title":"Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference"},{"key":"B125","first-page":"260","article-title":"\u201cMachine learning with annotator rationales to reduce annotation cost,\u201d","volume-title":"Proceedings of the NIPS* 2008 Workshop on Cost Sensitive Learning","author":"Zaidan","year":"2008"},{"key":"B126","doi-asserted-by":"crossref","first-page":"5736","DOI":"10.18653\/v1\/2020.acl-main.508","article-title":"\u201cwinowhy: a deep diagnosis of essential commonsense knowledge for answering winograd schema challenge,\u201d","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Zhang","year":"2020"},{"key":"B127","doi-asserted-by":"crossref","first-page":"795","DOI":"10.18653\/v1\/D16-1076","article-title":"\u201cRationale-augmented convolutional neural networks for text classification,\u201d","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","author":"Zhang","year":"2016"},{"key":"B128","article-title":"Fine-grained sentiment analysis with faithful attention","author":"Zhong","year":"2019","journal-title":"arXiv [Preprint]. arXiv:"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1260952\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,24]],"date-time":"2024-05-24T04:56:56Z","timestamp":1716526616000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1260952\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,24]]},"references-count":128,"alternative-id":["10.3389\/frai.2024.1260952"],"URL":"https:\/\/doi.org\/10.3389\/frai.2024.1260952","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,24]]}}}