{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2023,3,14]],"date-time":"2023-03-14T04:16:35Z","timestamp":1678767395513},"reference-count":40,"publisher":"Cambridge University Press (CUP)","issue":"2","license":[{"start":{"date-parts":[[2022,1,24]],"date-time":"2022-01-24T00:00:00Z","timestamp":1642982400000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":["cambridge.org"],"crossmark-restriction":true},"short-container-title":["Nat. Lang. Eng."],"published-print":{"date-parts":[[2023,3]]},"abstract":"Abstract<\/jats:title>In multiple-choice exams, students select one answer from among typically four choices and can explain why they made that particular choice. Students are good at understanding natural language questions and based on their domain knowledge can easily infer the question\u2019s answer by \u201cconnecting the dots\u201d across various pertinent facts. Considering automated reasoning for elementary science question answering, we address the novel task of generating explanations for answers from human-authored facts. For this, we examine the practically scalable framework of feature-rich support vector machines leveraging domain-targeted, hand-crafted features. Explanations are created from a human-annotated set of nearly 5000 candidate facts in the WorldTree corpus. Our aim is to obtain better matches for valid facts of an explanation for the correct answer of a question over the available fact candidates. To this end, our features offer a comprehensive linguistic and semantic unification paradigm<\/jats:italic>. The machine learning problem is the preference ordering of facts, for which we test pointwise regression versus pairwise learning-to-rank. Our contributions, originating from comprehensive evaluations against nine existing systems, are (1) a case study in which two preference ordering approaches are systematically compared, and where the pointwise approach is shown to outperform the pairwise approach, thus adding to the existing survey of observations on this topic; (2) since our system outperforms a highly-effective TF-IDF-based IR technique by 3.5 and 4.9 points on the development and test sets, respectively, it demonstrates some of the further task improvement possibilities (e.g., in terms of an efficient learning algorithm, semantic features) on this task; (3) it is a practically competent approach that can outperform some variants of BERT-based reranking models; and (4) the human-engineered features make it an interpretable machine learning model for the task.<\/jats:p>","DOI":"10.1017\/s1351324921000358","type":"journal-article","created":{"date-parts":[[2022,1,24]],"date-time":"2022-01-24T12:53:28Z","timestamp":1643028808000},"page":"228-253","update-policy":"http:\/\/dx.doi.org\/10.1017\/policypage","source":"Crossref","is-referenced-by-count":0,"title":["Ranking facts for explaining answers to elementary science questions"],"prefix":"10.1017","volume":"29","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-6616-9509","authenticated-orcid":false,"given":"Jennifer","family":"D\u2019Souza","sequence":"first","affiliation":[]},{"given":"Isaiah Onando","family":"Mulang","sequence":"additional","affiliation":[]},{"given":"S\u00f6ren","family":"Auer","sequence":"additional","affiliation":[]}],"member":"56","published-online":{"date-parts":[[2022,1,24]]},"reference":[{"key":"S1351324921000358_ref34","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1256"},{"key":"S1351324921000358_ref7","doi-asserted-by":"publisher","DOI":"10.1145\/2509558.2509565"},{"key":"S1351324921000358_ref29","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33013003"},{"key":"S1351324921000358_ref40","doi-asserted-by":"crossref","unstructured":"Vapnik, V.N. (1999). The nature of statistical learning theory. Springer science & business media.","DOI":"10.1007\/978-1-4757-3264-1"},{"key":"S1351324921000358_ref12","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-14125-6_4"},{"key":"S1351324921000358_ref22","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6319"},{"key":"S1351324921000358_ref10","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-5312"},{"key":"S1351324921000358_ref28","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1118"},{"key":"S1351324921000358_ref24","doi-asserted-by":"publisher","DOI":"10.1023\/B:BTTJ.0000047600.45421.6d"},{"key":"S1351324921000358_ref36","unstructured":"Seo, M.J. , Kembhavi, A. , Farhadi, A. and Hajishirzi, H. (2016). Bidirectional Attention Flow for Machine Comprehension. ArXiv, abs\/1611.01603."},{"key":"S1351324921000358_ref27","first-page":"73","article-title":"Pairwise versus pointwise ranking: A case study","volume":"25","author":"Melnikov","year":"2016","journal-title":"Schedae Informaticae"},{"key":"S1351324921000358_ref35","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"S1351324921000358_ref23","doi-asserted-by":"crossref","unstructured":"Khot, T. , Sabharwal, A. and Clark, P. (2018). Scitail: A textual entailment dataset from science question answering. In AAAI.","DOI":"10.1609\/aaai.v32i1.12022"},{"key":"S1351324921000358_ref4","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-5311"},{"key":"S1351324921000358_ref5","unstructured":"Clark, P. , Cowhey, I. , Etzioni, O. , Khot, T. , Sabharwal, A. , Schoenick, C. and Tafjord, O. (2018). Think you have Solved Question Answering? Try Arc, the ai2 Reasoning Challenge. ArXiv, abs\/1803.05457."},{"key":"S1351324921000358_ref17","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4615-0907-3"},{"key":"S1351324921000358_ref6","doi-asserted-by":"crossref","unstructured":"Clark, P. , Etzioni, O. , Khashabi, D. , Khot, T. , Mishra, B.D. , Richardson, K. , Sabharwal, A. , Schoenick, C. , Tafjord, O. , Tandon, N. , Bhakthavatsalam, S. , Groeneveld, D. , Guerquin, M. and Schmitz, M. (2019). From \u2018F\u2019 to \u2018A\u2019 on the N.Y. regents science exams: An overview of the aristo project. arXiv preprintarXiv: abs\/1909.01958.","DOI":"10.1609\/aimag.v41i4.5304"},{"key":"S1351324921000358_ref2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-5310"},{"key":"S1351324921000358_ref38","unstructured":"Swayamdipta, S. , Thomson, S. , Dyer, C. and Smith, N.A. (2017). Frame-Semantic Parsing with Softmax-Margin Segmental RNNs and a Syntactic Scaffold. arXiv preprint arXiv:1706.09528."},{"key":"S1351324921000358_ref37","unstructured":"Speer, R. , Chin, J. and Havasi, C. (2017). Conceptnet 5.5: An open multilingual graph of general knowledge. In Thirty-First AAAI Conference on Artificial Intelligence."},{"key":"S1351324921000358_ref20","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-14125-6_9"},{"key":"S1351324921000358_ref11","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00133"},{"key":"S1351324921000358_ref18","doi-asserted-by":"publisher","DOI":"10.1145\/1150402.1150429"},{"key":"S1351324921000358_ref19","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2005.138"},{"key":"S1351324921000358_ref25","unstructured":"Lundberg, S.M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, pp. 4765\u20134774."},{"key":"S1351324921000358_ref26","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_a_00239"},{"key":"S1351324921000358_ref31","unstructured":"Nogueira, R. and Cho, K. (2019). Passage Re-Ranking with Bert. arXiv preprint arXiv:1901.04085."},{"key":"S1351324921000358_ref15","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-5309"},{"key":"S1351324921000358_ref32","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1244"},{"key":"S1351324921000358_ref33","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1368"},{"key":"S1351324921000358_ref39","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4020-6710-5_3"},{"key":"S1351324921000358_ref9","unstructured":"Devlin, J. , Chang, M. , Lee, K. and Toutanova, K. (2018). BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. CoRR, abs\/1810.04805."},{"key":"S1351324921000358_ref3","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1454"},{"key":"S1351324921000358_ref1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1034"},{"key":"S1351324921000358_ref30","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287574"},{"key":"S1351324921000358_ref21","unstructured":"Khashabi, D. , Azer, E.S. , Khot, T. , Sabharwal, A. and Roth, D. (2019). On the Capabilities and Limitations of Reasoning for Natural Language Understanding. arXiv preprint arXiv:1901.02522."},{"key":"S1351324921000358_ref16","unstructured":"Jansen, P. , Wainwright, E. , Marmorstein, S. and Morrison, C.T. 2018. Worldtree: A corpus of explanation graphs for elementary science questions supporting multi-hop inference. In Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC)."},{"key":"S1351324921000358_ref8","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-5313"},{"key":"S1351324921000358_ref13","unstructured":"Jansen, P. , Balasubramanian, N. , Surdeanu, M. and Clark, P. (2016). What\u2019s in an explanation? characterizing knowledge and inference requirements for elementary science exams. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 2956\u20132965."},{"key":"S1351324921000358_ref14","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_a_00287"}],"container-title":["Natural Language Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1351324921000358","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,13]],"date-time":"2023-03-13T04:19:44Z","timestamp":1678681184000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1351324921000358\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,24]]},"references-count":40,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,3]]}},"alternative-id":["S1351324921000358"],"URL":"https:\/\/doi.org\/10.1017\/s1351324921000358","relation":{},"ISSN":["1351-3249","1469-8110"],"issn-type":[{"value":"1351-3249","type":"print"},{"value":"1469-8110","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,24]]},"assertion":[{"value":"\u00a9 The Author(s), 2022. Published by Cambridge University Press","name":"copyright","label":"Copyright","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}}]}}