{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,6,27]],"date-time":"2024-06-27T13:40:38Z","timestamp":1719495638559},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"6","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,7,31]]},"abstract":"\n Natural language processing (NLP) techniques have been widely applied in the requirements engineering (RE) field to support tasks such as classification and ambiguity detection. Despite its empirical vocation, RE research has given limited attention to replication of NLP for RE studies. Replication is hampered by several factors, including the context specificity of the studies, the heterogeneity of the tasks involving NLP, the tasks\u2019 inherent\n hairiness<\/jats:italic>\n , and, in turn, the heterogeneous reporting structure. To address these issues, we propose a new artifact, referred to as\n ID-Card<\/jats:sc>\n , whose goal is to provide a structured summary of research papers emphasizing replication-relevant information. We construct the\n ID-Card<\/jats:sc>\n through a structured, iterative process based on design science. In this article: (i) we report on hands-on experiences of replication; (ii) we review the state-of-the-art and extract replication-relevant information: (iii) we identify, through focus groups, challenges across two typical dimensions of replication: data annotation and tool reconstruction; and (iv) we present the concept and structure of the\n ID-Card<\/jats:sc>\n to mitigate the identified challenges. This study aims to create awareness of replication in NLP for RE. We propose an\n ID-Card<\/jats:sc>\n that is intended to foster study replication but can also be used in other contexts, e.g., for educational purposes.\n <\/jats:p>","DOI":"10.1145\/3658669","type":"journal-article","created":{"date-parts":[[2024,4,15]],"date-time":"2024-04-15T12:23:41Z","timestamp":1713183821000},"page":"1-33","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Replication in Requirements Engineering: The NLP for RE Case"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"http:\/\/orcid.org\/0000-0001-6095-447X","authenticated-orcid":false,"given":"Sallam","family":"Abualhaija","sequence":"first","affiliation":[{"name":"University of Luxembourg, Luxembourg, Luxembourg"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-3833-3997","authenticated-orcid":false,"given":"F. Basak","family":"Aydemir","sequence":"additional","affiliation":[{"name":"Utrecht University, Utrecht Netherlands"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-4480-3887","authenticated-orcid":false,"given":"Fabiano","family":"Dalpiaz","sequence":"additional","affiliation":[{"name":"Utrecht University, Utrecht Netherlands"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-1162-8341","authenticated-orcid":false,"given":"Davide","family":"Dell'Anna","sequence":"additional","affiliation":[{"name":"Utrecht University, Utrecht, Netherlands"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-0636-5663","authenticated-orcid":false,"given":"Alessio","family":"Ferrari","sequence":"additional","affiliation":[{"name":"CNR ISTI, Pisa Italy"}]},{"ORCID":"http:\/\/orcid.org\/0000-0001-9733-8830","authenticated-orcid":false,"given":"Xavier","family":"Franch","sequence":"additional","affiliation":[{"name":"Universitat Polit\u00e8cnica de Catalunya, Barcelona, Spain"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-0679-4361","authenticated-orcid":false,"given":"Davide","family":"Fucci","sequence":"additional","affiliation":[{"name":"Blekinge Institute of Technology, Karlskrona, Sweden"}]}],"member":"320","published-online":{"date-parts":[[2024,6,27]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","unstructured":"Sallam Abualhaija Fatma Ba\u015fak Aydemir Fabiano Dalpiaz Davide Dell\u2019Anna Alessio Ferrari Xavier Franch and Davide Fucci. 2024. Online Annex: Replication in Requirements Engineering: The NLP for RE Case. DOI:10.6084\/m9.figshare.21824481","DOI":"10.6084\/m9.figshare.21824481"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/SERA54885.2022.9806783"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2964587"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1038\/533452a"},{"key":"e_1_3_3_6_2","unstructured":"D. Berry E. Kamsties and M. Krieger. 2003. From Contract Drafting to Software Specification: Linguistic Sources of Ambiguity A Handbook. Retrieved from http:\/\/se.uwaterloo.ca\/dberry\/handbook\/ambiguityHandbook.pdf"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1080\/03098260600927575"},{"key":"e_1_3_3_8_2","first-page":"1","volume-title":"1st International Workshop on Replication in Empirical Software Engineering","author":"Carver Jeffrey C.","year":"2010","unstructured":"Jeffrey C. Carver. 2010. Towards reporting guidelines for experimental replications: A proposal. In 1st International Workshop on Replication in Empirical Software Engineering, Vol. 1. 1\u20134."},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1082"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2006.65"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-007-0045-1"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2952191"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-012-9227-7"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2019.00025"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","unstructured":"Fabiano Dalpiaz Davide Dell\u2019Anna Fatma Ba\u015fak Aydemir and Sercan \u00c7evikol. 2019. explainable-re\/RE-2019-Materials v0. DOI:10.5281\/zenodo.3309669","DOI":"10.5281\/zenodo.3309669"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2018.3571242"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-77243-1_8"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.2307\/249008"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-14192-8_21"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2015.02.001"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2023.102181"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","unstructured":"Saad Ezzini Sallam Abualhaija Chetan Arora and Mehrdad Sabetzadeh. 2021. taphsir v6. DOI:10.5281\/zenodo.6009790","DOI":"10.5281\/zenodo.6009790"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510157"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00133"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-016-9451-7"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2017.4121207"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2017.29"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-023-00399-7"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-30985-5_12"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2007.45"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-011-9181-9"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2023.107318"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409767"},{"key":"e_1_3_3_34_2","volume-title":"Joining Together: Group Theory and Group Skills","author":"Johnson David W.","year":"1991","unstructured":"David W. Johnson and Frank P. Johnson. 1991. Joining Together: Group Theory and Group Skills. Prentice-Hall, Inc."},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-010-9141-9"},{"key":"e_1_3_3_36_2","volume-title":"Proceedings of the 13th International Conference on Software and Systems Engineering and Applications (ICSSEA\u201900)","author":"Kamsties Erik","year":"2000","unstructured":"Erik Kamsties and Barbara Peach. 2000. Taming ambiguity in natural language requirements. In Proceedings of the 13th International Conference on Software and Systems Engineering and Applications (ICSSEA\u201900)."},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1249"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-008-0063-7"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2017.82"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-018-0293-2"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.2307\/2529786"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2014.6912271"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-32489-6_17"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-019-09712-x"},{"key":"e_1_3_3_45_2","first-page":"1","article-title":"Empirical research on requirements quality: A systematic mapping study","author":"Montgomery Lloyd","year":"2022","unstructured":"Lloyd Montgomery, Davide Fucci, Abir Bouraffa, Lisa Scholz, and Walid Maalej. 2022. Empirical research on requirements quality: A systematic mapping study. Requirements Engineering (2022), 1\u201327.","journal-title":"Requirements Engineering"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-37422-7_5"},{"key":"e_1_3_3_47_2","first-page":"1","volume-title":"International Conference on Model-Based Software and Systems Engineering","author":"Rahimi Shekoufeh","year":"2022","unstructured":"Shekoufeh Rahimi, Kevin Charles Lano, and Chenghua Lin. 2022. Requirement formalisation using natural language processing and machine learning: A systematic review. In International Conference on Model-Based Software and Systems Engineering. SCITEPRESS Digital Library, 1\u20138."},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2020.102472"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-15538-4_2"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2018.01.006"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-008-9060-1"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-43839-8"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/REW.2016.021"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549172"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-29044-2"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-011-0119-y"},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.5555\/1873781.1873918"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3444689"},{"key":"e_1_3_3_59_2","article-title":"A survey of large language models","author":"Zhao Wayne Xin","year":"2023","unstructured":"Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et\u00a0al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223 (2023).","journal-title":"arXiv preprint arXiv:2303.18223"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3658669","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,27]],"date-time":"2024-06-27T13:09:19Z","timestamp":1719493759000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3658669"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,27]]},"references-count":58,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,7,31]]}},"alternative-id":["10.1145\/3658669"],"URL":"https:\/\/doi.org\/10.1145\/3658669","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,27]]},"assertion":[{"value":"2023-06-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-20","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}