{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,13]],"date-time":"2024-09-13T07:51:26Z","timestamp":1726213886710},"reference-count":65,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2022,9,20]],"date-time":"2022-09-20T00:00:00Z","timestamp":1663632000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,9,20]],"date-time":"2022-09-20T00:00:00Z","timestamp":1663632000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010663","name":"H2020 European Research Council","doi-asserted-by":"publisher","award":["957254-COSMOS"],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/N023978\/2"],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2022,12]]},"abstract":"Abstract<\/jats:title>Test smells aim to capture design issues in test code that reduces its maintainability. These have been extensively studied and generally found quite prevalent in both human-written and automatically generated test-cases. However, most evidence of prevalence is based on specific static detection rules. Although those are based on the original, conceptual definitions of the various test smells, recent empirical studies indicate that developers perceive warnings raised by detection tools as overly strict and non-representative of the maintainability and quality of test suites. This leads us to re-assess test smell detection tools\u2019 detection accuracy and investigate the prevalence and detectability of test smells more broadly. Specifically, we construct a hand-annotated dataset spanning hundreds of test suites both written by developers and generated by two test generation tools (EvoSuite<\/jats:sc>andJTExpert<\/jats:sc>) and performed a multi-stage, cross-validated manual analysis to identify the presence of six types of test smells in these. We then use this manual labeling to benchmark the performance and external validity of two test smell detection tools\u2014one widely used in prior work and one recently introduced with the express goal to match developer perceptions of test smells. Our results primarily show that the current vocabulary of test smells is highly mismatched to real concerns: multiple smells were ubiquitous on developer-written tests but virtually never correlated with semantic or maintainability flaws; machine-generated tests actually often scored better, but in reality, suffered from a host of problems not well-captured by current test smells. Current test smell detection strategies poorly characterized the issues in these automatically generated test suites; in particular, the older tool\u2019s detection strategies misclassified over 70% of test smells, both missing real instances (false negatives) and marking many smell-free tests as smelly (false positives). We identify common patterns in these tests that can be used to improve the tools, refine and update the definition of certain test smells, and highlight as of yet uncharacterized issues. Our findings suggest the need for (i) more appropriate metrics to match development practice, (ii) more accurate detection strategies to be evaluated primarily in industrial contexts.<\/jats:p>","DOI":"10.1007\/s10664-022-10207-5","type":"journal-article","created":{"date-parts":[[2022,9,20]],"date-time":"2022-09-20T09:04:03Z","timestamp":1663664643000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Test smells 20 years later: detectability, validity, and reliability"],"prefix":"10.1007","volume":"27","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-7395-3588","authenticated-orcid":false,"given":"Annibale","family":"Panichella","sequence":"first","affiliation":[]},{"given":"Sebastiano","family":"Panichella","sequence":"additional","affiliation":[]},{"given":"Gordon","family":"Fraser","sequence":"additional","affiliation":[]},{"given":"Anand Ashok","family":"Sawant","sequence":"additional","affiliation":[]},{"given":"Vincent J.","family":"Hellendoorn","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,9,20]]},"reference":[{"key":"10207_CR1","doi-asserted-by":"crossref","unstructured":"Afshan S, McMinn P, Stevenson M (2013) Evolving readable string test inputs using a natural language model to reduce human oracle cost. In: 2013 IEEE Sixth international conference on software testing, verification and validation. IEEE, pp 352\u2013361","DOI":"10.1109\/ICST.2013.11"},{"key":"10207_CR2","doi-asserted-by":"crossref","unstructured":"Almasi M M, Hemmati H, Fraser G, Arcuri A, Benefelds J (2017) An industrial evaluation of unit test generation: finding real faults in a financial application. In: ICSE SEIP, pp 263\u2013272","DOI":"10.1109\/ICSE-SEIP.2017.27"},{"key":"10207_CR3","doi-asserted-by":"publisher","DOI":"10.1017\/9781316771273","volume-title":"Introduction to software testing","author":"P Ammann","year":"2016","unstructured":"Ammann P, Offutt J (2016) Introduction to software testing. Cambridge University Press, Cambridge"},{"issue":"1","key":"10207_CR4","doi-asserted-by":"publisher","first-page":"80","DOI":"10.1109\/TSE.2010.46","volume":"37","author":"JH Andrews","year":"2011","unstructured":"Andrews J H, Menzies T, Li F C (2011) Genetic algorithms for randomized unit testing. IEEE Trans Softw Eng 37(1):80\u201394","journal-title":"IEEE Trans Softw Eng"},{"issue":"3","key":"10207_CR5","doi-asserted-by":"publisher","first-page":"594","DOI":"10.1007\/s10664-013-9249-9","volume":"18","author":"A Arcuri","year":"2013","unstructured":"Arcuri A, Fraser G (2013) Parameter tuning or default values? An empirical investigation in search-based software engineering. Empir Softw Eng 18 (3):594\u2013623","journal-title":"Empir Softw Eng"},{"key":"10207_CR6","doi-asserted-by":"crossref","unstructured":"Arcuri A, Fraser G, Galeotti J P (2014) Automated unit test generation for classes with environment dependencies. In: International conference on automated software engineering, pp 79\u201390","DOI":"10.1145\/2642937.2642986"},{"key":"10207_CR7","doi-asserted-by":"crossref","unstructured":"Baresi L, Miraz M (2010) Testful: automatic unit-test generation for java classes. In: International conference on software engineering, vol 2, pp 281\u2013284","DOI":"10.1145\/1810295.1810353"},{"key":"10207_CR8","doi-asserted-by":"crossref","unstructured":"Bavota G, Qusef A, Oliveto R, Lucia A D, Binkley D W (2012) An empirical analysis of the distribution of unit test smells and their impact on software maintenance. In: ICSM, pp 56\u201365","DOI":"10.1109\/ICSM.2012.6405253"},{"issue":"4","key":"10207_CR9","doi-asserted-by":"publisher","first-page":"1052","DOI":"10.1007\/s10664-014-9313-0","volume":"20","author":"G Bavota","year":"2015","unstructured":"Bavota G, Qusef A, Oliveto R, De Lucia A, Binkley D (2015) Are test smells really harmful? An empirical study. Empir Softw Eng 20(4):1052\u20131094","journal-title":"Empir Softw Eng"},{"key":"10207_CR10","doi-asserted-by":"crossref","unstructured":"Birchler C, Ganz N, Khatiri S, Gambi A, Panichella S (2022a) Cost-effective simulationbased test selection in self-driving cars software with sdc-scissor. In: 2022 IEEE 29th international conference on software analysis, evolution and reengineering (SANER). doi:to appear","DOI":"10.1109\/SANER53432.2022.00030"},{"key":"10207_CR11","doi-asserted-by":"crossref","unstructured":"Birchler C, Khatiri S, Derakhshanfar P, Panichella S, Panichella A (2022b) Single and multi-objective test cases prioritization for self-driving cars in virtual environments. ACM Trans Softw Eng Methodol (TOSEM). doi:to appear","DOI":"10.1145\/3533818"},{"key":"10207_CR12","doi-asserted-by":"crossref","unstructured":"B\u00f6hme M, Szekeres L, Metzman J (2022) On the reliability of coverage-based fuzzer benchmarking","DOI":"10.1145\/3510003.3510230"},{"key":"10207_CR13","doi-asserted-by":"publisher","first-page":"207","DOI":"10.1016\/j.infsof.2018.08.010","volume":"104","author":"J Campos","year":"2018","unstructured":"Campos J, Ge Y, Albunian N, Fraser G, Eler M, Arcuri A (2018) An empirical evaluation of evolutionary algorithms for unit test suite generation. Inf Softw Technol 104:207\u2013235","journal-title":"Inf Softw Technol"},{"key":"10207_CR14","doi-asserted-by":"publisher","unstructured":"Canfora G, Lucia A D, Penta M D, Oliveto R, Panichella A, Panichella S (2013) Multi-objective cross-project defect prediction. In: Sixth IEEE international conference on software testing, verification and validation, ICST 2013, Luxembourg, Luxembourg, March 18\u201322, 2013. https:\/\/doi.org\/10.1109\/ICST.2013.38. IEEE Computer Society, pp 252\u2013261","DOI":"10.1109\/ICST.2013.38"},{"issue":"4","key":"10207_CR15","doi-asserted-by":"publisher","first-page":"426","DOI":"10.1002\/stvr.1570","volume":"25","author":"G Canfora","year":"2015","unstructured":"Canfora G, Lucia A D, Penta M D, Oliveto R, Panichella A, Panichella S (2015) Defect prediction as a multiobjective optimization problem. Softw Test Verific Reliab 25(4):426\u2013459. https:\/\/doi.org\/10.1002\/stvr.1570","journal-title":"Softw Test Verific Reliab"},{"issue":"11","key":"10207_CR16","first-page":"1025","volume":"34","author":"C Csallner","year":"2004","unstructured":"Csallner C, Smaragdakis Y (2004) Jcrasher: an automatic robustness tester for java. Softw: Pract Exp 34(11):1025\u20131050","journal-title":"Softw: Pract Exp"},{"key":"10207_CR17","doi-asserted-by":"crossref","unstructured":"Daka E, Campos J, Fraser G, Dorn J, Weimer W (2015) Modeling readability to improve unit tests. In: Joint meeting on foundations of software engineering, pp 107\u2013118","DOI":"10.1145\/2786805.2786838"},{"key":"10207_CR18","doi-asserted-by":"crossref","unstructured":"Daka E, Rojas J M, Fraser G (2017) Generating unit tests with descriptive names or: would you name your children thing1 and thing2?. In: International symposium on software testing and analysis, pp 57\u201367","DOI":"10.1145\/3092703.3092727"},{"issue":"3","key":"10207_CR19","doi-asserted-by":"crossref","first-page":"e1733","DOI":"10.1002\/stvr.1733","volume":"30","author":"P Derakhshanfar","year":"2020","unstructured":"Derakhshanfar P, Devroey X, Perrouin G, Zaidman A, van Deursen A (2020) Search-based crash reproduction using behavioural model seeding. Softw Test Verific Reliab 30(3):e1733","journal-title":"Softw Test Verific Reliab"},{"key":"10207_CR20","unstructured":"Deursen A, Moonen L, Bergh A, Kok G (2001) Refactoring test code. In: Proceedings of the 2nd international conference on extreme programming and flexible processes (XP2001), pp 92\u201395"},{"key":"10207_CR21","doi-asserted-by":"publisher","unstructured":"Devroey X, Panichella S, Gambi A (2020) Java unit testing tool competition\u2014eighth round, Seoul. https:\/\/doi.org\/10.1145\/3387940.3392265https:\/\/doi.org\/10.1145\/3387940.3392265","DOI":"10.1145\/3387940.3392265 10.1145\/3387940.3392265"},{"key":"10207_CR22","doi-asserted-by":"crossref","unstructured":"Elbaum S, Chin H N, Dwyer M B, Dokulil J (2006) Carving differential unit test cases from system test cases. In: Proceedings of the 14th ACM SIGSOFT international symposium on foundations of software engineering. Association for Computing Machinery, New York, pp 253\u2013264","DOI":"10.1145\/1181775.1181806"},{"key":"10207_CR23","unstructured":"Fowler M (1999) Refactoring: improving the design of existing code. Addison-Wesley Professional"},{"key":"10207_CR24","doi-asserted-by":"crossref","unstructured":"Fraser G, Arcuri A (2011) Evosuite: automatic test suite generation for object-oriented software. In: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on foundations of software engineering, pp 416\u2013419","DOI":"10.1145\/2025113.2025179"},{"issue":"2","key":"10207_CR25","doi-asserted-by":"publisher","first-page":"276","DOI":"10.1109\/TSE.2012.14","volume":"39","author":"G Fraser","year":"2012","unstructured":"Fraser G, Arcuri A (2012) Whole test suite generation. IEEE Trans Softw Eng 39(2):276\u2013291","journal-title":"IEEE Trans Softw Eng"},{"issue":"2","key":"10207_CR26","first-page":"8:1","volume":"24","author":"G Fraser","year":"2014","unstructured":"Fraser G, Arcuri A (2014) A large-scale evaluation of automated unit test generation using evosuite. Trans Softw Eng Methodol 24(2):8:1\u20138:42. ISSN 1049-331X","journal-title":"Trans Softw Eng Methodol"},{"issue":"3","key":"10207_CR27","doi-asserted-by":"publisher","first-page":"611","DOI":"10.1007\/s10664-013-9288-2","volume":"20","author":"G Fraser","year":"2015","unstructured":"Fraser G, Arcuri A (2015a) 1600 faults in 100 projects: automatically finding faults while achieving high coverage with evosuite. Empir Softw Eng 20 (3):611\u2013639","journal-title":"Empir Softw Eng"},{"issue":"3","key":"10207_CR28","doi-asserted-by":"publisher","first-page":"783","DOI":"10.1007\/s10664-013-9299-z","volume":"20","author":"G Fraser","year":"2015","unstructured":"Fraser G, Arcuri A (2015b) Achieving scalable mutation-based generation of whole test suites. Empir Softw Eng 20(3):783\u2013812","journal-title":"Empir Softw Eng"},{"issue":"2","key":"10207_CR29","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1109\/TSE.2011.93","volume":"38","author":"G Fraser","year":"2011","unstructured":"Fraser G, Zeller A (2011) Mutation-driven generation of unit tests and oracles. Trans Softw Eng 38(2):278\u2013292","journal-title":"Trans Softw Eng"},{"key":"10207_CR30","doi-asserted-by":"publisher","first-page":"312","DOI":"10.1016\/j.jss.2019.07.016","volume":"156","author":"G Grano","year":"2019","unstructured":"Grano G, Palomba F, Di Nucci D, De Lucia A, Gall H C (2019) Scented since the beginning: on the diffuseness of test smells in automatically generated test code. J Syst Softw 156:312\u2013327","journal-title":"J Syst Softw"},{"key":"10207_CR31","doi-asserted-by":"crossref","unstructured":"Just R, Ernst M D, Fraser G (2014) Efficient mutation analysis by propagating and partitioning infected execution states. In: Proceedings of the 2014 international symposium on software testing and analysis, pp 315\u2013326","DOI":"10.1145\/2610384.2610388"},{"key":"10207_CR32","doi-asserted-by":"crossref","unstructured":"Kifetew F, Devroey X, Rueda U (2019) Java unit testing tool competition-seventh round. In: International workshop on search-based software testing, pp 15\u201320","DOI":"10.1109\/SBST.2019.00014"},{"key":"10207_CR33","doi-asserted-by":"crossref","unstructured":"Ma L, Artho C, Zhang C, Sato H, Gmeiner J, Ramler R (2015) Grt: program-analysis-guided random testing (t). In: 2015 30th IEEE\/ACM international conference on automated software engineering (ASE). IEEE, pp 212\u2013223","DOI":"10.1109\/ASE.2015.49"},{"key":"10207_CR34","doi-asserted-by":"crossref","unstructured":"Messaoudi S, Shin D, Panichella A, Bianculli D, Briand L C (2021) Log-based slicing for system-level test cases. In: Proceedings of the 30th ACM SIGSOFT international symposium on software testing and analysis, pp 517\u2013528","DOI":"10.1145\/3460319.3464824"},{"key":"10207_CR35","doi-asserted-by":"crossref","unstructured":"Pacheco C, Lahiri S K, Ernst M D, Ball T (2007) Feedback-directed random test generation. In: International conference on software engineering, pp 75\u201384","DOI":"10.1109\/ICSE.2007.37"},{"key":"10207_CR36","doi-asserted-by":"crossref","unstructured":"Palomba F, Di Nucci D, Panichella A, Oliveto R, De Lucia A (2016) On the diffusion of test smells in automatically generated test code: an empirical study. In: 2016 IEEE\/ACM 9th international workshop on search-based software testing (SBST). IEEE, pp 5\u201314","DOI":"10.1145\/2897010.2897016"},{"key":"10207_CR37","doi-asserted-by":"publisher","unstructured":"Panichella S (2015) Supporting newcomers in software development projects. In: Koschke R, Krinke J, Robillard MP (eds) 2015 IEEE International conference on software maintenance and evolution, ICSME 2015, Bremen, Germany, September 29\u2013October 1, 2015. https:\/\/doi.org\/10.1109\/ICSM.2015.7332519. IEEE Computer Society, pp 586\u2013589","DOI":"10.1109\/ICSM.2015.7332519"},{"key":"10207_CR38","doi-asserted-by":"crossref","unstructured":"Panichella A, Molina U R (2017) Java unit testing tool competition-fifth round. In: International workshop on search-based software testing, pp 32\u201338","DOI":"10.1109\/SBST.2017.7"},{"key":"10207_CR39","doi-asserted-by":"crossref","unstructured":"Panichella S, Panichella A, Beller M, Zaidman A, Gall H C (2016) The impact of test case summaries on bug fixing performance: an empirical investigation. In: International conference on software engineering, pp 547\u2013558","DOI":"10.7287\/peerj.preprints.1467v3"},{"issue":"2","key":"10207_CR40","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1109\/TSE.2017.2663435","volume":"44","author":"A Panichella","year":"2017","unstructured":"Panichella A, Kifetew F M, Tonella P (2017) Automated test case generation as a many-objective optimisation problem with dynamic selection of the targets. Trans Softw Eng 44(2):122\u2013158","journal-title":"Trans Softw Eng"},{"key":"10207_CR41","doi-asserted-by":"crossref","unstructured":"Panichella A, Kifetew F M, Tonella P (2018a) Incremental control dependency frontier exploration for many-criteria test case generation. In: International symposium on search based software engineering. Springer, pp 309\u2013324","DOI":"10.1007\/978-3-319-99241-9_17"},{"key":"10207_CR42","doi-asserted-by":"publisher","first-page":"236","DOI":"10.1016\/j.infsof.2018.08.009","volume":"104","author":"A Panichella","year":"2018","unstructured":"Panichella A, Kifetew F M, Tonella P (2018b) A large scale empirical comparison of state-of-the-art search-based test case generators. Inf Softw Technol 104:236\u2013256","journal-title":"Inf Softw Technol"},{"key":"10207_CR43","doi-asserted-by":"crossref","unstructured":"Panichella A, Campos J, Fraser G (2020a) Evosuite at the sbst 2020 tool competition. In: Proceedings of the IEEE\/ACM 42nd international conference on software engineering workshops, pp 549\u2013552","DOI":"10.1145\/3387940.3392266"},{"key":"10207_CR44","doi-asserted-by":"crossref","unstructured":"Panichella A, Panichella S, Fraser G, Sawant A A, Hellendoorn V J (2020b) Revisiting test smells in automatically generated tests: limitations, pitfalls, and opportunities. In: 2020 IEEE International conference on software maintenance and evolution (ICSME). IEEE, pp 523\u2013533","DOI":"10.1109\/ICSME46990.2020.00056"},{"key":"10207_CR45","doi-asserted-by":"crossref","unstructured":"Panichella S, Gambi A, Zampetti F, Riccio V (2021) Sbst tool competition 2021. In: International conference on software engineering, workshops, Madrid, Spain, 2021. ACM","DOI":"10.1109\/SBST52555.2021.00011"},{"key":"10207_CR46","unstructured":"Peruma A S A (2018) What the smell? An empirical investigation on the distribution and severity of test smells in open source android applications"},{"key":"10207_CR47","doi-asserted-by":"crossref","unstructured":"Robinson B, Ernst M D, Perkins J H, Augustine V, Li N (2011) Scaling up automated test generation: automatically generating maintainable regression unit tests for programs. In: International conference on automated software engineering, pp 23\u201332","DOI":"10.1109\/ASE.2011.6100059"},{"key":"10207_CR48","doi-asserted-by":"crossref","unstructured":"Rojas J M, Campos J, Vivanti M, Fraser G, Arcuri A (2015) Combining multiple coverage criteria in search-based unit test generation. In: International symposium on search based software engineering. Springer, pp 93\u2013108","DOI":"10.1007\/978-3-319-22183-0_7"},{"issue":"5","key":"10207_CR49","doi-asserted-by":"publisher","first-page":"366","DOI":"10.1002\/stvr.1601","volume":"26","author":"JM Rojas","year":"2016","unstructured":"Rojas J M, Fraser G, Arcuri A (2016) Seeding strategies in search-based unit test generation. Softw Test Verif Reliab 26(5):366\u2013401","journal-title":"Softw Test Verif Reliab"},{"issue":"2","key":"10207_CR50","doi-asserted-by":"publisher","first-page":"852","DOI":"10.1007\/s10664-015-9424-2","volume":"22","author":"JM Rojas","year":"2017","unstructured":"Rojas J M, Vivanti M, Arcuri A, Fraser G (2017) A detailed investigation of the effectiveness of whole test suite generation. Empir Softw Eng 22 (2):852\u2013893","journal-title":"Empir Softw Eng"},{"key":"10207_CR51","doi-asserted-by":"crossref","unstructured":"Roy D, Zhang Z, Ma M, Arnaoudova V, Panichella A, Panichella S, Gonzalez D, Mirakhorli M (2020) Deeptc-enhancer: improving the readability of automatically generated tests. In: 2020 35th IEEE\/ACM international conference on automated software engineering (ASE). IEEE, pp 287\u2013298","DOI":"10.1145\/3324884.3416622"},{"issue":"3","key":"10207_CR52","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1109\/TSE.2014.2363479","volume":"41","author":"A Sakti","year":"2014","unstructured":"Sakti A, Pesant G, Gu\u00e9h\u00e9neuc Y -G (2014) Instance generator and problem representation to improve object oriented code coverage. Trans Softw Eng 41(3):294\u2013313","journal-title":"Trans Softw Eng"},{"key":"10207_CR53","doi-asserted-by":"publisher","unstructured":"Sakti A, Pesant G, Gu\u00e9h\u00e9neuc Y (2017) Jtexpert at the SBST 2017 tool competition. In: 10th IEEE\/ACM international workshop on search-based software testing, SBST@ICSE 2017, Buenos Aires, Argentina, May 22\u201323, 2017. https:\/\/doi.org\/10.1109\/SBST.2017.5. IEEE, pp 43\u201346","DOI":"10.1109\/SBST.2017.5"},{"key":"10207_CR54","doi-asserted-by":"crossref","unstructured":"Shamshiri S, Rojas J M, Galeotti J P, Walkinshaw N, Fraser G (2018) How do automatically generated unit tests influence software maintenance?. In: International conference on software testing, verification and validation, pp 250\u2013261","DOI":"10.1109\/ICST.2018.00033"},{"issue":"12","key":"10207_CR55","doi-asserted-by":"publisher","first-page":"1294","DOI":"10.1109\/TSE.2018.2877664","volume":"46","author":"M Soltani","year":"2018","unstructured":"Soltani M, Panichella A, Van Deursen A (2018) Search-based crash reproduction and its impact on debugging. IEEE Trans Softw Eng 46(12):1294\u20131317","journal-title":"IEEE Trans Softw Eng"},{"key":"10207_CR56","doi-asserted-by":"crossref","unstructured":"Spadini D, Palomba F, Zaidman A, Bruntink M, Bacchelli A (2018) On the relation of test smells to software code quality. In: International conference on software maintenance and evolution, pp 1\u201312","DOI":"10.1109\/ICSME.2018.00010"},{"key":"10207_CR57","doi-asserted-by":"crossref","unstructured":"Spadini D, Schvarcbacher M, Oprescu A -M, Bruntink M, Bacchelli A (2020) Investigating severity thresholds for test smells","DOI":"10.1145\/3379597.3387453"},{"issue":"4","key":"10207_CR58","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1145\/1013886.1007528","volume":"29","author":"P Tonella","year":"2004","unstructured":"Tonella P (2004) Evolutionary testing of classes. ACM SIGSOFT Softw Eng Notes 29(4):119\u2013128","journal-title":"ACM SIGSOFT Softw Eng Notes"},{"key":"10207_CR59","doi-asserted-by":"crossref","unstructured":"Tsantalis N, Chaikalis T, Chatzigeorgiou A (2018) Ten years of jdeodorant: lessons learned from the hunt for smells. In: Oliveto R, Penta MD, Shepherd DC (eds) 25th International conference on software analysis, evolution and reengineering, SANER 2018, Campobasso, Italy, March 20\u201323, 2018. IEEE Computer Society, pp 4\u201314","DOI":"10.1109\/SANER.2018.8330192"},{"key":"10207_CR60","doi-asserted-by":"crossref","unstructured":"Tufano M, Palomba F, Bavota G, Di Penta M, Oliveto R, De Lucia A, Poshyvanyk D (2016) An empirical investigation into the nature of test smells. In: International conference on automated software engineering, pp 4\u201315","DOI":"10.1145\/2970276.2970340"},{"issue":"11","key":"10207_CR61","doi-asserted-by":"publisher","first-page":"1063","DOI":"10.1109\/TSE.2017.2653105","volume":"43","author":"M Tufano","year":"2017","unstructured":"Tufano M, Palomba F, Bavota G, Oliveto R, Penta M D, Lucia A D, Poshyvanyk D (2017) When and why your code starts to smell bad (and whether the smells go away). Trans Softw Eng 43(11):1063\u20131088","journal-title":"Trans Softw Eng"},{"key":"10207_CR62","doi-asserted-by":"crossref","unstructured":"Xie T (2006) Augmenting automatically generated unit-test suites with regression oracle checking. In: European conference on object-oriented programming. Springer, pp 380\u2013403","DOI":"10.1007\/11785477_23"},{"key":"10207_CR63","doi-asserted-by":"crossref","unstructured":"Xuan J, Monperrus M (2014) Test case purification for improving fault localization. In: Proceedings of the 22nd ACM SIGSOFT international symposium on foundations of software engineering, pp 52\u201363","DOI":"10.1145\/2635868.2635906"},{"issue":"2","key":"10207_CR64","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1002\/stv.430","volume":"22","author":"S Yoo","year":"2012","unstructured":"Yoo S, Harman M (2012) Regression testing minimization, selection and prioritization: a survey. Softw Test Verif Reliab 22(2):67\u2013120","journal-title":"Softw Test Verif Reliab"},{"key":"10207_CR65","doi-asserted-by":"crossref","unstructured":"Zhang S (2013) Practical semantic test simplification. In: International conference on software engineering, pp 1173\u20131176","DOI":"10.1109\/ICSE.2013.6606671"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10207-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-022-10207-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10207-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T06:57:00Z","timestamp":1701068220000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-022-10207-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,20]]},"references-count":65,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["10207"],"URL":"https:\/\/doi.org\/10.1007\/s10664-022-10207-5","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,20]]},"assertion":[{"value":"2 July 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 September 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declared that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}}],"article-number":"170"}}