{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,10,27]],"date-time":"2024-10-27T17:40:48Z","timestamp":1730050848933,"version":"3.28.0"},"reference-count":59,"publisher":"Wiley","issue":"8","license":[{"start":{"date-parts":[[2023,9,7]],"date-time":"2023-09-07T00:00:00Z","timestamp":1694044800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Software Testing Verif & Rel"],"published-print":{"date-parts":[[2023,12]]},"abstract":"Summary<\/jats:title>Model\u2010based test design is increasingly being applied in practice and studied in research. Model\u2010based testing\u00a0(MBT) exploits abstract models of the software behaviour to generate abstract tests, which are then transformed into concrete tests ready to run on the code. Given that abstract tests are designed to cover models but are run on code (after transformation), the effectiveness of MBT is dependent on whether model coverage also ensures coverage of key functional code. In this article, we investigate how MBT approaches generate tests from model specifications and how the coverage of tests designed strictly based on the model translates to code coverage. We used snowballing to conduct a systematic literature review. We started with three primary studies, which we refer to as the initial seeds. At the end of our search iterations, we analysed 30 studies that helped answer our research questions. More specifically, this article characterizes how test sets generated at the model level are mapped and applied to the source code level, discusses how tests are generated from the model specifications, analyses how the test coverage of models relates to the test coverage of the code when the same test set is executed and identifies the technologies and software development tasks that are on focus in the selected studies. Finally, we identify common characteristics and limitations that impact the research and practice of MBT:(i)<\/jats:italic>\u00a0some studies did not fully describe how tools transform abstract tests into concrete tests,(ii)<\/jats:italic>\u00a0some studies overlooked the computational cost of model\u2010based approaches and (iii)\u00a0some studies found evidence that bears out a robust correlation between decision coverage at the model level and branch coverage at the code level. We also noted that most primary studies omitted essential details about the experiments.<\/jats:p>","DOI":"10.1002\/stvr.1860","type":"journal-article","created":{"date-parts":[[2023,9,7]],"date-time":"2023-09-07T09:28:14Z","timestamp":1694078894000},"update-policy":"http:\/\/dx.doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["On transforming model\u2010based tests into code: A systematic literature review"],"prefix":"10.1002","volume":"33","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-7339-8529","authenticated-orcid":false,"given":"Fabiano C.","family":"Ferrari","sequence":"first","affiliation":[{"name":"Computing Department Federal University of S\u00e3o Carlos S\u00e3o Carlos Brazil"}]},{"given":"Vinicius H. S.","family":"Durelli","sequence":"additional","affiliation":[{"name":"Computer Science Department Federal University of S\u00e3o Jo\u00e3o del\u2010Rei S\u00e3o Jo\u00e3o del\u2010Rei Brazil"}]},{"given":"Sten F.","family":"Andler","sequence":"additional","affiliation":[{"name":"School of Informatics University of Sk\u00f6vde Sk\u00f6vde Sweden"}]},{"given":"Jeff","family":"Offutt","sequence":"additional","affiliation":[{"name":"Department of Computer Science University at Albany Albany New York USA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-1512-0844","authenticated-orcid":false,"given":"Mehrdad","family":"Saadatmand","sequence":"additional","affiliation":[{"name":"RISE Research Institutes of Sweden V\u00e4ster\u00e5s Sweden"}]},{"given":"Nils","family":"M\u00fcllner","sequence":"additional","affiliation":[{"name":"DLR (Deutsche Luft\u2010 und Raumfahrt, German Aerospace Center) Cologne Germany"}]}],"member":"311","published-online":{"date-parts":[[2023,9,7]]},"reference":[{"key":"e_1_2_11_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2003.1231146"},{"key":"e_1_2_11_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2013.65"},{"key":"e_1_2_11_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2003.1231149"},{"volume-title":"Executable UML: a foundation for model driven architecture","year":"2002","author":"Mellor SJ","key":"e_1_2_11_5_1"},{"key":"e_1_2_11_6_1","doi-asserted-by":"crossref","unstructured":"Elberzhager F Rosbach A Bauer T.Analysis and testing of Matlab Simulink models: a systematic mapping study. InProceedings of the 2013 International Workshop on Joining Academia and Industry Contributions to Testing Automation (JAMAICA).ACM:Lugano Switzerland 2013;29\u201334.","DOI":"10.1145\/2489280.2489285"},{"key":"e_1_2_11_7_1","doi-asserted-by":"crossref","unstructured":"Offutt J Abdurazik A.Generating tests from UML specifications. InProceedings of the Second IEEE International Conference on the Unified Modeling Language (UML99).Springer\u2010Verlag:Fort Collins CO 1999;416\u2013429.","DOI":"10.1007\/3-540-46852-8_30"},{"volume-title":"Practical model\u2010based testing: a tools approach","year":"2006","author":"Utting M","key":"e_1_2_11_8_1"},{"key":"e_1_2_11_9_1","unstructured":"RTCA.Software considerations in airborne systems and equipment certification DO\u2010178C RTCA Inc. 2011."},{"key":"e_1_2_11_10_1","doi-asserted-by":"publisher","DOI":"10.1201\/b19467"},{"key":"e_1_2_11_11_1","doi-asserted-by":"crossref","unstructured":"Wohlin C.Guidelines for snowballing in systematic literature studies and a replication in software engineering. InProceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering (EASE).ACM:London UK 2014;1\u201310.","DOI":"10.1145\/2601248.2601268"},{"volume-title":"Introduction to software testing","year":"2017","author":"Ammann P","key":"e_1_2_11_12_1"},{"key":"e_1_2_11_13_1","doi-asserted-by":"publisher","DOI":"10.1049\/sej.1994.0025"},{"key":"e_1_2_11_14_1","unstructured":"Object Management Group.Model driven architecture (MDA)\u2014MDA guide rev. 2.0. In ormsc\/2014\u201006\u201001 Object Management Group 2014\u00a0https:\/\/www.omg.org\/cgi-bin\/doc?ormsc\/14-06-01.pdf"},{"volume-title":"Advanced compiler design and implementation","year":"1997","author":"Muchnick SS","key":"e_1_2_11_15_1"},{"key":"e_1_2_11_16_1","unstructured":"Baresel A Conrad M Sadeghipour S Wegener J.The interplay between model coverage and code coverage. InProceedings of the 10th EUROSTAR Software Testing Conference.Qualtech Group:Amsterdam The Netherlands 2003;1\u201314."},{"key":"e_1_2_11_17_1","doi-asserted-by":"crossref","unstructured":"Amalfitano D De Simone V Fasolino AR Riccio V.Comparing model coverage and code coverage in model driven testing: an exploratory study. InProceedings of the 6th International Workshop on Testing Techniques for Event Based Software (TESTBEDS).IEEE:Lincoln NE USA 2015;70\u201373.","DOI":"10.1109\/ASEW.2015.18"},{"issue":"6","key":"e_1_2_11_18_1","first-page":"1","article-title":"Using tool integration for improving traceability management testing processes: an automotive industrial experience","volume":"31","author":"Amalfitano D","year":"2019","journal-title":"Software: Evolution and Process"},{"key":"e_1_2_11_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.70708"},{"key":"e_1_2_11_20_1","doi-asserted-by":"crossref","unstructured":"Briand L Nejati S Sabetzadeh M Bianculli D.Testing the untestable: model testing of complex software\u2010intensive systems. InProceedings of the 38th International Conference on Software Engineering (ICSE) \u2010 Visions of 2025 and Beyond Track.ACM:Austin TX USA 2016;789\u2013792.","DOI":"10.1145\/2889160.2889212"},{"key":"e_1_2_11_21_1","unstructured":"Camus J\u2010L Haudebourg C Schlickling M Barrho J.Data Flow model coverage analysis: principles and practice. InProceedings of the 8th European Congress on Embedded Real Time Software and Systems (ERTS).Centre pour la Communication Scientifique Directe:Toulouse France 2016;1\u201310."},{"key":"e_1_2_11_22_1","doi-asserted-by":"crossref","unstructured":"Eriksson A Lindstr\u00f6m B.UML associations: reducing the gap in test coverage between model and code. InProceedings of the 4th International Conference on Model\u2010Driven Engineering and Software Development (MODELSWARD).IEEE:Rome Italy 2016;589\u2013599.","DOI":"10.5220\/0005745205890599"},{"key":"e_1_2_11_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2009.125"},{"key":"e_1_2_11_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2014.05.007"},{"key":"e_1_2_11_25_1","doi-asserted-by":"crossref","unstructured":"Aniculaesei A Vorwald A Rausch A.Using the SCADE toolchain to generate requirements\u2010based test cases for an adaptive cruise control system. InProceedings of the 16th Workshop on Model\u2010Driven Engineering Verification and Validation (MODEVVA).IEEE:Munich Germany 2019;503\u2013513.","DOI":"10.1109\/MODELS-C.2019.00079"},{"key":"e_1_2_11_26_1","doi-asserted-by":"publisher","DOI":"10.1155\/2009\/127945"},{"key":"e_1_2_11_27_1","doi-asserted-by":"crossref","unstructured":"Eriksson A Lindstrm B Offutt J.Transformation rules for platform independent testing: an empirical study. InProceedings of the 6th International Conference on Software Testing Verification and Validation (ICST).IEEE:Luxembourg City Luxembourg 2013;202\u2013211.","DOI":"10.1109\/ICST.2013.28"},{"key":"e_1_2_11_28_1","doi-asserted-by":"crossref","unstructured":"Eriksson A Lindstr\u00f6m B Andler S Offutt J.Model transformation impact on test artifacts: an empirical study. InProceedings of the 9th Workshop on Model\u2010Driven Engineering Verification and Validation (MODEVVA).ACM:Innsbruck Austria 2012;5\u201310.","DOI":"10.1145\/2427376.2427378"},{"key":"e_1_2_11_29_1","doi-asserted-by":"crossref","unstructured":"St\u00fcrmer I Weinberg D Conrad M.Overview of existing safeguarding techniques for automatically generated code. InProceedings of the 2nd International Workshop on Software Engineering for Automotive Systems.ACM:St. Louis MO USA 2005;1\u20136.","DOI":"10.1145\/1082983.1083192"},{"key":"e_1_2_11_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCE.2012.6227464"},{"key":"e_1_2_11_31_1","doi-asserted-by":"crossref","unstructured":"Li G Zhou R Li R He W Lv G Koo TJ.A case study on SDF\u2010based code generation for ECU software development. InProceedings of the 3rd International Workshop on Component\u2010Based Design of Resource\u2010Constrained Systems (CORCS).IEEE:Munich Germany 2011;211\u2013217.","DOI":"10.1109\/COMPSACW.2011.45"},{"key":"e_1_2_11_32_1","unstructured":"Durak U M\u00fcller D M\u00f6cke F Koch CB.Modeling and simulation based development of an enhanced ground proximity warning system for multicore targets. InProceedings of the 2018 international symposium on model\u2010driven approaches for simulation engineering (mod4sim).ACM:Baltimore MD USA 2018;1\u201312."},{"key":"e_1_2_11_33_1","unstructured":"Koch CB Durak U M\u00fcller D.Simulation\u2010based verification for parallelization of model\u2010based applications. InProceedings of the 50th Computer Simulation Conference (SUMMERSIM).ACM:Bordeaux France 2018;1\u201310."},{"key":"e_1_2_11_34_1","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1489"},{"key":"e_1_2_11_35_1","doi-asserted-by":"crossref","unstructured":"Li N Offutt J.A test automation language framework for behavioral models. InProceedings of the 11th Workshop on Advances in Model Based Testing (A\u2010MOST).IEEE:Graz Austria 2015;1\u201310.","DOI":"10.1109\/ICSTW.2015.7107402"},{"key":"e_1_2_11_36_1","doi-asserted-by":"crossref","unstructured":"Abade A Ferrari F Lucr\u00e9dio D.Testing M2T transformations: a systematic literature review. InProceedings of the 17th International Conference on Enterprise Information Systems (ICEIS).SCITEPRESS Digital Library:Barcelona Spain 2015;177\u2013187.","DOI":"10.5220\/0005378501770187"},{"issue":"9","key":"e_1_2_11_37_1","article-title":"Systematic testing of model\u2010based code generators","volume":"33","author":"St\u00farmer I","year":"2007","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_2_11_38_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10703-009-0082-0"},{"key":"e_1_2_11_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2597136"},{"key":"e_1_2_11_40_1","doi-asserted-by":"crossref","unstructured":"Li N Escalona A Kamal T.Skyfire: model\u2010based testing with cucumber. InProceedings of the 9th International Conference on Software Testing Verification and Validation (ICST) \u2010 Testing Tool Papers.IEEE:Chicago IL USA 2016;393\u2013400.","DOI":"10.1109\/ICST.2016.41"},{"key":"e_1_2_11_41_1","doi-asserted-by":"crossref","unstructured":"Vanhecke J Devroey X Perrouin G.AbsCon: a test concretizer for model\u2010based testing. InProceedings of the 15 Workshop on Advances in Model Based Testing (A\u2010MOST).IEEE:Xi'an China 2019;15\u201322.","DOI":"10.1109\/ICSTW.2019.00027"},{"key":"e_1_2_11_42_1","doi-asserted-by":"crossref","unstructured":"Pretschner A Prenninger W Wagner S K\u00fchnel C Baumgartner M Sostawa B Z\u00f6lch R StaunerT.One evaluation of model\u2010based testing and its automation. InProceedings of the 27th International Conference on Software Engineering (ICSE).ACM:St. Louis MO USA 2005;392\u2013401.","DOI":"10.1145\/1062455.1062529"},{"key":"e_1_2_11_43_1","first-page":"583","article-title":"Automatic evaluation of ECU software tests","volume":"114","author":"Conrad M","year":"2005","journal-title":"SAE Transactions"},{"key":"e_1_2_11_44_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2018.12.001"},{"key":"e_1_2_11_45_1","doi-asserted-by":"crossref","unstructured":"Lamancha BP Reales P Polo M Caivano D.Model\u2010driven testing\u2014transformations from test models to test code. InProceedings of the 6th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE).SCITEPRESS Digital Library:Beijing China 2011;121\u2013130.","DOI":"10.5220\/0003466201210130"},{"key":"e_1_2_11_46_1","doi-asserted-by":"crossref","unstructured":"Fraternali P Tisi M.Multi\u2010level tests for model driven web applications. InProceedings of the 10th International Conference on Web Engineering (ICWE).Springer:Vienna Austria 2010;158\u2013172.","DOI":"10.1007\/978-3-642-13911-6_11"},{"key":"e_1_2_11_47_1","doi-asserted-by":"crossref","unstructured":"Veanes M Campbell C Grieskamp W Schulte W Tillmann N Nachmanson L.Model\u2010based testing of object\u2010oriented reactive systems with spec explorer. InProceedings of the 2008 Formal Methods and Testing Workshop (FORTEST).Springer 2008;39\u201376.","DOI":"10.1007\/978-3-540-78917-8_2"},{"key":"e_1_2_11_48_1","doi-asserted-by":"publisher","DOI":"10.1002\/spe.2650"},{"key":"e_1_2_11_49_1","doi-asserted-by":"crossref","unstructured":"Drave I Hillemacher S Greifenberg T Rumpe B Wortmann A Markthaler M Kriebel S.Model\u2010based testing of software\u2010based system functions. InProceedings of the 44th EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA).IEEE 2018;146\u2013153.","DOI":"10.1109\/SEAA.2018.00032"},{"key":"e_1_2_11_50_1","doi-asserted-by":"crossref","unstructured":"Markthaler M Kriebel S Salman KS Greifenberg T Hillemacher S Rumpe B Schulze C WortmannA Orth P Richenhagen J.Improving model\u2010based testing in automotive software engineering. InProceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE\u2010SEIP).ACM 2018;172\u2013180.","DOI":"10.1145\/3183519.3183533"},{"key":"e_1_2_11_51_1","doi-asserted-by":"crossref","unstructured":"Zhou X Jin Y Zhang H Li S Huang X.A map of threats to validity of systematic literature reviews in software engineering. In2016 23rd Asia\u2010Pacific Software Engineering Conference (APSEC) 2016;153\u2013160.","DOI":"10.1109\/APSEC.2016.031"},{"key":"e_1_2_11_52_1","doi-asserted-by":"crossref","unstructured":"Paul TK Lau MF.A systematic literature review on modified condition and decision coverage. InProceedings of the 29th Annual ACM Symposium on Applied Computing (SAC).ACM:Gyeongju Republic of Korea 2014;1301\u20131308.","DOI":"10.1145\/2554850.2555004"},{"key":"e_1_2_11_53_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2016.08.030"},{"key":"e_1_2_11_54_1","doi-asserted-by":"publisher","DOI":"10.1049\/iet-sen.2015.0154"},{"key":"e_1_2_11_55_1","doi-asserted-by":"crossref","unstructured":"Li W Le Gall F Spaseski N.A survey on model\u2010based testing tools for test case generation. InProceedings of the 4th Tools & Methods of Program Analysis International Conference (TMPA).Springer:Moscow Russia 2017;77\u201389.","DOI":"10.1007\/978-3-319-71734-0_7"},{"key":"e_1_2_11_56_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-017-9386-2"},{"key":"e_1_2_11_57_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110608"},{"key":"e_1_2_11_58_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-017-9392-4"},{"key":"e_1_2_11_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2013.46"},{"key":"e_1_2_11_60_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.csi.2017.08.002"}],"container-title":["Software Testing, Verification and Reliability"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/stvr.1860","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,27]],"date-time":"2024-10-27T17:15:10Z","timestamp":1730049310000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/stvr.1860"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,7]]},"references-count":59,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["10.1002\/stvr.1860"],"URL":"https:\/\/doi.org\/10.1002\/stvr.1860","archive":["Portico"],"relation":{},"ISSN":["0960-0833","1099-1689"],"issn-type":[{"type":"print","value":"0960-0833"},{"type":"electronic","value":"1099-1689"}],"subject":[],"published":{"date-parts":[[2023,9,7]]},"assertion":[{"value":"2022-03-03","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-07-30","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}