{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,6,20]],"date-time":"2024-06-20T09:58:54Z","timestamp":1718877534154},"reference-count":58,"publisher":"Wiley","issue":"4","license":[{"start":{"date-parts":[[2022,4,12]],"date-time":"2022-04-12T00:00:00Z","timestamp":1649721600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["J Software Evolu Process"],"published-print":{"date-parts":[[2023,4]]},"abstract":"Abstract<\/jats:title>Microservice\u2010based applications consist of multiple services that can evolve independently. When a service must be updated, it is first tested with in\u2010house regression test suites. However, the test suites that are executed are usually designed without the exact knowledge about how the services will be accessed and used in the field; therefore, they may easily miss relevant test scenarios, failing to prevent the deployment of faulty services. To address this problem, we introduce ExVivoMicroTest<\/jats:sc>, an approach that analyzes the execution of deployed services at run\u2010time in the field, in order to generate test cases for future versions of the same services. ExVivoMicroTest<\/jats:sc> implements lightweight monitoring and tracing capabilities, to inexpensively record executions that can be later turned into regression test cases that capture how services are used in the field. To prevent accumulating an excessive number of test cases, ExVivoMicroTest<\/jats:sc> uses a test coverage model that can discriminate the recorded executions between the ones that are worth to be turned into test cases and the ones that should be discarded. The resulting test cases use a mocked environment that fully isolates the service under test from the rest of the system to faithfully reply interactions. We assessed ExVivoMicroTest<\/jats:sc> with the PiggyMetrics<\/jats:sc> and Train Ticket<\/jats:sc> open source microservice applications and studied how different configurations of the monitoring and tracing logic impact on the capability to generate test cases.<\/jats:p>","DOI":"10.1002\/smr.2452","type":"journal-article","created":{"date-parts":[[2022,4,13]],"date-time":"2022-04-13T01:39:37Z","timestamp":1649813977000},"update-policy":"http:\/\/dx.doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["ExVivoMicroTest<\/scp>: ExVivo Testing of Microservices"],"prefix":"10.1002","volume":"35","author":[{"given":"Luca","family":"Gazzola","sequence":"first","affiliation":[{"name":"Department of Informatics Systems and Communication (DISCo) University of Milano\u2010Bicocca Milan Italy"}]},{"given":"Maayan","family":"Goldstein","sequence":"additional","affiliation":[{"name":"Nokia Bell Labs Tel Aviv Israel"}]},{"ORCID":"http:\/\/orcid.org\/0000-0001-9527-7042","authenticated-orcid":false,"given":"Leonardo","family":"Mariani","sequence":"additional","affiliation":[{"name":"Department of Informatics Systems and Communication (DISCo) University of Milano\u2010Bicocca Milan Italy"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-3499-0159","authenticated-orcid":false,"given":"Marco","family":"Mobilio","sequence":"additional","affiliation":[{"name":"Department of Informatics Systems and Communication (DISCo) University of Milano\u2010Bicocca Milan Italy"}]},{"given":"Itai","family":"Segall","sequence":"additional","affiliation":[{"name":"Nokia Bell Labs Murray Hill New Jersey USA"}]},{"ORCID":"http:\/\/orcid.org\/0000-0001-8840-8948","authenticated-orcid":false,"given":"Alessandro","family":"Tundo","sequence":"additional","affiliation":[{"name":"Department of Informatics Systems and Communication (DISCo) University of Milano\u2010Bicocca Milan Italy"}]},{"given":"Luca","family":"Ussi","sequence":"additional","affiliation":[{"name":"Department of Informatics Systems and Communication (DISCo) University of Milano\u2010Bicocca Milan Italy"}]}],"member":"311","published-online":{"date-parts":[[2022,4,12]]},"reference":[{"key":"e_1_2_12_2_1","volume-title":"Building microservices","author":"Newman S","year":"2015"},{"key":"e_1_2_12_3_1","doi-asserted-by":"crossref","unstructured":"SavorT DouglasM GentiliM WilliamsL BeckK StummM.Continuous deployment at Facebook and OANDA. In: Proceedings of the 38th International Conference on Software Engineering Companion;2016.","DOI":"10.1145\/2889160.2889223"},{"key":"e_1_2_12_4_1","unstructured":"HuntN.Netflix and AWS lambda case study. Online: Accessed 11\u201002\u20102021;2014."},{"key":"e_1_2_12_5_1","doi-asserted-by":"crossref","unstructured":"ChenL.Microservices: Architecting for continuous delivery and DevOps. In: Proceedings of the IEEE International Conference on Software Architecture (ICSA);2018.","DOI":"10.1109\/ICSA.2018.00013"},{"issue":"5","key":"e_1_2_12_6_1","first-page":"92:1","article-title":"Fault analysis and debugging of microservice systems: Industrial survey, benchmark system, and empirical study","volume":"54","author":"Bertolino A","year":"2021","journal-title":"ACM Comput Surv (CSUR)"},{"key":"e_1_2_12_7_1","doi-asserted-by":"crossref","unstructured":"MoranJ BertolinoA de laRivaC TuyaJ.Towards ex vivo testing of MapReduce applications. In: Proceedings of the International Conference on Software Quality Reliability and Security (QRS);2017.","DOI":"10.1109\/QRS.2017.17"},{"key":"e_1_2_12_8_1","doi-asserted-by":"crossref","unstructured":"ElbaumS HardojoM.An empirical study of profiling strategies for released software and their impact on testing activities. In: Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA);2004.","DOI":"10.1145\/1007512.1007522"},{"key":"e_1_2_12_9_1","doi-asserted-by":"publisher","DOI":"10.1504\/IJES.2016.080388"},{"key":"e_1_2_12_10_1","doi-asserted-by":"crossref","unstructured":"FredericksEM DeVriesB ChengBHC.Towards run\u2010time adaptation of test cases for self\u2010adaptive systems in the face of uncertainty. In: Proceedings of the International Symposium on Software Engineering for Adaptive and Self\u2010Managing Systems (SEAMS);2014.","DOI":"10.1145\/2593929.2593937"},{"key":"e_1_2_12_11_1","doi-asserted-by":"crossref","unstructured":"GazzolaL GoldsteinM MarianiL SegallI UssiL.Automatic ex\u2010vivo regression testing of microservices. In: Proceedings of the IEEE\/ACM 1st International Conference on Automation of Software Test (AST);2020.","DOI":"10.1145\/3387903.3389309"},{"key":"e_1_2_12_12_1","unstructured":"PiggyMetrics Contributors.Piggymetrics. Online: Accessed 11\u201002\u20102021;2021."},{"key":"e_1_2_12_13_1","unstructured":"Train Tricket Contributors.Train ticket. Online: Accessed 11\u201009\u20102021;2021."},{"key":"e_1_2_12_14_1","unstructured":"DockerI.Docker. Online: Accessed 11\u201002\u20102021;2021."},{"key":"e_1_2_12_15_1","unstructured":"The Linux Foundation.Kubernetes. Online: Accessed 11\u201002\u20102021;2021."},{"key":"e_1_2_12_16_1","doi-asserted-by":"crossref","unstructured":"DallmeierV LindigC WasylkowskiA ZellerA.Mining object behavior with ADABU. In: Proceedings of the International Workshop on Dynamic Analysis (WODA);2006.","DOI":"10.1145\/1138912.1138918"},{"key":"e_1_2_12_17_1","doi-asserted-by":"publisher","DOI":"10.1049\/iet-sen.2010.0152"},{"key":"e_1_2_12_18_1","doi-asserted-by":"publisher","DOI":"10.1201\/b10928"},{"key":"e_1_2_12_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-012-0103-x"},{"key":"e_1_2_12_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1883612.1883618"},{"key":"e_1_2_12_21_1","doi-asserted-by":"crossref","unstructured":"DingX HuangH RuanY ShaikhA PetersonB ZhangX.Splitter: A proxy\u2010based approach for post\u2010migration testing of web applications. In: Proceedings of the European Conference on Computer Systems;2010.","DOI":"10.1145\/1755913.1755925"},{"key":"e_1_2_12_22_1","unstructured":"Baeldung.How to define a spring boot filter?Online: Accessed 11\u201002\u20102021;2020."},{"key":"e_1_2_12_23_1","unstructured":"ElasticsearchBV.ELK stack: Elasticsearch Logstash Kibana. Online: Accessed 11\u201002\u20102021;2021."},{"key":"e_1_2_12_24_1","unstructured":"The Linux Foundation.Prometheus. Online: Accessed 11\u201002\u20102021;2021."},{"key":"e_1_2_12_25_1","unstructured":"Hewlett\u2010Packard Enterprise Development LP.Monasca. Online: Accessed 11\u201002\u20102021;2017."},{"key":"e_1_2_12_26_1","doi-asserted-by":"crossref","unstructured":"ShatnawiA OrruM MobilioM RiganelliO MarianiL.Cloudhealth: A model\u2010driven approach to watch the health of cloud services. In: Proceedings of the 1st International Workshop on Software Health;2018.","DOI":"10.1145\/3194124.3194130"},{"key":"e_1_2_12_27_1","doi-asserted-by":"crossref","unstructured":"TundoA MobilioM OrrM RiganelliO GuzmanM MarianiL.VARYS: An agnostic model\u2010driven monitoring\u2010as\u2010a\u2010service framework for the cloud. In: Proceedings of the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering(ESEC\/SIGSOFT FSE) Tool Demo Track;2019.","DOI":"10.1145\/3338906.3341185"},{"key":"e_1_2_12_28_1","unstructured":"The Tcpdump Group.Tcpdump\/libpcap. Online: Accessed 11\u201002\u20102021;2021."},{"key":"e_1_2_12_29_1","unstructured":"SantosW.Programmableweb's most popular APIS of 2017. Online: Accessed 11\u201002\u20102021;2018."},{"key":"e_1_2_12_30_1","unstructured":"DOR Green and contributors.Pyshark. Online: Accessed 25\u201009\u20102021;2021."},{"key":"e_1_2_12_31_1","unstructured":"Wireshark Foundation.tshark. Online: Accessed 25\u201009\u20102021;2021."},{"key":"e_1_2_12_32_1","unstructured":"Brandon Byars and Contributors.Mountebank\u2014Over the wire test doubles. Online: Accessed 24\u201009\u20102021;2021."},{"key":"e_1_2_12_33_1","unstructured":"MongoDBIL.mongoreplay. Online: Accessed 24\u201009\u20102021;2021."},{"key":"e_1_2_12_34_1","unstructured":"restfulapi.net.REST api. Online: Accessed 11\u201002\u20102021;2020."},{"key":"e_1_2_12_35_1","unstructured":"MongoDBI.Mongodb wire protocol. Online: Accessed 11\u201002\u20102021;2021."},{"key":"e_1_2_12_36_1","doi-asserted-by":"crossref","unstructured":"ZhouX PengX XieT et al.Delta debugging microservice systems. In: Proceedings of the 33rd ACM\/IEEE International Conference on Automated Software Engineering ASE 2018 Montpellier France September 3\u20107 2018;2018:802\u2010807.","DOI":"10.1145\/3238147.3240730"},{"key":"e_1_2_12_37_1","doi-asserted-by":"crossref","unstructured":"ZhouX PengX XieT et al.Latent error prediction and fault localization for microservice applications by learning from system trace logs. In: Proceedings of the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering ESEC\/SIGSOFT FSE 2019 Tallinn Estonia August 26\u201030 2019;2019:683\u2010694.","DOI":"10.1145\/3338906.3338961"},{"issue":"2","key":"e_1_2_12_38_1","article-title":"Fault analysis and debugging of microservice systems: Industrial survey, benchmark system, and empirical study","volume":"47","author":"Zhou X","year":"2018","journal-title":"IEEE Trans Softw Eng"},{"key":"e_1_2_12_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/367008.367015"},{"key":"e_1_2_12_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.536955"},{"key":"e_1_2_12_41_1","doi-asserted-by":"crossref","unstructured":"LegunsenO ShiA MarinovD.Starts: Static regression test selection. In: Proceedings of the 32Nd IEEE\/ACM International Conference on Automated Software Engineering;2017.","DOI":"10.1109\/ASE.2017.8115710"},{"key":"e_1_2_12_42_1","doi-asserted-by":"crossref","unstructured":"KimJ\u2010M PorterA.A history\u2010based test prioritization technique for regression testing in resource constrained environments. In: Proceedings of the 24th International Conference on Software Engineering (ICSE);2002.","DOI":"10.1145\/581339.581357"},{"key":"e_1_2_12_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.605761"},{"key":"e_1_2_12_44_1","doi-asserted-by":"crossref","unstructured":"MirandaB CrucianiE VerdecchiaR BertolinoA.Fast approaches to scalable similarity\u2010based test case prioritization. In: Proceedings of the 40th International Conference on Software Engineering;2018:222\u2010232.","DOI":"10.1145\/3180155.3180210"},{"key":"e_1_2_12_45_1","first-page":"185","volume-title":"Advances in Computers","author":"Hemmati H","year":"2019"},{"key":"e_1_2_12_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2010.12"},{"key":"e_1_2_12_47_1","doi-asserted-by":"crossref","unstructured":"MarianiL PapagiannakisS PezzeM.Compatibility and regression testing of cots\u2010component\u2010based software. In: Proceedings of the International Conference on Software Engineering (ICSE);2007.","DOI":"10.1109\/ICSE.2007.26"},{"key":"e_1_2_12_48_1","doi-asserted-by":"crossref","unstructured":"BiagiolaM StoccoA RiccaF TonellaP.Diversity\u2010based web test generation. In: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering;2019:142\u2010153.","DOI":"10.1145\/3338906.3338970"},{"key":"e_1_2_12_49_1","doi-asserted-by":"crossref","unstructured":"GodefroidP LehmannD PolishchukM.Differential regression testing for rest APIS. In: Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis;2020:312\u2010323.","DOI":"10.1145\/3395363.3397374"},{"key":"e_1_2_12_50_1","doi-asserted-by":"crossref","unstructured":"WeiX GomezL NeamtiuI FaloutsosM.Profiledroid: Multi\u2010layer profiling of android applications. In: Proceedings of the Annual International Conference on Mobile Computing and Networking (Mobicom);2012.","DOI":"10.1145\/2348543.2348563"},{"key":"e_1_2_12_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2005.50"},{"key":"e_1_2_12_52_1","doi-asserted-by":"crossref","unstructured":"Milani FardA MirzaaghaeiM MesbahA.Leveraging existing tests in automated test generation for web applications. In: Proceedings of the 29th ACM\/IEEE international conference on Automated software engineering;2014:67\u201078.","DOI":"10.1145\/2642937.2642991"},{"key":"e_1_2_12_53_1","doi-asserted-by":"crossref","unstructured":"SavchenkoDI RadchenkoGI TaipaleO.Microservices validation: MJOLNIRR platform case study. In: 2015 38th International Convention on Information and Communication Technology Electronics and Microelectronics (MIPRO) IEEE;2015:235\u2010240.","DOI":"10.1109\/MIPRO.2015.7160271"},{"key":"e_1_2_12_54_1","doi-asserted-by":"crossref","unstructured":"RahmanM GaoJ.A reusable automated acceptance testing architecture for microservices in behavior\u2010driven development. In: 2015 IEEE Symposium on Service\u2010Oriented System Engineering IEEE;2015:321\u2010325.","DOI":"10.1109\/SOSE.2015.55"},{"key":"e_1_2_12_55_1","doi-asserted-by":"crossref","unstructured":"HeorhiadiV RajagopalanS JamjoomH ReiterMK SekarV.Gremlin: Systematic resilience testing of microservices. In: 2016 IEEE 36th International Conference on Distributed Computing Systems (ICDCS) IEEE;2016:57\u201066.","DOI":"10.1109\/ICDCS.2016.11"},{"key":"e_1_2_12_56_1","doi-asserted-by":"crossref","unstructured":"JanesA RussoB.Automatic performance monitoring and regression testing during the transition from monolith to microservices. In: 2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW);2019:163\u2010168.","DOI":"10.1109\/ISSREW.2019.00067"},{"key":"e_1_2_12_57_1","doi-asserted-by":"crossref","unstructured":"SampathS BryceRC ViswanathG KandimallaV KoruAG.Prioritizing user\u2010session\u2010based test cases for web applications testing. In: Proceedings of the International Conference on Software Testing Verification and Validation (ICST);2008.","DOI":"10.1109\/ICST.2008.42"},{"key":"e_1_2_12_58_1","doi-asserted-by":"crossref","unstructured":"AlshahwanN HarmanM.Automated session data repair for web application regression testing. In: Proceedings of International Conference on Software Testing Verification and Validation (ICST);2008.","DOI":"10.1109\/ICST.2008.56"},{"key":"e_1_2_12_59_1","unstructured":"[dataset]GazzolaL GoldsteinM MarianiL MobilioM SegallI TundoA UssiL.Exvivomicrotest replication package;2021.https:\/\/gitlab.com\/learnERC\/exvivomicrotest"}],"container-title":["Journal of Software: Evolution and Process"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/smr.2452","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/smr.2452","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/smr.2452","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,21]],"date-time":"2023-08-21T02:36:01Z","timestamp":1692585361000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/smr.2452"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,12]]},"references-count":58,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,4]]}},"alternative-id":["10.1002\/smr.2452"],"URL":"https:\/\/doi.org\/10.1002\/smr.2452","archive":["Portico"],"relation":{},"ISSN":["2047-7473","2047-7481"],"issn-type":[{"value":"2047-7473","type":"print"},{"value":"2047-7481","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,12]]},"assertion":[{"value":"2021-02-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-03-09","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-04-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}