Abstract
Testing is a vital part of the development of a new software product. With the rise of test automation, companies more and more rely on large sets of test cases. This leads to situations in which it is unfeasible to run all tests due to a limited time budget which eventually results in the need for selecting an optimal subset of tests to execute. Recently, this test selection problem has been approached using machine learning methods. In this work, we design an Organic Computing (OC) system which makes use of these methods. While OC design techniques have originally been targeted at creating embedded systems, we show that these methodologies can be employed to software verification as well. We are able to demonstrate that the implemented system is a robust and highly autonomous solution which fits modern development practices such as continuous integration well.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
In the original MLOC variant the monitoring focuses on one isolated agent.
- 2.
For example Bosch and Siemens home appliances are from the same producer and a Bosch home appliance often shares many things with a Siemens one.
- 3.
We decided to use a Friedman test over a more widespread ANOVA test as it does not require equality of variance. The latter is not given here which renders an ANOVA analysis unfeasible.
References
Angara, J., Gutta, S., Prasad, S.: DevOps with continuous testing architecture and its metrics model. In: Sa, P.K., Bakshi, S., Hatzilygeroudis, I.K., Sahoo, M.N. (eds.) Recent Findings in Intelligent Computing Techniques. AISC, vol. 709, pp. 271–281. Springer, Singapore (2018). https://doi.org/10.1007/978-981-10-8633-5_28
Dingsyr, T., Dyb, T., Moe, N.B.: Agile Software Development: Current Research and Future Directions, 1st edn. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12575-1
Dustin, E., Rashka, J., Paul, J.: Automated Software Testing: Introduction, Management, and Performance. Addison-Wesley Longman Publishing Co. Inc., Boston (1999)
Ferguson, J.: Jenkins: The Definitive Guide. O’Reilly, Beijing (2011)
Herzig, K., Greiler, M., Czerwonka, J., Murphy, B.: The art of testing less without sacrificing quality. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, vol. 1, pp. 483–493 (2015)
Jfrog: Jfrog Artifactory (2020). https://jfrog.com/artifactory/. Accessed 26 Jan 2021
Larman, C., Vodde, B.: Scaling Lean and Agile Development: Thinking and Organizational Tools for Large-Scale Scrum, 1st edn. Addison-Wesley Professional, Boston (2008)
Lazic, L., Mastorakis, N.: Cost effective software test metrics, vol. 7 (2008)
Müller-Schloer, C., Schmeck, H., Ungerer, T.: Organic Computing - A Paradigm Shift for Complex Systems, 1st edn. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-0348-0130-0
Prothmann, H., et al.: Decentralised route guidance in organic traffic control (2011)
Qu, X., Cohen, M.B., Woolf, K.M.: Combinatorial interaction regression testing: a study of test case generation and prioritization. In: 2007 IEEE International Conference on Software Maintenance, pp. 255–264 (2007)
Rosenbauer, L., Pätzel, D., Stein, A., Hähner, J.: Transfer learning for automated test case prioritization using XCSF. In: Castillo, P.A., Jiménez Laredo, J.L. (eds.) EvoApplications 2021. LNCS, vol. 12694, pp. 681–696. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72699-7_43
Rosenbauer, L., Stein, A., Pätzel, D., Hähner, J.: XCSF with experience replay for automatic test case prioritization. In: 2020 IEEE Symposium Series on Computational Intelligence (SSCI) (2020)
Rudolph, S.: Mutual Influences in Self adaptive and Autonomously Learning Systems. Ph.D. thesis (2020)
Schmeck, H., Müller-Schloer, C., Çakar, E., Mnif, M., Richter, U.: Adaptivity and self-organisation in organic computing systems. In: Müller-Schloer, C., Schmeck, H., Ungerer, T. (eds.) Organic Computing—A Paradigm Shift for Complex Systems. ASYS, vol. 1, pp. 5–37. Springer, Basel (2011). https://doi.org/10.1007/978-3-0348-0130-0_1
Spieker, H., Gotlieb, A., Marijan, D., Mossige, M.: Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration. In: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis. p. 12–22. ISSTA 2017, Association for Computing Machinery, New York, NY, USA (2017)
Stein, A., Maier, R., Rosenbauer, L., Hähner, J.: XCS classifier system with experience replay. In: Proceedings of the 2020 Genetic and Evolutionary Computation Conference. GECCO 2020, pp. 404–413. Association for Computing Machinery, New York, NY, USA (2020)
Ståhl, D., Bosch, J.: Modeling continuous integration practice differences in industry software development. J. Syst. Softw. 87, 48–59 (2014)
Ståhl, D., Bosch, J.: Cinders: the continuous integration and delivery architecture framework. Inf. Softw. Technol. 83, 76–93 (2017)
Tomforde, S., et al.: Decentralised progressive signal systems for organic traffic control. In: 2008 Second IEEE International Conference on Self-Adaptive and Self-Organizing Systems, pp. 413–422 (2008)
Urbanowicz, R.J., Browne, W.N.: Introduction to Learning Classifier Systems, 1st edn. Springer, Heidelberg (2017). https://doi.org/10.1007/978-3-662-55007-6
Wilson, S.W.: Classifier fitness based on accuracy. Evol. Comput. 3(2), 149–175 (1995)
Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. Softw. Test. Verif. Reliab. 22(2), 67–120 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Rosenbauer, L., Pätzel, D., Stein, A., Hähner, J. (2021). An Organic Computing System for Automated Testing. In: Hochberger, C., Bauer, L., Pionteck, T. (eds) Architecture of Computing Systems. ARCS 2021. Lecture Notes in Computer Science(), vol 12800. Springer, Cham. https://doi.org/10.1007/978-3-030-81682-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-81682-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-81681-0
Online ISBN: 978-3-030-81682-7
eBook Packages: Computer ScienceComputer Science (R0)