Abstract
Analytical quality assurance, especially testing, is an integral part of software-intensive system development. With the increased usage of Artificial Intelligence (AI) and Machine Learning (ML) as part of such systems, this becomes more difficult as well-understood software testing approaches cannot be applied directly to the AI-enabled parts of the system. The required adaptation of classical testing approaches and the development of new concepts for AI would benefit from a deeper understanding and exchange between AI and software engineering experts. We see the different terminologies used in the two communities as a major obstacle on this way. As we consider a mutual understanding of the testing terminology a key, this paper contributes a mapping between the most important concepts from classical software testing and AI testing. In the mapping, we highlight differences in the relevance and naming of the mapped concepts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zhang, J.M., Harman, M., Ma, L., Liu, Y.: Machine learning testing: survey, landscapes and horizons. IEEE Trans. Softw. Eng. (2020). https://doi.org/10.1109/TSE.2019.2962027
Kläs, M., Adler, R., Jöckel, L., et al.: Using complementary risk acceptance criteria to structure assurance cases for safety-critical AI components. In: AISafety 2021 (2021)
Kläs, M., Sembach, L.: Uncertainty wrappers for data-driven models – increase the transparency of AI/ML-based models through enrichment with dependable situation-aware uncertainty estimates. In: Romanovsky, A., Troubitsyna, E., Gashi, I., Schoitsch, E., Bitsch, F. (eds.) SAFECOMP 2019. LNCS, vol. 11699, pp. 358–364. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26250-1_29
Riccio, V., Jahangirova, G., Stocco, A., et al.: Testing machine learning based systems: a systematic mapping. Empir. Softw. Eng. 25, 5193–5254 (2020)
IEEE Standard for Software and System Test Documentation. IEEE Std. 829 (2008)
IEEE Standard Glossary of Software Engineering Terminology. IEEE Std. 610:1990 (1990)
ISO/IEC/IEEE Standard for Software Testing – Part 1: Concepts and definitions. ISO/IEC/IEEE 29119-1:2013 (2013)
Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Softw. Test. Verif. Reliabil. 22(5), 297–312 (2012)
Felderer, M., Ramler, R.: Quality assurance for AI-based systems: overview and challenges. In: SWQD 2021 (2021)
Lenarduzzi, V., Lomio, F., Moreschini, S., Taibi, D., Tamburri, D.A.: Software quality for AI: where we are now? In: Winkler, D., Biffl, S., Mendez, D., Wimmer, M., Bergsmann, J. (eds.) SWQD 2021. LNBIP, vol. 404, pp. 43–53. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-65854-0_4
Ammann, P., Offutt, J.: Introduction to Software Testing. Cambridge University Press, Cambridge (2016)
Burnstein, I.: Practical Software Testing – A Process-Oriented Approach. Springer Professional Computing. Springer, Heidelberg (2003).https://doi.org/10.1007/b97392
Siebert, J., Jöckel, L., Heidrich, J., et al.: Construction of a quality model for machine learning systems. Softw. Qual. J. – Spec. Issue Inf. Syst. Qual. (2021). https://doi.org/10.1007/s11219-021-09557-y
Jöckel, L., Kläs, M.: Increasing trust in data-driven model validation – a framework for probabilistic augmentation of images and meta-data generation using application scope characteristics. In: Romanovsky, A., Troubitsyna, E., Bitsch, F. (eds.) SAFECOMP 2019. LNCS, vol. 11698, pp. 155–164. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26601-1_11
Pei, K., Cao, Y., Yang, J., Jana, S.: DeepXplore: automated whitebox testing of deep learning systems. In: SOSP 2017 (2017)
Hendrycks, D., Dietterich, T.: Benchmarking neural network robustness to common corruptions and perturbations. In: ICLR 2019 (2019)
Acknowledgments
Parts of this work have been funded by the Observatory for Artificial Intelligence in Work and Society (KIO) of the Denkfabrik Digitale Arbeitsgesellschaft in the project “KI Testing & Auditing” and by the project “AIControl” as part of the internal funding program “KMU akut” of the Fraunhofer-Gesellschaft.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Jöckel, L., Bauer, T., Kläs, M., Hauer, M.P., Groß, J. (2021). Towards a Common Testing Terminology for Software Engineering and Data Science Experts. In: Ardito, L., Jedlitschka, A., Morisio, M., Torchiano, M. (eds) Product-Focused Software Process Improvement. PROFES 2021. Lecture Notes in Computer Science(), vol 13126. Springer, Cham. https://doi.org/10.1007/978-3-030-91452-3_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-91452-3_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-91451-6
Online ISBN: 978-3-030-91452-3
eBook Packages: Computer ScienceComputer Science (R0)