Evaluating Different Strategies for Testing Software Product Lines | Journal of Electronic Testing Skip to main content
Springer Nature Link
Log in

Evaluating Different Strategies for Testing Software Product Lines

  • Published:
Journal of Electronic Testing Aims and scope Submit manuscript

Abstract

The Software Product Line (SPL) approach is associated with many benefits, and with some challenges too. In the SPL testing, a great challenge is the integration of test methods and techniques with the SPL engineering. To do this, some strategies have been proposed, however, they have not been properly evaluated and compared. In this paper a case study is described comparing three different test strategies: the product by product strategy that tests each product individually; an incremental strategy that tests the products reusing test cases from products previously tested; and a strategy that instantiates test data derived in the domain engineering, considering SPL commonalities and variabilities. In the study an SPL from the games domain was used, and the test data were generated from use cases. The results show that SPL oriented strategies are associated to greater percentages of reuse, and consequently, with lower effort to write test cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Assunção WKG, Trindade DFG, Colanzi TE, Vergilio SR (2011) Evaluating test reuse of a software product line strategies. In: Latin-American test workshop (LATW)

  2. Bertolino A, Gnesi S (2003) Use case-based testing of product lines. In: 9th European software engineering conference held jointly with 11th ACM SIGSOFT international symposium on foundations of software engineering, Helsinki, Finland, pp 355–358

  3. Bertolino A, Gnesi S (2004) Pluto: a test methodology for product families. Lect Notes Comput Sci 3014:181–197

    Article  Google Scholar 

  4. Clements PC, Jones LG, Northrop LM, McGregor JD (2005) Project management in a software product line organization. IEEE Softw 22(5):54–62

    Article  Google Scholar 

  5. Clements PC, Jones LG, Mcgregor JD, Northrop LM (2006) Getting there from here: a roadmap for software product line adoption. Commun ACM 49(12):33–36

    Article  Google Scholar 

  6. Cockburn A (2001) Writing effective use cases. Addison Wesley, Reading

    Google Scholar 

  7. Deus GD, Vincenzi AMR, Lucena FN, Delamaro ME (2009) Quality evaluation of J2ME products by using experimentation packages. In: Simpósio Brasileiro de qualidade de software, SBQS’2009, pp 264–278 (in Portuguese)

  8. Edwin OO (2007) Testing in software product lines. Master thesis, School of Engineering, Blekinge Institute of Technology, Sweden

  9. Heumann J (2001) Generating test cases from use cases. The rational edge. Technical report. Available at: http://www.ibm.com/developerworks/rational/library/content/RationalEdge/jun01/GeneratingTestCasesFromUseCasesJune01.pdf. Accessed June 2010

  10. Jaaksi A (2002) Developing mobile browsers in a product line. IEEE Softw 19:73–80

    Article  Google Scholar 

  11. Jazayeri M, Ran A, Linden F (2000) Software architecture for product families. Addison Wesley, Reading

    Google Scholar 

  12. Krueger C (2002) Eliminating the adoption barrier. IEEE Softw (Special issue on Iniciating Software Product Lines) 19(4):28–31

    Google Scholar 

  13. Lamancha BP, Usaola M (2010) Testing product generation in software product lines using pairwise for features coverage. In: Petrenko A, Simão A, Maldonado J (eds) Testing software and systems. Lecture notes in computer science, vol 6435, pp 111–125

  14. Lamancha BP, Usaola M, Piattini M (2009). In: International conference on software, vol 49, pp 78–81

  15. McGregor JD (2001) Testing a software product line. Technical report, Software Engineering Institute, Carnegie Mellon University, Technical Report CMU/SEI-2001-TR-022

  16. Morin B, Klein J, Barais O, Jézéquel J (2008) A generic weaver for supporting product lines. In: International conference on software engineering (ICSE). Proceedings of the 13th international workshop on early aspects, Leipzig, Germany, pp 11–18

  17. Nebut C, Fleurey F, Le Traon Y, Jezequel J-M (2006) Automatic test generation: a use case driven approach. IEEE Trans Softw Eng 32(3):140–155

    Article  Google Scholar 

  18. Oliveira Jr E, Gimenes I, Maldonado J (2010) Systematic management of variability in UML-based software product lines. J Univers Comput Sci 16(17):2374–2393

    Google Scholar 

  19. Oster S, Markert F, Ritter P (2010) Automated incremental pair-wise testing of software product lines. In: Bosch J, Lee J (eds) Software product lines: going beyond. Lecture notes in computer science, vol 6287, pp 196–210

  20. Perrouin G, Sen S, Klein J, Baudry B, Traon YL (2010) Automated and scalable t-wise test case generation strategies for software product lines. In: Proceedings of the 2010 third international conference on software testing, verification and validation, ICST ’10. IEEE Computer Society, Washington, DC, pp 459–468

  21. Pohl K,Metzger A (2006) Software product line testing: exploring principles and potential solutions. Commun ACM 49(12):78–81

    Article  Google Scholar 

  22. Reuys A, Kamsties E, Pohl K, Reis S (2005) Model-based system testing of software product families. In: Conference on advanced information systems engineering (CAISE), pp 519–534

  23. Segura S, Hierons RM, Benavides D, Ruiz-Cortés A (2010) Automated test data generation on the analyses of feature models: a metamorphic testing approach. In: Third international conference on software testing, verification and validation, ICST ’10, pp 35–44

  24. SEI (2011) Arcade game maker pedagogical product line. Technical report, Software Engineering Institute, Carnegie Mellon University. Available at: http://www.sei.cmu.edu/productlines/ppl/. Accessed June 2010

  25. Tevanlinna A, Taina J, Kauppinen R (2004) Product family testing: a survey. ACM SIGSOFT Softw Eng Notes 29(2):1–6

    Article  Google Scholar 

  26. Trew T (2005) Enabling the smooth integration of core assets: defining and packaging architectural rules for a family of embedded products. In: Software product line conference (SPLC), pp 137–149

  27. Uzuncaova E, Khurshid S, Batory DF (2010) Incremental test generation for software product lines. IEEE Trans Softw Eng 36(3):309–322

    Article  Google Scholar 

  28. van der Linden F, Schimd K, Rommes E (2007) Software product lines in action—the best industrial practice in product line engineering. Springer, New York

    Google Scholar 

Download references

Acknowledgments

We would like to thank to Edison Klafke Fillus, Alessandro G. F. Fior, Alssio M. Junior and Rafaella A. L. Silva for their contribution in the experiment. Thelma is sponsored by CNPQ. Daniela is sponsored by Fundação Araucária de Apoio ao Desenvolvimento Científico e Tecnológico (Araucária Foundation for Scientific and Technological Research). This work was also supported by CNPq.

Author information

Authors and Affiliations

  1. Computer Science Department, State University of Maringá (UEM), Paraná, Brazil

    Thelma Elita Colanzi

  2. Computer Science Department, Federal University of Paraná (UFPR), Paraná, Brazil

    Wesley Klewerton Guez Assunção & Silvia Regina Vergilio

  3. Technological Science Center, State University of Northern Paraná (UENP), Paraná, Brazil

    Daniela de Freitas Guilhermino Trindade

  4. Informatics Department, Alto Vale do Rio do Peixe University (UNIARP), Caçador, SC, Brazil

    Carlos Alberto Zorzo

Authors
  1. Thelma Elita Colanzi
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Wesley Klewerton Guez Assunção
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Daniela de Freitas Guilhermino Trindade
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Carlos Alberto Zorzo
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Silvia Regina Vergilio
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Wesley Klewerton Guez Assunção.

Additional information

Responsible Editor: F. Lima Kastensmidt

Rights and permissions

Reprints and permissions

About this article

Cite this article

Colanzi, T.E., Assunção, W.K.G., de Freitas Guilhermino Trindade, D. et al. Evaluating Different Strategies for Testing Software Product Lines. J Electron Test 29, 9–24 (2013). https://doi.org/10.1007/s10836-012-5343-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10836-012-5343-y

Keywords