Design Principles for Collaborative Generative AI Systems in Software Development | SpringerLink
Skip to main content
Springer Nature Link
Log in

Design Principles for Collaborative Generative AI Systems in Software Development

  • Conference paper
  • First Online:
Design Science Research for a Resilient Future (DESRIST 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14621))

  • 1095 Accesses

Abstract

Generative artificial intelligence (GAI) has the potential to transform software development practices with prior research indicating significant overall enhancements in developers’ productivity. However, there exists a lack of design knowledge for organization-specific GAI systems to assist software development. To bridge this research gap, we derive a design framework for collaborative GAI systems in software development following design science research. Specifically, we conducted eight interviews with practitioners and reviewed extant literature to formulate design requirements and design principles. In our analysis of the literature and our qualitative data, we identify problems surrounding usability, data privacy, hallucination and transparency. To address these problems, we propose GAI system designs that enable user-centricity, data protection, quality control and communication. Our findings contribute valuable design knowledge to the field of generative AI and organizational software development practices.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 14871
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 10581
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    LLMs represent a type of GAI models specifically designed to understand, generate, and engage with human language [39]. While the majority of GAI systems that we refer to throughout our paper are based on LLMs, we will use the term GAI for consistency.

References

  1. Alshenqeeti, H.: Interviewing as a data collection method: a critical review. English Linguist. Res. 3(1), 39–45 (2014)

    Google Scholar 

  2. Banh, L., Strobel, G.: Generative artificial intelligence. Electron. Mark. 33(1), 1–17 (2023)

    Article  Google Scholar 

  3. Baskerville, R., Baiyere, A., Gregor, S., Hevner, A., Rossi, M.: Design science research contributions: finding a balance between artifact and theory. J. Assoc. Inf. Syst. 19(5), 358–376 (2018)

    Google Scholar 

  4. Bauer, K., Hinz, O., van der Aalst, W., Weinhardt, C.: Expl (AI) n it to me–explainable AI and information systems research (2021)

    Google Scholar 

  5. Bauer, K., von Zahn, M., Hinz, O.: Expl (AI) ned: the impact of explainable artificial intelligence on users’ information processing. Inf. Syst. Res. 34(4), 1582–1602 (2023)

    Article  Google Scholar 

  6. Britto, R., Wohlin, C., Mendes, E.: An extended global software engineering taxonomy. J. Softw. Eng. Res. Dev. 4, 1–24 (2016)

    Article  Google Scholar 

  7. Cao, Y., Li, S., Liu, Y., Yan, Z., Dai, Y., Yu, P.S., Sun, L.: A comprehensive survey of AI-generated content (AIGC): a history of generative AI from GAN to ChatGPT. arXiv preprint arXiv:2303.04226 (2023)

  8. Chen, J., et al.: Designing expert-augmented clinical decision support systems to predict mortality risk in ICUs. KI-Künstliche Intelligenz 1–10 (2023)

    Google Scholar 

  9. Dakhel, A.M., Majdinasab, V., Nikanjam, A., Khomh, F., Desmarais, M.C., Jiang, Z.M.J.: Github copilot AI pair programmer: asset or liability? J. Syst. Softw. 203, 111734 (2023)

    Article  Google Scholar 

  10. Dang, H., Mecke, L., Lehmann, F., Goller, S., Buschek, D.: How to prompt? Opportunities and challenges of zero-and few-shot learning for human-AI interaction in creative applications of generative models. arXiv preprint arXiv:2209.01390 (2022)

  11. Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 13(3), 319–340 (1989)

    Article  Google Scholar 

  12. Dietvorst, B.J., Simmons, J.P., Massey, C.: Algorithm aversion: people erroneously avoid algorithms after seeing them err. J. Exp. Psychol. Gen. 144(1), 114–126 (2015)

    Article  Google Scholar 

  13. Eloundou, T., Manning, S., Mishkin, P., Rock, D.: GPTs are GPTs: an early look at the labor market impact potential of large language models. arXiv preprint arXiv:2303.10130 (2023)

  14. EU: Regulation EU 2016/679 of the european parliament and of the council of 27 april 2016, article 22. Official Journal of the European Union L 119 59 (2016)

    Google Scholar 

  15. Feuerriegel, S., Hartmann, J., Janiesch, C., Zschech, P.: Generative AI. Bus. Inf. Syst. Eng. 66(1), 111–126 (2023)

    Google Scholar 

  16. Fügener, A., Grahl, J., Gupta, A., Ketter, W.: Will humans-in-the-loop become borgs? Merits and pitfalls of working with AI. MIS Q. 45(3), 1527–1556 (2021)

    Article  Google Scholar 

  17. Hadi, M.U., et al.: A survey on large language models: applications, challenges, limitations, and practical usage. Authorea Preprints (2023)

    Google Scholar 

  18. Hevner, A.R., March, S.T., Park, J., Ram, S.: Design science in information systems research. MIS Q. 28(1), 75–105 (2004)

    Article  Google Scholar 

  19. Hsieh, H.F., Shannon, S.E.: Three approaches to qualitative content analysis. Qual. Health Res. 15(9), 1277–1288 (2005)

    Article  Google Scholar 

  20. Huff, K.E., Lesser, V.R.: A plan-based intelligent assistant that supports the software development. In: Proceedings of the Third ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments, pp. 97–106 (1988)

    Google Scholar 

  21. Iivari, J., Rotvit Perlt Hansen, M., Haj-Bolouri, A.: A proposal for minimum reusability evaluation of design principles. Eur. J. Inf. Syst. 30(3), 286–303 (2021)

    Google Scholar 

  22. Jo, H., Bang, Y.: Analyzing chatgpt adoption drivers with the toek framework. Sci. Rep. 13(1), 22606 (2023)

    Article  Google Scholar 

  23. Kuechler, B., Vaishnavi, V.: On theory development in design science research: anatomy of a research project. Eur. J. Inf. Syst. 17(5), 489–504 (2008)

    Article  Google Scholar 

  24. Liang, J.T., Yang, C., Myers, B.A.: A large-scale survey on the usability of AI programming assistants: successes and challenges. In: 2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE), pp. 605–617. IEEE Computer Society (2023)

    Google Scholar 

  25. Meth, H., Mueller, B., Maedche, A.: Designing a requirement mining system. J. Assoc. Inf. Syst. 16(9), 799–837 (2015)

    Google Scholar 

  26. Myers, B.A., Ko, A.J., LaToza, T.D., Yoon, Y.: Programmers are users too: human-centered methods for improving programming tools. Computer 49(7), 44–52 (2016)

    Article  Google Scholar 

  27. Nguyen-Duc, A., et al.: Generative artificial intelligence for software engineering–a research agenda. arXiv preprint arXiv:2310.18648 (2023)

  28. Peng, S., Kalliamvakou, E., Cihon, P., Demirer, M.: The impact of AI on developer productivity: evidence from GitHub Copilot. arXiv preprint arXiv:2302.06590 (2023)

  29. Plant, R., Giuffrida, V., Gkatzia, D.: You are what you write: preserving privacy in the era of large language models. arXiv preprint arXiv:2204.09391 (2022)

  30. Rai, A.: Explainable AI: from black box to glass box. J. Acad. Mark. Sci. 48, 137–141 (2020)

    Article  Google Scholar 

  31. Ross, S.I., Martinez, F., Houde, S., Muller, M., Weisz, J.D.: The programmer’s assistant: Conversational interaction with a large language model for software development. In: Proceedings of the 28th International Conference on Intelligent User Interfaces, pp. 491–514 (2023)

    Google Scholar 

  32. Ruhe, G.: Software engineering decision support-a new paradigm for learning software organizations. In: Henninger, S., Maurer, F. (eds.) LSO 2002. LNCS, vol. 2640, pp. 104–113. Springer, Heidelberg (2002). https://doi.org/10.1007/978-3-540-40052-3_10

    Chapter  Google Scholar 

  33. Shi, W., Cui, A., Li, E., Jia, R., Yu, Z.: Selective differential privacy for language modeling. arXiv preprint arXiv:2108.12944 (2021)

  34. Susarla, A., Gopal, R., Thatcher, J.B., Sarker, S.: The Janus effect of generative AI: charting the path for responsible conduct of scholarly activities in information systems. Inf. Syst. Res. 34(2), 399–408 (2023)

    Article  Google Scholar 

  35. Wessel, M., Adam, M., Benlian, A., Thies, F.: Generative AI and its transformative value for digital platforms. J. Manag. Inf. Syst. (2023)

    Google Scholar 

  36. Winograd, A.: Loose-lipped large language models spill your secrets: the privacy implications of large language models. Harvard J. Law Technol. 36(2) (2023)

    Google Scholar 

  37. Xu, Y., Gong, M., Chen, J., Liu, T., Zhang, K., Batmanghelich, K.: Generative-discriminative complementary learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 6526–6533 (2020)

    Google Scholar 

  38. Zacharias, J., von Zahn, M., Chen, J., Hinz, O.: Designing a feature selection method based on explainable artificial intelligence. Electron. Mark. 32(4), 2159–2184 (2022)

    Article  Google Scholar 

  39. Zhao, W.X., et al.: A survey of large language models. arXiv preprint arXiv:2303.18223 (2023)

Download references

Author information

Authors and Affiliations

  1. Chair of Information Systems and Information Management, Goethe University Frankfurt, Frankfurt, Germany

    Johannes Chen & Jan Zacharias

Authors
  1. Johannes Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Zacharias
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Johannes Chen .

Editor information

Editors and Affiliations

  1. Temple University, Philadelphia, PA, USA

    Munir Mandviwalla

  2. University of Kassel, Kassel, Germany

    Matthias Söllner

  3. University of Jyväskylä, Jyväskylä, Finland

    Tuure Tuunanen

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chen, J., Zacharias, J. (2024). Design Principles for Collaborative Generative AI Systems in Software Development. In: Mandviwalla, M., Söllner, M., Tuunanen, T. (eds) Design Science Research for a Resilient Future. DESRIST 2024. Lecture Notes in Computer Science, vol 14621. Springer, Cham. https://doi.org/10.1007/978-3-031-61175-9_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-61175-9_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-61174-2

  • Online ISBN: 978-3-031-61175-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation