The (Un)Answered Question: A Data Science Powered Music Experiment | SpringerLink
Skip to main content
Springer Nature Link
Log in

The (Un)Answered Question: A Data Science Powered Music Experiment

  • Conference paper
  • First Online:
Culture and Computing (HCII 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14717))

Included in the following conference series:

Abstract

This paper describes the intentions, setup, and live performance of a musical experiment that explores the complex intersection of human-technology interactions, music, and data collection. It brings art and data science together through a novel experimental music installation. The interdisciplinary project “The (Un)Answered Question: A Data Science Powered Music Experiment” explored integrating data science and biomedical imaging techniques with theatrical and compositional ideas. This combination leads to the creation of interactive music. Gestural interfaces and sensory input devices translate physiological behavior into music through digital signal processing. Ralph Waldo Emerson’s poem “The Sphynx” and Charles Ives’ composition “The Unanswered Question” serve as foundational elements to create a live remix of the original music using biometric data from performers and an audience of 180 people. The audience became a powerful instrument of musical expression. Each live performance was experiential and unique, depending on the different people involved.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 14871
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 18589
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.youtube.com/watch?v=kkaOz48cq2g.

  2. 2.

    https://www.mi.com/global/miband/.

  3. 3.

    registration number EA1/256/19, Ethikkomission, Ethikausschuss am Campus Charité - Mitte, Berlin, Germany.

  4. 4.

    MAGNETOM, Siemens Healthineers, Erlangen, Germany.

  5. 5.

    RFPA, Stolberg HF-Technik AG, Stolberg-Vicht, Germany.

  6. 6.

    EasyACT, MRI.TOOLS GmbH, Berlin, Germany.

  7. 7.

    https://inscore.grame.fr/.

  8. 8.

    https://guidodoc.grame.fr/.

  9. 9.

    http://www.cemfi.de/.

  10. 10.

    https://poets.org/poem/sphinx.

References

  1. Albaghli, R., Anderson, K.M.: A vision for heart rate health through wearables. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, UbiComp 2016, pp. 1101–1105. Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/2968219.2972715

  2. Baumol, W.J.: Performing Arts. In: Eatwell, J., Milgate, M., Newman, P. (eds.) The World of Economics. The New Palgrave. Palgrave Macmillan, London, pp. 544–548 (1991). https://doi.org/10.1007/978-1-349-21315-3_72

  3. Bogaert, J., Dymarkowski, S., Taylor, A.M., Muthurangu, V.: Clinical cardiac MRI. Springer Science & Business Media (2012)

    Google Scholar 

  4. Borkenau, P., Ostendorf, F.: NEO-Fünf-Faktoren Inventar: Nach Costa u. McCrae; NEO-FFI. Hogrefe, Verlag f. Psychologie (2008)

    Google Scholar 

  5. Costa, P.T., McCrae, R.R.: Normal personality assessment in clinical practice: the NEO personality inventory. Psychol. Assess. 4(1), 5 (1992)

    Article  Google Scholar 

  6. Dagar, D., Hudait, A., Tripathy, H.K., Das, M.N.: Automatic emotion detection model from facial expression. In: 2016 International Conference on Advanced Communication Control and Computing Technologies (ICACCCT), pp. 77–85 (2016). https://doi.org/10.1109/ICACCCT.2016.7831605

  7. Dhakal, S.: MIBAND 4 - Python Library, October 2022. https://github.com/satcar77/miband4

  8. Dzedzickis, A., Kaklauskas, A., Bucinskas, V.: Human emotion recognition: review of sensors and methods 20(3), 592 (2020). https://doi.org/10.3390/s20030592, https://www.mdpi.com/1424-8220/20/3/592

  9. Eigentler, T.W., et al.: 32-channel self-grounded bow-tie transceiver array for cardiac MR at 7.0t. Magnetic Resonance in Medicine 86, 2862–2879 (2021). https://api.semanticscholar.org/CorpusID:235636158

  10. Harvey, I.: bluepy, May 2021. https://github.com/IanHarvey/bluepy

  11. Körner, A., et al.: Persönlichkeitsdiagnostik mit dem neo-fünf-faktoren-inventar: Die 30-item-kurzversion (neo-ffi-30). PPmP-Psychotherapie\(\cdot \) Psychosomatik\(\cdot \) Medizinische Psychologie 58(06), 238–245 (2008)

    Google Scholar 

  12. Niendorf, T., Barth, M., Kober, F., Trattnig, S.: From ultrahigh to extreme field magnetic resonance: where physics, biology and medicine meet 29(3), 309–311 (2016). https://doi.org/10.1007/s10334-016-0564-1

  13. Perra, M., Brinkman, T.: Seeing science: using graphics to communicate research 12(10), e03786 (2021). https://doi.org/10.1002/ecs2.3786, https://esajournals.onlinelibrary.wiley.com/doi/abs/10.1002/ecs2.3786

  14. Rani, J., Garg, K.: Emotion detection using facial expressions-a review. Int. J. Adv. Res. Comput. Sci. Softw. Eng. 4(4) (2014)

    Google Scholar 

  15. Serengil, S.I., Ozpinar, A.: Lightface: a hybrid deep face recognition framework. In: 2020 Innovations in Intelligent Systems and Applications Conference (ASYU), pp. 23–27. IEEE (2020). https://doi.org/10.1109/ASYU50717.2020.9259802

  16. Serengil, S.I., Ozpinar, A.: Hyperextended lightface: a facial attribute analysis framework. In: 2021 International Conference on Engineering and Emerging Technologies (ICEET), pp. 1–4. IEEE (2021). https://doi.org/10.1109/ICEET53442.2021.9659697

  17. Steinley, D.: K-means clustering: a half-century synthesis 59(1), 1–34 (2006). https://doi.org/10.1348/000711005X48266, https://bpspsychub.onlinelibrary.wiley.com/doi/abs/10.1348/000711005X48266

  18. Wright, M.J., Freed, A.: Open soundcontrol: a new protocol for communicating with sound synthesizers. In: International Conference on Mathematics and Computing (1997). https://api.semanticscholar.org/CorpusID:27393683

Download references

Acknowledgments

We acknowledge the Helmholtz Information & Data Science Academy (HIDA) for providing financial support that allowed a short-term research stay of Martin Hennecke at the German Aerospace Center (DLR) and at the Academy for Theatre and Digitality to work together with researchers from the Institute of Software Technology on the work described in this paper.

Author information

Authors and Affiliations

  1. German Aerospace Center (DLR), Institute for Software Technology, Linder Höhe, 51147, Cologne, Germany

    Lynn von Kurnatowski, Benjamin Wolff, Sophie Kernchen, Adriana Klapproth-Rieger, David Heidrich, Carina Haupt & Andreas Schreiber

  2. Max Delbrueck Center for Molecular Medicine (MDC), Berlin Ultrahigh Field Facility, 13125, Berlin, Germany

    Thoralf Niendorf

  3. Experimental and Clinical Research Center (ECRC), 13125, Berlin, Germany

    Thoralf Niendorf

  4. Falling Walls Foundation gGmbH, 10969, Berlin, Germany

    Andreas Kosmider

  5. Academy for Theatre and Digitality, 44137, Dortmund, Germany

    Marcus Lobbes

  6. Saarländisches Staatstheater, 66111, Saarbrücken, Germany

    Martin Hennecke

Authors
  1. Lynn von Kurnatowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benjamin Wolff
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sophie Kernchen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Adriana Klapproth-Rieger
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David Heidrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Carina Haupt
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Andreas Schreiber
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Thoralf Niendorf
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Andreas Kosmider
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Marcus Lobbes
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Martin Hennecke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lynn von Kurnatowski .

Editor information

Editors and Affiliations

  1. Eindhoven University of Technology, Eindhoven, The Netherlands

    Matthias Rauterberg

Ethics declarations

Disclosure of Interests

Author Martin Hennecke received scholarship grants from the Helmholtz Information & Data Science Academy (HIDA).

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

von Kurnatowski, L. et al. (2024). The (Un)Answered Question: A Data Science Powered Music Experiment. In: Rauterberg, M. (eds) Culture and Computing. HCII 2024. Lecture Notes in Computer Science, vol 14717. Springer, Cham. https://doi.org/10.1007/978-3-031-61147-6_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-61147-6_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-61146-9

  • Online ISBN: 978-3-031-61147-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation