Abstract
The economic growth and social transformation in the 21st century are hardly based on creativity. To help with the development of this skill, the concept of Creativity Support Tools (CST) was proposed. In this paper, we introduce Musical Brush (MB), an artistic mobile application whose main focus is to allow novices to improvise music while creating drawings. We investigated different types of interactions and audiovisual feedbacks in the context of a mobile application that combines music with drawings in a natural way, measuring their impact on creativity support. In this study, we tested different user interactions with real-time sound generation, including 2D drawings, three-dimensional device movements, and visual representations on Augmented Reality (AR). A user study was conducted to explore the support for creativity of each setup. Results showed the suitability of the association of Musical Brush with augmented reality for creating sounds and drawings as a tool that supports the exploration and expressiveness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Carroll, E.A., Latulipe, C.: Triangulating the personal creative experience: self-report, external judgments, and physiology. In: Proceedings of Graphics Interface 2012, GI ’12, pp. 53–60. Canadian Information Processing Society (2012)
Cherry, E., Latulipe, C.: Quantifying the creativity support of digital tools through the creativity support index. ACM Trans. Comput.-Hum. Interact. 21(4), 1–25 (2014)
Farbood, M., Pasztor, E., Jennings, K.: Hyperscore: a graphical sketchpad for novice composers. IEEE Comput. Graph. Appl. 24(1), 50–54 (2004)
Fels, S., Mase, K.: Iamascope: a graphical musical instrument. Comput. Graph. 23(2), 277–286 (1999)
Florida, R.L.: The flight of the creative class: the new global competition for talent. HarperBusiness (2005)
Hunt, A., Kirk, R.: Mapping strategies for musical performance. Trends Gestural Control Music 21(2000), 231–258 (2000)
Hunt, A., Wanderley, M.M.: Mapping performer parameters to synthesis engines. Organised Sound 7(02), 97 (2002)
Ione, A, Mitchell, W.J., Inouye, A.S., Blumenthal, M.S (eds.): Beyond productivity: information technology, innovation, and creativity, illus. Paper, Leonardo, vol. 37, pp. 408–410. The National Academies Press, Washington (2004). ISBN: 0-309-08868-268
Jordà, S., Geiger, G., Alonso, M., Kaltenbrunner, M.: The reacTable. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction - TEI 2007, p. 139. ACM Press (2007)
Karjalainen, M., Mäki-patola, T., Kanerva, A., Huovilainen, A.: Virtual air guitar. J. Audio Eng. Soc. 54(10), 964–980 (2006)
Lohner, H.: The UPIC system: a user’s report. Comput. Music J. 10(4), 42 (1986)
Lyons, M., Fels, S.: Creating new interfaces for musical expression. In: SIGGRAPH Asia 2013 Courses on - SA ’13, pp. 1–164. ACM Press (2013)
Mäki-Patola, T., Laitinen, J., Kanerva, A., Takala, T.: Experiments with Virtual Reality Instruments. Technical report (2004)
Martin, C.P., Tørresen, J.: MicroJam: an app for sharing tiny touch-screen performances. In: Proceedings of the International Conference on New Interfaces for Musical Expression, pp. 495–496. Aalborg University Copenhagen (2017)
Naylor, T.D., Florida, R.: The rise of the creative class: and how it’s transforming work, leisure, community and everyday life. Can. Public Policy/Anal. de Politiques 29(3), 378 (2003)
Nymoen, K., Glette, K., St, S., Skogstad, S., Torresen, J., Jensenius, A.R.: Searching for Cross-Individual Relationships between Sound and Movement Features using an SVM Classifier. Technical report (2010)
Paradiso, J.: Electronic music: new ways to play. IEEE Spectr. 34(12), 18–30 (1997)
Renaud, A., Charbonnier, C., Chagué, S.: 3DinMotion a mocap based interface for real time visualisation and sonification of multi-user interactions. In: NIME, pp. 495–496 (2014)
Serafin, S., Erkut, C., Kojs, J., Nilsson, N.C., Nordahl, R.: Virtual reality musical instruments: state of the art, design principles, and future directions. Comput. Music J. 40(3), 22–40 (2016)
Shneiderman, B.: Ben: creativity support tools. Commun. ACM 45(10), 116–120 (2002)
Shneiderman, B.: Ben: creativity support tools: accelerating discovery and innovation. Commun. ACM 50(12), 20–32 (2007)
Shneiderman, B., et al.: Creativity Support Tools: Report From a U.S. National Science Foundation Sponsored Workshop. Technical Report 2 (2006)
Wanderley, M., Battier, M.: Trends in gestural control of music. Ircam (2000)
Wanderley, M.M., Orio, N.: Evaluation of input devices for musical expression: Borrowing tools from HCI. Technical Report 3 (2002)
Wanderley, M., Depalle, P.: Gestural control of sound synthesis. Proc. IEEE 92(4), 632–644 (2004)
Wessel, D., Wright, M.: Problems and prospects for intimate musical control of computers. Comput. Music J. 26(3), 11–22 (2002)
Wessel, D.L.: Timbre space as a musical control structure. Comput. Music J. 3(2), 45 (1979)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Valer, R., Schramm, R., Nedel, L. (2020). Musical Brush: Exploring Creativity Through an AR-Based Tool for Sketching Music and Drawings. In: Magnenat-Thalmann, N., et al. Advances in Computer Graphics. CGI 2020. Lecture Notes in Computer Science(), vol 12221. Springer, Cham. https://doi.org/10.1007/978-3-030-61864-3_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-61864-3_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-61863-6
Online ISBN: 978-3-030-61864-3
eBook Packages: Computer ScienceComputer Science (R0)