Multimodal exploration in elementary music classroom | Journal on Multimodal User Interfaces Skip to main content
Springer Nature Link
Log in

Multimodal exploration in elementary music classroom

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

Vibrotactile feedback can support and enhance various musical learning processes of both children and adults, thus improving their musical experience. However, until now, to the best knowledge of the authors, only a few studies have focused on the auditory-tactile music perception in childhood. In this paper, we evaluate a bimodal auditory-tactile teaching approach for elementary school children. The experiment was designed based on the two-group-pretest-posttest design. Children allocated to the experimental group were subjected to auditory-tactile music training, while those allocated to the control-group completed the same training sessions stimulated by audio-only signals. Both groups received common, especially designed musical activities, divided into sixteen sessions. In order to evaluate the experimental procedure, children completed an auditory ability test in three different timepoints: before, half-way through, and at the end of the study. Data from 60 children were analyzed. The main aim of the study was to explore beat sensitivity in simple and complex music metrical structures and examine whether auditory-tactile stimulation could enhance the children’s ability to recognize beat within various musical contexts. Results demonstrate notable performance improvements within the experimental group, suggesting that a multimodal music experience can improve beat induction abilities of both simple and complex meters for elementary school children.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Einarson K, Trainor L (2016) Hearing the beat: young children’s perceptual sensitivity to beat alignment varies according to metric structure. Music Percept Interdiscip J 34:56–70. https://doi.org/10.1525/.2016.34.1.56

    Article  Google Scholar 

  2. Nozaradan S, Peretz I, Missal M, Mouraux A (2011) Tagging the neuronal entrainment to beat and meter. J Neurosci 31(28):10234–40. https://doi.org/10.1523/JNEUROSCI.0411-11.2011

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Repp BH, Su YH (2013) Sensorimotor synchronization: a review of recent research (2006–2012). Psychon Bull Rev 20(3):403–52. https://doi.org/10.3758/s13423-012-0371-2

    Article  PubMed  Google Scholar 

  4. Patel AD, Iversen JR (2014) The evolutionary neuroscience of musical beat perception: the action simulation for auditory prediction (ASAP) hypothesis. Front Syst Neurosci 13(8):57. https://doi.org/10.3389/fnsys.2014.00057

    Article  Google Scholar 

  5. Burger B, Thompson M, Luck G, Saarikallio S, Toiviainen P (2012) Music moves us: beat-related musical features influence regularity of music-induced movement. In: Cambouropoulos E, Tsougras C, Mavromatis P, and Pastiadis K (eds) Proceedings of the 12th international conference on music perception and cognition (ICMPC) and the 8th triennial conference of the european society for the cognitive sciences of music (ESCOM) July 23–28, 2012 Thessaloniki, Greece. Aristotle University of Thessaloniki, pp 183–187. http://icmpc-escom2012.web.auth.gr/sites/default/files/papers/183_Proc.pdf

  6. Honing H (2012) Without it no music: beat induction as a fundamental musical trait. Ann N Y Acad Sci 1252:85–91. https://doi.org/10.1111/j.1749-6632.2011.06402.x

    Article  ADS  PubMed  Google Scholar 

  7. Haumann NT, Vuust P, Bertelsen F, Garza-Villarreal EA (2018) Influence of musical enculturation on brain responses to metric deviants. Front Neurosci 18(12):218. https://doi.org/10.3389/fnins.2018.00218

    Article  Google Scholar 

  8. Kalender B, Trehub SE, Schellenberg EG (2012) Cross-cultural differences in meter perception. Psychol Res 77(2):196–203. https://doi.org/10.1007/s00426-012-0427-y

    Article  PubMed  Google Scholar 

  9. Van der Weij B, Pearce MT, Honing H (2017) A probabilistic model of meter perception: simulating enculturation. Front Psychol 8:824. https://doi.org/10.3389/fpsyg.2017.00824

    Article  PubMed  PubMed Central  Google Scholar 

  10. Large EW, Snyder JS (2009) Pulse and meter as neural resonance. Ann N Y Acad Sci 1169:46–57. https://doi.org/10.1111/j.1749-6632.2009.04550.x

    Article  ADS  PubMed  Google Scholar 

  11. Grahn JA, Henry MJ, McAuley JD (2011) FMRI investigation of cross-modal interactions in beat perception: audition primes vision, but not vice versa. Neuroimage 54(2):1231–43. https://doi.org/10.1016/j.neuroimage.2010.09.033

    Article  PubMed  Google Scholar 

  12. Elliott MT, Wing AM, Welchman AE (2010) Multisensory cues improve sensorimotor synchronisation. Eur J Neurosci 31(10):1828–35. https://doi.org/10.1111/j.1460-9568.2010.07205.x

    Article  CAS  PubMed  Google Scholar 

  13. Grondin S, McAuley D (2009) Duration discrimination in crossmodal sequences. Perception 38(10):1542–59. https://doi.org/10.1068/p6359

    Article  PubMed  Google Scholar 

  14. Kosonen K, Raisamo R (2006) Rhythm perception through different modalities. In: Proceedings of Eurohaptics 2006, pp 365–370

  15. Lerens E, De Volder A, Araneda R (2013) Perception of rhythm through auditory, vibro-tactile and visual stimulations: an fMRI study. Multisens Res 26:117–117. https://doi.org/10.1163/22134808-000S0085

    Article  Google Scholar 

  16. Butler JS, Foxe JJ, Fiebelkorn IC, Mercier MR, Molholm S (2012) Multisensory representation of frequency across audition and touch: high density electrical mapping reveals early sensory-perceptual coupling. J Neurosci 32(44):15338–44. https://doi.org/10.1523/JNEUROSCI.1796-12.2012

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Jokiniemi M, Raisamo R, Lylykangas J, Surakka V (2008) Crossmodal rhythm perception. In: Pirhonen A, Brewster S (eds) Haptic and audio interaction design, vol 5270. HAID 2008. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87883-4_12

    Chapter  Google Scholar 

  18. Godde B, Stauffenberg B, Spengler F, Dinse HR (2000) Tactile coactivation-induced changes in spatial discrimination performance. J Neurosci 20(4):1597–604. https://doi.org/10.1523/JNEUROSCI.20-04-01597.2000

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Brochard R, Touzalin P, Després O, Dufour A (2008) Evidence of beat perception via purely tactile stimulation. Brain Res. 5(1223):59–64. https://doi.org/10.1016/j.brainres.2008.05.050

    Article  CAS  Google Scholar 

  20. Bach-y-Rita P (2004) Tactile sensory substitution studies. Ann N Y Acad Sci 1013(1):83–91. https://doi.org/10.1196/annals.1305.006

    Article  ADS  PubMed  Google Scholar 

  21. Nanayakkara SC, Wyse L, Ong SH, Taylor EA (2013) Enhancing musical experience for the hearing-impaired using visual and haptic displays. Hum Comput Interact 28(2):115–160. https://doi.org/10.1080/07370024.2012.697006

    Article  Google Scholar 

  22. Jack R, McPherson DA, Stockman DT (2015) Designing tactile musical devices with and for deaf users: a case study. In: Proceedings of the international conference on the multimodal experience of music, pp 23–25

  23. Tranchant P, Shiell MM, Giordano M, Nadeau A, Peretz I, Zatorre RJ (2017) Feeling the beat: bouncing synchronization to vibrotactile music in hearing and early deaf people. Front Neurosci 12(11):507. https://doi.org/10.3389/fnins.2017.00507

    Article  Google Scholar 

  24. Phillips-Silver J, Toiviainen P, Gosselin N, Turgeon C, Lepore F, Peretz I (2015) Cochlear implant users move in time to the beat of drum music. Hear Res 321:25–34. https://doi.org/10.1016/j.heares.2014.12.007

    Article  PubMed  Google Scholar 

  25. Petry B, Illandara T, Elvitigala D, Nanayakkara S (2018) Supporting rhythm activities of deaf children using music-sensory-substitution systems. In: Proceedings of the 2018 CHI conference on human factors in computing systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, pp 1–10. https://doi.org/10.1145/3173574.3174060

  26. Miura S, Sugimoto M (2006) Supporting children’s rhythm learning using vibration devices. In: CHI ’06 extended abstracts on human factors in computing systems (CHI EA ’06). Association for Computing Machinery, New York, NY, USA, pp 1127–1132. https://doi.org/10.1145/1125451.1125664

  27. Trainor LJ, Corrigall KA (2010) Music acquisition and effects of musical experience. In: Jones MR, Fay RR, Popper AN (eds) Music perception. Springer, New York, pp 89–127. https://doi.org/10.1007/978-1-4419-6114-3_4

    Chapter  Google Scholar 

  28. Reifinger JL (2006) Skill development in rhythm perception and performance: a review of literature. Update Appl Res Music Educ 25(1):15–27. https://doi.org/10.1177/87551233060250010103

    Article  Google Scholar 

  29. Savage PE, Brown S, Sakai E, Currie TE (2015) Statistical universals reveal the structures and functions of human music. Proc Natl Acad Sci U S A 112(29):8987–92. https://doi.org/10.1073/pnas.1414495112

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  30. Hannon EE, Trainor LJ (2007) Music acquisition: effects of enculturation and formal training on development. Trends Cogn Sci 11(11):466–72. https://doi.org/10.1016/j.tics.2007.08.008

    Article  PubMed  Google Scholar 

  31. Soley G, Hannon EE (2010) Infants prefer the musical meter of their own culture: a cross-cultural comparison. Dev Psychol 46(1):286–92. https://doi.org/10.1037/a0017555

    Article  PubMed  Google Scholar 

  32. Cirelli LK, Spinelli C, Nozaradan S, Trainor LJ (2016) Measuring neural entrainment to beat and meter in infants: effects of music background. Front Neurosci 24(10):229. https://doi.org/10.3389/fnins.2016.00229

    Article  Google Scholar 

  33. Trehub SE, Hannon EE (2006) Infant music perception: Domain-general or domain-specific mechanisms? Cognition 100(1):73–99. https://doi.org/10.1016/j.cognition.2005.11.006

    Article  PubMed  Google Scholar 

  34. Trehub SE (2003) The developmental origins of musicality. Nat Neurosci 6(7):669–73. https://doi.org/10.1038/nn1084

    Article  CAS  PubMed  Google Scholar 

  35. Wallentin M, Nielsen AH, Friis-Olivarius M, Vuust C, Vuust P (2010) The musical ear test, a new reliable test for measuring musical competence. Learn Individ Differ 20(3):188–196. https://doi.org/10.1016/j.lindif.2010.02.004

    Article  Google Scholar 

  36. Vanden Hannon EE, Bosch der Nederlanden CM, Tichko P (2012) Effects of perceptual experience on children’s and adults’ perception of unfamiliar rhythms. Ann N Y Acad Sci 1252:92–9. https://doi.org/10.1111/j.1749-6632.2012.06466.x

    Article  ADS  Google Scholar 

  37. Trainor LJ, Marie C, Gerry D, Whiskin E, Unrau A (2012) Becoming musically enculturated: effects of music classes for infants on brain and behavior. Ann N Y Acad Sci 1252:129–38. https://doi.org/10.1111/j.1749-6632.2012.06462.x

    Article  ADS  PubMed  Google Scholar 

  38. Michaelis K, Wiener M, Thompson JC (2014) Passive listening to preferred motor tempo modulates corticospinal excitability. Front Hum Neurosci 24(8):252. https://doi.org/10.3389/fnhum.2014.00252

    Article  Google Scholar 

  39. Drake C, Ben El Heni J (2003) Synchronizing with music: intercultural differences. Ann N Y Acad Sci 999:429–37. https://doi.org/10.1196/annals.1284.053

    Article  ADS  PubMed  Google Scholar 

  40. Drake C, Penel A, Bigand E (2000) Tapping in time with mechanically and expressively performed music. Music Percept 18(1):1–23. https://doi.org/10.2307/40285899

    Article  Google Scholar 

  41. Hannon EE, Soley G, Ullal S (2012) Familiarity overrides complexity in rhythm perception: a cross-cultural comparison of American and Turkish listeners. J Exp Psychol Hum Percept Perform 38(3):543–8. https://doi.org/10.1037/a0027225

    Article  PubMed  Google Scholar 

  42. Cameron DJ, Bentley J, Grahn JA (2015) Cross-cultural influences on rhythm processing: reproduction, discrimination, and beat tapping. Front Psychol 9(6):366. https://doi.org/10.3389/fpsyg.2015.00366

    Article  Google Scholar 

  43. Kalender B, Trehub SE, Schellenberg EG (2012) Cross-cultural differences in meter perception. Psychol Res 77(2):196–203. https://doi.org/10.1007/s00426-012-0427-y

    Article  PubMed  Google Scholar 

  44. Snyder J, Hannon E, Edward L, Christiansen M (2006) Synchronization and continuation tapping to complex meters. Music Percept 24:135–146. https://doi.org/10.1525/mp.2006.24.2.135

    Article  Google Scholar 

  45. Morrison SJ, Demorest SM, Stambaugh LA (2008) Enculturation effects in music cognition: the role of age and music complexity. J. Res. Music Educ 56(2):118–129

    Article  Google Scholar 

  46. Trainor LJ, Hannon EE (2013) Musical development. In: Deutsch D (ed) The psychology of music. Elsevier Academic Press, Amsterdam, pp 423–497. https://doi.org/10.1016/B978-0-12-381460-9.00011-0

    Chapter  Google Scholar 

  47. Law LNC, Zentner M (2012) Assessing musical abilities objectively: construction and validation of the profile of music perception skills. PLoS one 7(12):e52508. https://doi.org/10.1371/journal.pone.0052508

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  48. Fujii S, Schlau G (2013) The harvard beat assessment test (H-BAT): a battery for assessing beat perception and production and their dissociation. Front Hum Neurosci 7:771. https://doi.org/10.3389/fnhum.2013.00771

    Article  PubMed  PubMed Central  Google Scholar 

  49. Iversen J, Patel A (2008) The beat alignment test (BAT): surveying beat processing abilities in the general population. In: Proceedings of the 10th international conference on music perception and cognition (ICMPC10)

  50. Dauvergne C, Begel V, Benoit CE, Kotz S, Dalla Bella S (2015) Battery for the assessment of auditory sensorimotor and timing abilities (BAASTA): a rehabilitation perspective. Ann Phys Rehabil Med 58:e72

    Article  Google Scholar 

  51. Dalla Bella S, Farrugia N, Benoit CE et al (2017) BAASTA: battery for the assessment of auditory sensorimotor and timing abilities. Behav Res 49:1128–1145. https://doi.org/10.3758/s13428-016-0773-6

    Article  Google Scholar 

  52. Gallace A, Tan HZ, Spence C (2007) The body surface as a communication system: the state of the art after 50 years. Presence Teleoperators Virtual Environ 16:655–676. https://doi.org/10.1162/pres.16.6.655

    Article  Google Scholar 

  53. Giordano M, Wanderley MM (2015) Follow the tactile metronome: vibrotactile stimulation for tempo synchronization in music performance. In: Proceedings of the SMC conference, Maynooth, Ireland

  54. Giordano M, Sullivan J, Wanderley MM (2018) Design of vibrotactile feedback and stimulation for music performance. In: Papetti S, Saitis C (eds) Musical haptics. Springer, New York, pp 193–214. https://doi.org/10.1007/978-3-319-58316-7_10

    Chapter  Google Scholar 

  55. Gunther E, Davenport G, O’Modhrain S (2002) Cutaneous grooves: composing for the sense of touch. In: Proceedings of the 2002 conference on new instruments for musical expression (NIME-02), Dublin, Ireland, May 24–26, vol 32. pp 37–43

  56. Sakuragi R, Ikeno S, Okazaki R, Kajimoto H (2015) CollarBeat: whole body vibrotactile presentation via the collarbone to enrich music listening experience. In: ICAT-EGVE 2015 - international conference on artificial reality and telexistence and eurographics symposium on virtual environments 6 pages. https://doi.org/10.2312/EGVE.20151321

  57. Benali-Khoudja M, Hafez M, Alexandre JM, Kheddar A (2004) Tactile interfaces: a state-of-the-art survey. Information Systems Research - ISR

  58. Cibrian FL, Pena O, Ortega D, Tentori M (2017) BendableSound: an elastic multisensory surface using touch-based interactions to assist children with severe autism during music therapy. Int J Hum Comput Stud 107:22–37. https://doi.org/10.1016/j.ijhcs.2017.05.003

    Article  Google Scholar 

  59. Ifukube T (2017) Tactile stimulation methods for the deaf and/or blind. In: Ifukube T (ed) Sound-based assistive technology support to hearing, speaking and seeing. Springer, New York, pp 111–143. https://doi.org/10.1007/978-3-319-47997-2_4

    Chapter  Google Scholar 

  60. Choi S, Kuchenbecker KJ (2013) Vibrotactile display: perception, technology, and applications. Proc IEEE 101:2093–2104. https://doi.org/10.1109/JPROC.2012.2221071

    Article  Google Scholar 

  61. Miura S, Sugimoto M (2005) T-RHYTHM: a system for supporting rhythm learning by using tactile devices. In: IEEE international workshop on wireless and mobile technologies in education (WMTE’05). Tokushima, Japan, pp 5–268. https://doi.org/10.1109/WMTE.2005.63

  62. Baijal A, Kim J, Branje C, Russo F, Fels DI (2012) Composing vibrotactile music: a multisensory experience with the Emoti-chair. In: 2012 IEEE haptics symposium (HAPTICS), pp 509–515. https://doi.org/10.1109/HAPTIC.2012.6183839

  63. Merchel S, Altinsoy ME (2013) Auditory-tactile music perception. J Acoust Soc Am 133(5 Supplement):3256. https://doi.org/10.1121/1.4805254

    Article  ADS  Google Scholar 

  64. Altinsoy E (2006) Auditory tactile interaction in virtual environments. Ruhr University Bochum, pp 1-166

  65. Merchel S, Altinsoy E (2009) Vibratory and acoustical factors in multimodal reproduction of concert DVDs. In: Altinsoy ME, Jekosch U, Brewster S (eds) Haptic and audio interaction design. Springer, Berlin, pp 119–127

    Chapter  Google Scholar 

  66. Leman M, Maes PJ (2015) The role of embodiment in the perception of music. Empir Musicol Rev. https://doi.org/10.18061/emr.v9i3-4.4498

    Article  Google Scholar 

  67. Maes PJ, Leman M, Palmer C, Wanderley MM (2014) Action-based effects on music perception. Front Psychol 3(4):1008. https://doi.org/10.3389/fpsyg.2013.01008

    Article  Google Scholar 

  68. Crippen M (2017) Embodied cognition and perception: Dewey. science and skepticism. Contemp Pragmatism 14:112–134. https://doi.org/10.1163/18758185-01401007

    Article  Google Scholar 

  69. Macedonia M (2019) Embodied learning: why at school the mind needs the body. Front Psychol 10:2098. https://doi.org/10.3389/fpsyg.2019.02098

  70. Merchel S, Altinsoy ME (2018) Auditory-tactile experience of music. In: Papetti S, Saitis C (eds) Musical haptics. Springer Series on Touch and Haptic Systems. Springer, Cham. https://doi.org/10.1007/978-3-319-58316-7_7

  71. Dimitrov DM, Rumrill PD Jr (2003) Pretest-posttest designs and measurement of change. Work 20(2):159–65

    PubMed  Google Scholar 

  72. Juntunen M (2006) Exploring and learning music through embodied experiences. Music and development - challenges for music education : In: The proceedings of the first European conference on developmental psychology of music: 17–19 November 2005, University of Jyvaskyla, Department of Music, Finland, pp 273–276

  73. Kivijärvi S, Sutela K, Ahokas JR (2016) A conceptual discussion of embodiment in special music education: Dalcroze Eurhythmics as a case. Approaches Interdiscip J Music Ther 8:169–178

    Google Scholar 

  74. Parianen Lesemann FH, Reuter EM, Godde B (2015) Tactile stimulation interventions: influence of stimulation parameters on sensorimotor behavior and neurophysiological correlates in healthy and clinical samples. Neurosci Biobehav Rev 51:126–137. https://doi.org/10.1016/j.neubiorev.2015.01.005

    Article  PubMed  Google Scholar 

  75. Einarson KM, Trainor LJ (2015) The effect of visual information on young children’s perceptual sensitivity to musical beat alignment. Timing Time Percept 3(1–2):88–101. https://doi.org/10.1163/22134468-03002039

    Article  Google Scholar 

  76. Provasi J, Anderson DI, Barbu-Roth M (2014) Rhythm perception, production, and synchronization during the perinatal period. Front Psychol 18(5):1048. https://doi.org/10.3389/fpsyg.2014.01048

    Article  Google Scholar 

  77. Kirschner S, Tomasello M (2009) Joint drumming: social context facilitates synchronization in preschool children. J Exp Child Psychol 102(3):299–314. https://doi.org/10.1016/j.jecp.2008.07.005

    Article  PubMed  Google Scholar 

  78. Barros CG, Swardfager W, Moreno S, Bortz G, Ilari B, Jackowski AP, Ploubidis G, Little TD, Lamont A, Cogo-Moreira H (2017) Assessing music perception in young children: evidence for and psychometric features of the M-factor. Front Neurosci 11:18. https://doi.org/10.3389/fnins.2017.00018

    Article  PubMed  PubMed Central  Google Scholar 

  79. Asztalos K, Csapó B (2017) Development of musical abilities: cross-sectional computer-based assessments in educational contexts. Psychol Music 45(5):682–698. https://doi.org/10.1177/0305735616678055

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Music Studies, National Kapodistrian University of Athens, University Campus, Zografou, 15784, Athens, Greece

    Martha Papadogianni & Areti Andreopoulou

  2. Institute of Acoustics and Speech Communication, Technische Universität Dresden Fakultät ET u. IT IAS, Helmholtzstraße 10, 01069, Dresden, Germany

    Martha Papadogianni & Ercan Altinsoy

Authors
  1. Martha Papadogianni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ercan Altinsoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Areti Andreopoulou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martha Papadogianni.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Papadogianni, M., Altinsoy, E. & Andreopoulou, A. Multimodal exploration in elementary music classroom. J Multimodal User Interfaces 18, 55–68 (2024). https://doi.org/10.1007/s12193-023-00420-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-023-00420-x

Keywords

Search

Navigation