Multi-level mental representations of written, auditory, and audiovisual text in children and adults | Cognitive Processing Skip to main content

Advertisement

Springer Nature Link
Log in

Multi-level mental representations of written, auditory, and audiovisual text in children and adults

  • Research Report
  • Published:
Cognitive Processing Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

We examined whether the comprehension of narrative texts differed between auditory, audiovisual, and written text presentations in a sample of 8- and 10-year-olds and adults. Based on multi-level theories of text comprehension that assume text comprehension to involve at least three levels of mental representation, we applied a sentence recognition task that enabled the separate assessment of the memory of the text surface, text base, and the situation model. Results indicate that 8-year-olds benefit from audiovisual and auditory text presentations in comparison with written text presentations in terms of their memory of situation model information. For 10-year-olds and adults, their text comprehension did not differ between audiovisual, auditory, and written text presentations. Additionally, the mode of text presentation had no effect on the memory of text surface and text base information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Artelt C et al. (2007) Förderung von Lesekompetenz—Expertise (BMBF ed.). BMBF, Bonn

  • Barsalou LW (1999) Language comprehension: Archival memory or preparation for situated action? Discourse Processes 28(1):61–80

    Article  Google Scholar 

  • Barsalou LW (2008) Grounded cognition. Annu Rev Psychol 59:617–645

    Article  PubMed  Google Scholar 

  • Bäuerlein K (2014) Leseverständnisdiagnostik in der Sekundarstufe-Theoretische Grundlagen sowie Konstruktion und empirische Erprobung der Lesetests LESEN 6-7 und LESEN 8-9. Würzburg University Press, Würzburg

    Google Scholar 

  • Beagles-Roos J, Gat I (1983) Specific impact of radio and television on children’s story comprehension. J Educ Psychol 75(1):128

    Article  Google Scholar 

  • Beentjes JW, Van Der Voort TH (1991) Children’s written accounts of televised and printed stories. Educ Technol Res Dev 39(3):15–26

    Article  Google Scholar 

  • Beentjes JW, Vooijs MW, Van Der Voort TH (1993) Children’s recall of televised and printed news as a function of test expectation. J Educ Televis 19(1):5–13

    Article  Google Scholar 

  • Brunyé TT et al (2010) You heard it here first: readers mentally simulate described sounds. Acta Physiol (Oxf) 135(2):209–215

    Google Scholar 

  • Byrne M, Curtis R (2000) Designing health communication: testing the explanations for the impact of communication medium on effectiveness. Br J Health Psychol 5(2):189–199

    Article  Google Scholar 

  • Carney RN, Levin JR (2002) Pictorial illustrations still improve students’ learning from text. Educ Psychol Rev 14(1):5–26

    Article  Google Scholar 

  • Christie B, Collyer J (2008) Do video clips add more value than audio clips? Presenting industrial research and development results using multimedia. Behav Inf Technol 27(5):395–405

    Article  Google Scholar 

  • Chun DM, Plass JL (1997) Research on text comprehension in multimedia environments. Lang Learn Technol 1(1):60–81

    Google Scholar 

  • Clement CA, Gentner D (1991) Systematicity as a selection constraint in analogical mapping. Cogn Sci 15(1):89–132

    Article  Google Scholar 

  • Corston R, Colman AM (1997) Modality of communication and recall of health-related information. J Health Psychol 2(2):185–194

    Article  CAS  PubMed  Google Scholar 

  • Danks JH, End LJ (1987) Processing strategies for reading and listening. In: Horowitz R, Samuels SJ (eds) Comprehending oral and written language. Academic Press, San Diego, pp 271–294

    Google Scholar 

  • DeFleur ML et al (1992) Audience recall of news stories presented by newspaper, computer, television and radio. Journal Mass Commun Q 69(4):1010–1022

    Google Scholar 

  • Diergarten AK, Nieding G (2015) Children’s and adults’ ability to build online emotional inferences during comprehension of audiovisual and auditory texts. J Cogn Dev 16(2):381–406

    Article  Google Scholar 

  • Diergarten AK, Nieding G (2016) Online emotional inferences in written and auditory texts: a study with children and adults. Read Writ 29(7):1383–1407. doi:10.1007/s11145-016-9642-z

    Article  Google Scholar 

  • Engelen JA et al (2011) Perceptual simulation in developing language comprehension. J Exp Child Psychol 110(4):659–675

    Article  PubMed  Google Scholar 

  • Faul F et al (2007) G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 39(2):175–191

    Article  PubMed  Google Scholar 

  • Field A (2013) Discovering statistics using IBM SPSS statistics. Sage, London

    Google Scholar 

  • Fletcher CR, Chrysler ST (1990) Surface forms, textbases, and situation models: recognition memory for three types of textual information. Discourse Processes 13(2):175–190

    Article  Google Scholar 

  • Fletcher JD, Tobias S (2005) Cognitive theory of multimedia learning. In: Mayer RE (ed) The Cambridge handbook of multimedia learning. Cambridge University Press, New York, pp 117–134

    Chapter  Google Scholar 

  • Florit E, Cain K (2011) The simple view of reading: is it valid for different types of alphabetic orthographies? Educ Psychol Rev 23(4):553–576

    Article  Google Scholar 

  • Furnham A, Gunter B (1985) Sex, presentation mode and memory for violent and non-violent news. J Educ Televis 11(2):99–105

    Article  Google Scholar 

  • Furnham A, Gunter B (1987) Effects of time of day and medium of presentation on immediate recall of violent and non-violent news. Appl Cogn Psychol 1(4):255–262

    Article  Google Scholar 

  • Furnham A, Williams C (1987) Remembering commercials presented in different media. J Educ Televis 13(2):115–124

    Article  Google Scholar 

  • Furnham A, Benson I, Gunter B (1989) Memory for television commercials as a function of the channel of communication. Soc Behav 3:105–112

    Google Scholar 

  • Gentner D (1983) Structure-mapping: a theoretical framework for analogy. Cogn Sci 7(2):155–170

    Article  Google Scholar 

  • Gernsbacher M (1990) Language comprehension as structure building. Erlbaum, Hillsdale

    Book  Google Scholar 

  • Gibbons J et al (1986) Young children’s recall and reconstruction of audio and audiovisual narratives. Child Dev 57(4):1014–1023

    Article  CAS  PubMed  Google Scholar 

  • Glenberg AM, Kaschak MP (2002) Grounding language in action. Psychon Bull Rev 9(3):558–565

    Article  PubMed  Google Scholar 

  • Gough PB, Tunmer WE (1986) Decoding, reading, and reading disability. Remedial and special education 7(1):6–10

    Article  Google Scholar 

  • Gough PB, Hoover WA, Peterson CL (1996) Some observations on a simple view of reading. In: Cornoldi C, Oakhill J (eds) Reading comprehension difficulties: processes and intervention. Erlbaum, Mahwah, pp 1–13

    Google Scholar 

  • Greenhoot AF, Semb PA (2008) Do illustrations enhance preschoolers’ memories for stories? Age-related change in the picture facilitation effect. J Exp Child Psychol 99(4):271–287

    Article  PubMed  Google Scholar 

  • Gunter B, Furnham A, Griffiths S (2000) Children’s memory for news: a comparison of three presentation media. Media Psychol 2(2):93–118

    Article  Google Scholar 

  • Guttmann J, Levin JR, Pressley M (1977) Pictures, partial pictures, and young children’s oral prose learning. J Educ Psychol 69(5):473–480

    Article  Google Scholar 

  • Hayes DS, Kelly SB, Mandel M (1986) Media differences in children’s story synopses: radio and television contrasted. J Educ Psychol 78(5):341

    Article  Google Scholar 

  • Hemforth B, Konieczny L (2008) Saetze und Texte verstehen und produzieren. In: Muesseler J, Prinz W (eds) Lehrbuch Allgemeine Psychologie. Spektrum, Heidelberg, pp 504–552

    Google Scholar 

  • Hildyard A, Olson DR (1978) Memory and inference in the comprehension of oral and written discourse. Discourse Processes 1(2):91–117

    Article  Google Scholar 

  • Hoover WA, Gough PB (1990) The simple view of reading. Read Writ 2(2):127–160

    Article  Google Scholar 

  • Horton WS, Rapp DN (2003) Out of sight, out of mind: occlusion and the accessibility of information in narrative comprehension. Psychon Bull Rev 10(1):104–110

    Article  PubMed  Google Scholar 

  • Isberner M-B et al (2013) Comprehending conflicting science-related texts: graphs as plausibility cues. Instr Sci 41(5):849–872

    Article  Google Scholar 

  • Johnson-Laird PN (1983) Mental models: towards a cognitive science of language, inference, and consciousness. Harvard University Press, Cambridge

    Google Scholar 

  • Kalyuga S et al (2003) The expertise reversal effect. Educ Psychol 38(1):23–31

    Article  Google Scholar 

  • Kendeou P et al (2005) Developing successful readers: building early comprehension skills through television viewing and listening. Early Childhood Educ J 33(2):91–98

    Article  Google Scholar 

  • Kendeou P et al (2008) Children’s inference generation across different media. J Res Read 31(3):259–272

    Article  Google Scholar 

  • Kintsch W (1988) The role of knowledge in discourse comprehension: a construction-integration model. Psychol Rev 95(2):163

    Article  CAS  PubMed  Google Scholar 

  • Kintsch W (1998) Comprehension: a paradigm for cognition. Cambridge University Press, Cambridge

    Google Scholar 

  • Kürschner C, Schnotz W, Eid M (2006) Konstruktion mentaler Repräsentationen beim Hör-und Leseverstehen. Z Medienpsychol 18(2):48–59

    Article  Google Scholar 

  • MacWhinney B (1999) The emergence of language from embodiment. In: MacWhinney B (ed) The emergence of language. Lawrence Erlabaum, Mahwah, pp 213–256

    Google Scholar 

  • Maier J, Richter T (2013) How nonexperts understand conflicting information on social science issues: the role of perceived plausibility and reading goals. J Media Psychol Theories Methods Appl 25(1):14

    Article  Google Scholar 

  • Marx H, Jungmann T (2000) Abhängigkeit der Entwicklung des Leseverstehens von Hörverstehen und grundlegenden Lesefertigkeiten im Grundschulalter: eine Prüfung des Simple View of reading-Ansatzes. Z für Entwicklungspsychol Pädagog Psychol 32(2):81–93

    Google Scholar 

  • Mayer RE (1997) Multimedia learning: are we asking the right questions? Educ Psychol 32(1):1–19

    Article  Google Scholar 

  • Mayer RE (2005) Principles for reducing extraneous processing in multimedia learning: coherence, signaling, redundancy, spatial contiguity, and temporal contiguity principles. In: Mayer RE (ed) The Cambridge handbook of multimedia learning. Cambridge University Press, New York, pp 183–200

    Chapter  Google Scholar 

  • Mayer RE, Heiser J, Lonn S (2001) Cognitive constraints on multimedia learning: when presenting more material results in less understanding. J Educ Psychol 93(1):187

    Article  Google Scholar 

  • McKoon G, Ratcliff R (1980) Priming in item recognition: the organization of propositions in memory for text. J Verbal Learn Verbal Behav 19(4):369–386

    Article  Google Scholar 

  • Narvaez D et al (2011) Are older adults more attuned to morally charged information? Exp Aging Res 37(4):398–434

    Article  PubMed  Google Scholar 

  • Nieding G (2006) Wie verstehen Kinder Texte? Die Entwicklung kognitiver Repräsentationen [How do children understand texts? The development of cognitive representations]. Pabst Science Publ, Lengerich

    Google Scholar 

  • Paivio A (1986) Mental representations. A dual coding approach. Oxford University Press, New York

    Google Scholar 

  • Peirce JW (2007) PsychoPy—psychophysics software in Python. J Neurosci Methods 162(1):8–13

    Article  PubMed  PubMed Central  Google Scholar 

  • Peirce JW (2009) Generating stimuli for neuroscience using PsychoPy. In: Frontiers in Neuroinformatics, vol 2, p 10. http://journal.frontiersin.org/article/10.3389/neuro.11.010.2008

  • Pezdek K, Lehrer A, Simon S (1984) The relationship between reading and cognitive processing of television and radio. Child Dev 55(6):2072–2082

    Article  Google Scholar 

  • Pollack I, Norman DA (1964) A non-parametric analysis of recognition experiments. Psychon Sci 1(1–12):125–126

    Article  Google Scholar 

  • Radvansky GA et al (2003) Aging and situation model updating. Aging Neuropsychol Cogn 10(2):158–166

    Article  Google Scholar 

  • Reder LM (1982) Plausibility judgments versus fact retrieval: alternative strategies for sentence verification. Psychol Rev 89(3):250

    Article  Google Scholar 

  • Ricci CM, Beal CR (2002) The effect of interactive media on children’s story memory. J Educ Psychol 94(1):138

    Article  Google Scholar 

  • Richter T, Christmann U (2002) Lesekompetenz: Prozessebenen und interindividuelle Unterschiede. In: Groeben N, Hurrelmann B (eds) Lesekompetenz: Bedingungen, Dimensionen, Funktionen. Juventa, Weinheim, pp 25–58

    Google Scholar 

  • Richter T, Zwaan RA (2009) Processing of color words activates color representations. Cognition 111(3):383–389

    Article  PubMed  Google Scholar 

  • Rideout V (2013) Zero to eight: children’s media use in America. Common Sense Media, San Francisco. https://www.commonsensemedia.org/file/zero-to-eight-2013pdf-0/download

  • Rosebrock C, Nix D (2011) Grundlagen der Lesedidaktik und der systematischen schulischen Leseförderung. Schneider-Verlag, Hohengehren

    Google Scholar 

  • Sachs JS (1974) Memory in reading and listening to discourse. Mem Cogn 2(1):95–100

    Article  CAS  Google Scholar 

  • Salomon G (1979) Media and symbol systems as related to cognition and learning. J Educ Psychol 71(2):131–148

    Article  Google Scholar 

  • Schmalhofer F, Glavanov D (1986) Three components of understanding a programmer’s manual: verbatim, propositional, and situational representations. J Mem Lang 25(3):279–294

    Article  Google Scholar 

  • Schnotz W (1993) On the relation of dual coding and mental models in graphics comprehension. Learning and Instruction 3(3):247–249

    Article  Google Scholar 

  • Schnotz W (2014) Integrated model of text and picture comprehension. In: Mayer R (ed) The Cambridge handbook of multimedia learning. Cambridge University Press, New York, pp 72–103

    Chapter  Google Scholar 

  • Schnotz W, Bannert M (2003) Construction and interference in learning from multiple representation. Learning and instruction 13(2):141–156

    Article  Google Scholar 

  • Schroeder S et al (2015) childLex: a lexical database of German read by children. Behav Res Methods 47(4):1085–1094

    Article  PubMed  Google Scholar 

  • Spearritt D (1962) Listening comprehension: A factorial analysis. Australian Council for Education Research, Melbourne

    Google Scholar 

  • Stanfield RA, Zwaan RA (2001) The effect of implied orientation derived from verbal context on picture recognition. Psychol Sci 12(2):153–156

    Article  CAS  PubMed  Google Scholar 

  • Stanislaw H, Todorov N (1999) Calculation of signal detection theory measures. Behav Res Methods Instrum Comput 31(1):137–149

    Article  CAS  PubMed  Google Scholar 

  • Stanovich KE (1991) Discrepancy definitions of reading disability: Has intelligence led us astray? Read Res Q 26(1):7–29

    Article  Google Scholar 

  • Stanovich KE, Siegel LS (1994) Phenotypic performance profile of children with reading disabilities: a regression-based test of the phonological-core variable-difference model. J Educ Psychol 86(1):24

    Article  Google Scholar 

  • Stanovich KE, West RF (1989) Exposure to print and orthographic processing. Read Res Q 24(4):402–433

    Article  Google Scholar 

  • Sticht TG, James JH (1984) Listening and reading. In: Pearson PD et al (eds) Handbook of reading research. Longman, New York, pp 293–317

    Google Scholar 

  • Sweller J (2005) The redundancy principle in multimedia learning. In: Mayer RE (ed) The Cambridge handbook of multimedia learning. Cambridge University Press, Cambridge, pp 159–168

    Chapter  Google Scholar 

  • Therriault DJ, Yaxley RH, Zwaan RA (2009) The role of color diagnosticity in object recognition and representation. Cogn Process 10(4):335–342

    Article  PubMed  Google Scholar 

  • Tibus M, Heier A, Schwan S (2013) Do films make you learn? Inference processes in expository film comprehension. J Educ Psychol 105(2):329

    Article  Google Scholar 

  • Trabasso T, Van Den Broek P (1985) Causal thinking and the representation of narrative events. J Mem Lang 24(5):612–630

    Article  Google Scholar 

  • Unsöld I, Nieding G (2009) Die Bildung prädiktiver Inferenzen von Kindern und Erwachsenen bei der kognitiven Verarbeitung audiovisueller und auditiver Texte [Predictive inferences in children and adults during the cognitive processing of audiovisual and auditory texts]. Z Entwicklungspsychol Pädagog Psychol 41(2):87–95

    Google Scholar 

  • Van Dijk TA, Kintsch W (1983) Strategies of discourse comprehension. Academic Press, New York

    Google Scholar 

  • Vosniadou S, Ortony A (1989) Similarity and analogical reasoning. Cambridge University Press, New York

    Book  Google Scholar 

  • Walma Van Der Molen JH, Van Der Voort THA (1997) Children’s recall of television and print news: a media comparison study. J Educ Psychol 89(1):82

    Article  Google Scholar 

  • Walma Van Der Molen JH, Van Der Voort THA (2000) The impact of television, print, and audio on children’s recall of the news. Hum Commun Res 26(1):3–26

    Google Scholar 

  • Wannagat W et al (2016) Mental representations of the text surface, the text base, and the situation model in auditory and audiovisual texts in 7-, 9-, and 11-year-olds. Discourse Process. doi:10.1080/0163853X.2016.1237246

    Google Scholar 

  • Wicks RH, Drew DG (1991) Learning from news: effects of message consistency and medium on recall and inference making. Journal Mass Commun Q 68(1–2):155–164

    Google Scholar 

  • Zwaan RA (2004) The immersed experiencer: toward an embodied theory of language comprehension. In: Ross B (ed) Psychology of learning and motivation. Academic Press, New York, pp 35–62

    Google Scholar 

  • Zwaan RA (2014) Embodiment and language comprehension: reframing the discussion. Trends Cogn Sci 18(5):229–234

    Article  PubMed  Google Scholar 

  • Zwaan RA, Radvansky GA (1998) Situation models in language comprehension and memory. Psychol Bull 123(2):162

    Article  CAS  PubMed  Google Scholar 

  • Zwaan RA, Singer M (2003) Text comprehension. In: Graesser AC, Gernsbacher MA, Goldman SR (eds) Handbook of discourse processes. Erlbaum, Mahwah, pp 83–121

    Google Scholar 

Download references

Acknowledgements

This research was funded by a grant of the Deutsche Forschungsgemeinschaft (DFG) awarded to Gerhild Nieding (Ni496/9-1). We wish to thank the participating children, parents, and teachers for their participation and cooperation. We also thank our research assistants and master students for their help with the data collection. Frank Maier supervised the recording of the stories, and Pauline Feichtinger drew the pictures. Thank you!

Author information

Authors and Affiliations

  1. Department of Psychology (Developmental Psychology), University of Würzburg, Röntgenring 10, 97070, Würzburg, Germany

    Wienke Wannagat, Gesine Waizenegger & Gerhild Nieding

Authors
  1. Wienke Wannagat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gesine Waizenegger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerhild Nieding
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wienke Wannagat.

Additional information

Handling editor: Ekaterina Shutova (University of Cambridge); Reviewers: Rolf Zwaan (Erasmus Universiteit Rotterdam), Jan Engelen (Tilburg University).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wannagat, W., Waizenegger, G. & Nieding, G. Multi-level mental representations of written, auditory, and audiovisual text in children and adults. Cogn Process 18, 491–504 (2017). https://doi.org/10.1007/s10339-017-0820-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10339-017-0820-y

Keywords

Search

Navigation