Abstract
For neural networks to be considered as realistic models of human linguistic behavior, they must be able to display the level of systematicity that is present in language. This paper investigates the systematic capacities of a sentence-processing Echo State Network. The network is trained on sentences in which particular nouns occur only as subjects and others only as objects. It is then tested on novel sentences in which these roles are reversed. Results show that the network displays so-called strong systematicity.
This research was supported by grant 451-04-043 of the Netherlands Organization for Scientific Research (NWO).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Fodor, J.A., Pylyshyn, Z.W.: Connectionism and cognitive architecture: a critical analysis. Cognition 28, 3–71 (1988)
Chalmers, D.J.: Connectionism and compositionality: why Fodor and Pylyshyn were wrong. Philosophical Psychology 6(3), 305–319 (1993)
Hadley, R.F.: Systematicity in connectionist language learning. Mind & Language 9(3), 247–272 (1994)
Niklasson, L.F., Van Gelder, T.: On being systematically connectionist. Mind & Language 9, 288–302 (1994)
Fodor, J.A., McLaughlin, B.: Connectionism and the problem of systematicity: Why Smolensky’s solution does not work. Cognition 35, 183–204 (1990)
Aizawa, K.: The systematicity arguments. Kluwer Academic Publishers, Dordrecht (2003)
Christiansen, M.H., Chater, N.: Generalization and connectionist language learning. Mind & Language 9(3), 273–287 (1994)
Hadley, R.F., Rotaru-Varga, A., Arnold, D.V., Cardei, V.C.: Syntactic systematicity arising from semantic predictions in a Hebbian-competetive network. Connection Science 13(1), 73–94 (2001)
Bodén, M.: Generalization by symbolic abstraction in cascaded recurrent networks. Neurocomputing 57, 87–104 (2004)
Elman, J.L.: Finding structure in time. Cognitive Science 14, 179–211 (1990)
Jaeger, H.: Adaptive nonlinear system identification with echo state networks. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in neural information processing systems, vol. 15, pp. 593–600. MIT Press, Cambridge (2003)
Jaeger, H., Haas, H.: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304, 78–80 (2004)
Frank, S.L.: Learn more by training less: systematicity in sentence processing by recurrent networks. Connection Science (in press)
Van der Velde, F., Van der Voort van der Kleij, G.T., De Kamps, M.: Lack of combinatorial productivity in language processing with simple recurrent networks. Connection Science 16(1), 21–46 (2004)
Hadley, R.F.: Systematicity revisited: reply to Christiansen and Chater and Niklasson and van Gelder. Mind & Language 9(4), 431–444 (1994)
Hadley, R.F.: On the proper treatment of semantic systematicity 14, 145–172 (2004)
Frank, S.L., Haselager, W.F.G.: Robust semantic systematicity and distributed representations in a connectionist model of sentence comprehension. In: Miyake, N., Sun, R. (eds.) Proceedings of the 28th Annual Conference of the Cognitive Science Society, Lawrence Erlbaum, Mahwah (in press)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Frank, S.L. (2006). Strong Systematicity in Sentence Processing by an Echo State Network. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds) Artificial Neural Networks – ICANN 2006. ICANN 2006. Lecture Notes in Computer Science, vol 4131. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11840817_53
Download citation
DOI: https://doi.org/10.1007/11840817_53
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-38625-4
Online ISBN: 978-3-540-38627-8
eBook Packages: Computer ScienceComputer Science (R0)