Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects | Nature Neuroscience
Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects

Nature Neuroscience volume 2, pages 79–87 (1999)Cite this article

Abstract

We describe a model of visual processing in which feedback connections from a higher- to a lower-order visual cortical area carry predictions of lower-level neural activities, whereas the feedforward connections carry the residual errors between the predictions and the actual lower-level activities. When exposed to natural images, a hierarchical network of model neurons implementing such a model developed simple-cell-like receptive fields. A subset of neurons responsible for carrying the residual errors showed endstopping and other extra-classical receptive-field effects. These results suggest that rather than being exclusively feedforward phenomena, nonclassical surround effects in the visual cortex may also result from cortico-cortical feedback as a consequence of the visual system using an efficient hierarchical strategy for encoding natural images.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Hierarchical network for predictive coding.
Figure 2: Receptive fields of feedforward model neurons after training on natural images.
Figure 6: Nonclassical surround effects in the model.
Figure 3: Endstopping in the model network.
Figure 4: Autocorrelation along dominant orientation directions in natural images.
Figure 5: Predictive feedback and endstopping.

Similar content being viewed by others

References

  1. Hubel, D. H. & Wiesel, T. N. Receptive fields and functional architecture in two non-striate visual areas (18 and 19) of the cat. J. Neurophysiol. 28, 229–289 (1965).

    Article  CAS  Google Scholar 

  2. Hubel, D. H. & Wiesel, T. N. Receptive fields and functional architecture of monkey striate cortex . J. Physiol. (Lond.) 195, 215–243 (1968).

    Article  CAS  Google Scholar 

  3. Bolz, J. & Gilbert, C. D. Generation of end-inhibition in the visual cortex via interlaminar connections. Nature 320, 362–365 (1986).

    Article  CAS  Google Scholar 

  4. Hubel, D. H. & Livingstone, M. S. Segregation of form, color, and stereopsis in primate area 18. J. Neurosci. 7, 3378–3415 (1987).

    Article  CAS  Google Scholar 

  5. Desimone, R. & Schein, S. J. Visual properties of neurons in area V4 of the macaque: sensitivity to stimulus form. J. Neurophysiol. 57, 835–868 ( 1987).

    Article  CAS  Google Scholar 

  6. Allman, J., Miezin, F. & McGuinness, E. Stimulus specific responses from beyond the classical receptive field: Neurophysiological mechanisms for local-global comparisons in visual neurons. Annu. Rev. Neurosci. 8, 407–429 (1985).

    Article  CAS  Google Scholar 

  7. Dobbins, A., Zucker, S. W. & Cynader, M. S. Endstopped neurons in the visual cortex as a substrate for calculating curvature. Nature 329, 438 –441 (1987).

    Article  CAS  Google Scholar 

  8. Bolz, J., Gilbert, C. D. & Wiesel, T. N. Pharmacological analysis of cortical circuitry. Trends Neurosci. 12, 292–296 (1989).

    Article  CAS  Google Scholar 

  9. Peterhans, E. & von der Heydt, R. in Representations of Vision. Trends and Tacit Assumptions (eds Gorea, A., Frégnac, Y., Kapoulis, Z. & Findlay, J.) 111–124 (Cambridge Univ. Press, Cambridge, UK, 1991).

    Google Scholar 

  10. Grossberg, S., Mingolla, E. & Ross, W. D. Visual brain and visual perception: how does the cortex do perceptual grouping? Trends Neurosci. 20, 106–111 (1997).

    Article  CAS  Google Scholar 

  11. Peterhans, E. & von der Heydt, R. Subjective contours—bridging the gap between psychophysics and physiology. Trends Neurosci. 14, 112–119 ( 1991).

    Article  CAS  Google Scholar 

  12. Rao, R. P. N. & Ballard, D. H. Dynamic model of visual recognition predicts neural response properties in the visual cortex. Neural Comput. 9, 721–763 ( 1997).

    Article  CAS  Google Scholar 

  13. Gallant, J. L., Connor, C. E. & Van Essen, D. C. Neural activity in areas V1, V2 and V4 during free viewing of natural scenes compared to controlled viewing. Neuroreport 9, 2153–2158 ( 1998).

    Article  CAS  Google Scholar 

  14. Attneave, F. Some informational aspects of visual perception. Psychol. Rev. 61, 183–193 ( 1954).

    Article  CAS  Google Scholar 

  15. MacKay, D. M. in Automata Studies (eds Shannon, C. E. & McCarthy, J.) 235– 251 (Princeton Univ. Press, Princeton, NJ, 1956).

    Google Scholar 

  16. Barlow, H. B. in Sensory Communication (ed. Rosenblith, W. A.) 217– 234 (MIT Press, Cambridge, MA, 1961).

    Google Scholar 

  17. Atick, J. J. Could information theory provide an ecological theory of sensory processing? Network Comput. Neural Sys. 3, 213– 251 (1992).

    Article  Google Scholar 

  18. Buchsbaum, G. & Gottschalk, A. Trichromacy, opponent colours coding and optimum colour information transmission in the retina. Proc. R. Soc. Lond. B Biol. Sci. 220, 89– 113 (1983).

    Article  CAS  Google Scholar 

  19. Srinivasan, M. V., Laughlin, S. B. & Dubs A. Predictive coding: A fresh view of inhibition in the retina. Proc. R. Soc. Lond. B Biol. Sci. 216, 427–459 (1982).

    Article  CAS  Google Scholar 

  20. Dan, Y., Atick, J. J. & Reid, R. C. Efficient coding of natural scenes in the lateral geniculate nucleus: experimental test of a computational theory. J. Neurosci. 16, 3351–3362 (1996).

    Article  CAS  Google Scholar 

  21. Dong, D. W. & Atick, J. J. Temporal decorrelation: a theory of lagged and nonlagged responses in the lateral geniculate nucleus. Network Comput. Neural Sys. 6, 159– 178 (1995).

    Article  Google Scholar 

  22. Mumford, D. On the computational architecture of the neocortex. II. The role of cortico-cortical loops. Biol. Cybern. 66, 241– 251 (1992).

    Article  CAS  Google Scholar 

  23. Pece, A. E. C. in Artificial Neural Networks 2 (eds Aleksander, I. & Taylor, J.) 865–868 (Elsevier, Amsterdam, 1992).

    Book  Google Scholar 

  24. Softky, W. R. in Advances in Neural Information Processing Systems 8 (eds Touretzky, D., Mozer, M. & Hasselmo, M.) 809–815 (MIT Press, Cambridge, MA, 1996).

    Google Scholar 

  25. Ullman, S. in Large-Scale Neuronal Theories of the Brain (eds Koch, C. & Davis, J. L.) 257–270 (MIT Press, Cambridge, MA, 1994).

    Google Scholar 

  26. Olshausen, B. A. & Field, D. J. Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Res. 37, 3311–3325 ( 1997).

    Article  CAS  Google Scholar 

  27. Dayan, P., Hinton, G.E., Neal, R.M. & Zemel, R.S. The Helmholtz machine. Neural Comput. 7, 889– 904, (1995).

    Article  CAS  Google Scholar 

  28. Luettgen, M. R. & Willsky, A. S. Likelihood calculation for a class of multiscale stochastic models, with application to texture discrimination. IEEE Trans. Image Proc. 4, 194–207 (1995).

    Article  CAS  Google Scholar 

  29. Felleman, D. J. & Van Essen, D. C. Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex 1, 1–47 ( 1991).

    Article  CAS  Google Scholar 

  30. Field, D. J. Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A 4, 2379–2394 (1987).

    Article  CAS  Google Scholar 

  31. Bell, A. J. & Sejnowski, T. J. The "independent components" of natural scenes are edge filters. Vision Res. 37, 3327–3338 (1997).

    Article  CAS  Google Scholar 

  32. Olshausen, B. A. & Field, D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607– 609 (1996).

    Article  CAS  Google Scholar 

  33. Zipser, K., Lamme, V. A. F. & Schiller, P. H. Contextual modulation in primary visual cortex. J. Neurosci. 16, 7376–7389 (1996).

    Article  CAS  Google Scholar 

  34. Sandell, J. H. & Schiller, P. H. Effect of cooling area 18 on striate cortex cells in the squirrel monkey. J. Neurophysiol. 48, 38–48 (1982).

    Article  CAS  Google Scholar 

  35. Knierim, J. & Van Essen, D. C. Neural responses to static texture patterns in area V1 of the alert macaque monkey. J. Neurophysiol. 67, 961–980 ( 1992).

    Article  CAS  Google Scholar 

  36. Sillito, A. M., Grieve, K. L., Jones, H. E., Cudeiro, J. & Davis, J. Visual cortical mechanisms detecting focal orientation discontinuities. Nature 378, 492–496 (1995).

    Article  CAS  Google Scholar 

  37. Hupé, J. M. et al. Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons. Nature 394 , 784–787 (1998).

    Article  Google Scholar 

  38. Murphy, P. C. & Sillito, A. M. Corticofugal feedback influences the generation of length tuning in the visual pathway. Nature 329, 727–729 (1987).

    Article  CAS  Google Scholar 

  39. Gilbert, C. D. Adult cortical dynamics. Physiol. Rev. 78, 467–485 (1998).

    Article  CAS  Google Scholar 

  40. Lee, D. D. & Seung, H. S. in Advances in Neural Information Processing Systems 9 (eds Mozer, M., Jordan, M. & Petsche, T.) 515–521 (MIT Press, Cambridge, MA, 1997).

    Google Scholar 

  41. Heeger, D. J., Simoncelli, E. P. & Movshon, J. A. Computational models of cortical visual processing. Proc. Natl. Acad. Sci. USA 93, 623– 627 (1996).

    Article  CAS  Google Scholar 

  42. Miller, E. K., Li, L. & Desimone, R. A neural mechanism for working and recognition memory in inferior temporal cortex. Science 254, 1377–1379 (1991).

    Article  CAS  Google Scholar 

  43. Gross, C. G. & Sergent, J. Face recognition. Curr. Opin. Neurobiol. 2, 156–161 (1992).

    Article  CAS  Google Scholar 

  44. Logothetis, N. K. & Pauls, J. Psychophysical and physiological evidence for viewer-centered object representations in the primate. Cereb. Cortex 5, 270–288 (1995).

    Article  CAS  Google Scholar 

  45. Poggio, T. & Edelman, S. A network that learns to recognize 3D objects. Nature 343, 263– 266 (1990).

    Article  CAS  Google Scholar 

  46. Bell, C., Bodznick, D., Montgomery, J. & Bastian, J. The generation and subtraction of sensory expectations within cerebellum-like structures. Brain Behav. Evol. 50 Suppl. 1, 17–31 (1997).

    Article  Google Scholar 

  47. Schultz, W. et al. in Models of Information Processing in the Basal Ganglia (eds Houk, J. C., Davis, J. L. & Beiser, D. G.) 233– 248 (MIT Press, Cambridge, MA, 1995).

    Google Scholar 

  48. Creutzfeldt, O. D. Generality of the functional structure of the neocortex. Naturwissenschaften 64, 507–517 ( 1977).

    Article  CAS  Google Scholar 

  49. Rao, R. P. N. An optimal estimation approach to visual perception and learning. Vision Res. (in press).

  50. Rissanen, J. Stochastic Complexity in Statistical Inquiry (World Scientific, Singapore, 1989).

    Google Scholar 

Download references

Acknowledgements

We thank Christof Koch for comments on the manuscript and Mary Hayhoe, Terrence Sejnowski and members of the Computational Neurobiology Lab at the Salk Institute for discussions. This work was supported by research grants from the National Institute of Health (NIH), the National Science Foundation (NSF) and the Alfred P. Sloan Foundation.

Author information

Authors and Affiliations

  1. The Salk Institute, Sloan Center for Theoretical Neurobiology and Computational Neurobiology Laboratory, 10010 N. Torrey Pines Road, La Jolla, 92037, California, USA

    Rajesh P. N. Rao

  2. Department of Computer Science, University of Rochester, Rochester, 14627-0226, New York, USA

    Dana H. Ballard

Authors
  1. Rajesh P. N. Rao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dana H. Ballard
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rajesh P. N. Rao.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rao, R., Ballard, D. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat Neurosci 2, 79–87 (1999). https://doi.org/10.1038/4580

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1038/4580

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing