Dropout in Neural Networks Simulates the Paradoxical Effects of Deep Brain Stimulation on Memory - PubMed Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Sep 14:12:273.
doi: 10.3389/fnagi.2020.00273. eCollection 2020.

Dropout in Neural Networks Simulates the Paradoxical Effects of Deep Brain Stimulation on Memory

Shawn Zheng Kai Tan  1 , Richard Du  2 , Jose Angelo Udal Perucho  2 , Shauhrat S Chopra  3 , Varut Vardhanabhuti  2 , Lee Wei Lim  1
Affiliations

Dropout in Neural Networks Simulates the Paradoxical Effects of Deep Brain Stimulation on Memory

Shawn Zheng Kai Tan et al. Front Aging Neurosci. .

Abstract

Neuromodulation techniques such as deep brain stimulation (DBS) are a promising treatment for memory-related disorders including anxiety, addiction, and dementia. However, the outcomes of such treatments appear to be somewhat paradoxical, in that these techniques can both disrupt and enhance memory even when applied to the same brain target. In this article, we hypothesize that disruption and enhancement of memory through neuromodulation can be explained by the dropout of engram nodes. We used a convolutional neural network (CNN) to classify handwritten digits and letters and applied dropout at different stages to simulate DBS effects on engrams. We showed that dropout applied during training improved the accuracy of prediction, whereas dropout applied during testing dramatically decreased the accuracy of prediction, which mimics enhancement and disruption of memory, respectively. We further showed that transfer learning of neural networks with dropout had increased the accuracy and rate of learning. Dropout during training provided a more robust "skeleton" network and, together with transfer learning, mimicked the effects of chronic DBS on memory. Overall, we showed that the dropout of engram nodes is a possible mechanism by which neuromodulation techniques such as DBS can both disrupt and enhance memory, providing a unique perspective on this paradox.

Keywords: deep brain stimulation; dropout; memory; neural network; neuromodulation.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Methodology. A convolutional neural network (CNN) was trained to classify handwritten digits in the EMNIST dataset (A). Dropout was applied to 50% of nodes in the fully connected layer (B). Transfer learning was performed to retrain the network to recognize uppercase letters (C).
Figure 2
Figure 2
Dropout in neural nodes disrupted or enhanced learning in the neural networks depending on when it was applied. Dropout applied during training improved prediction accuracy, whereas dropout applied during testing dramatically decreased prediction accuracy (A). (B) Prediction accuracy of different rates of dropout during training. (C) The training loss and (D) a zoomed in version of the first 20 epochs in panel (C).
Figure 3
Figure 3
Transfer learning of networks that underwent dropout showed higher prediction accuracy and faster learning. Compared to non-dropout networks, transfer learning of neural networks with dropout had increased accuracy (A) with lower training loss (B) and testing loss (C).
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Cohen G., Afshar S., Tapson J., van Schaik A. (2017). “EMNIST: an extension of MNIST to handwritten letters,” in 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, pp. 2921–2926.
    1. Gouveia F. V., Davidson B., Meng Y., Gidyk D. C., Rabin J. S. (2020). Treating post-traumatic stress disorder with neuromodulation therapies: transcranial magnetic stimulation, transcranial direct current stimulation and deep brain stimulation. Neurotherapeutics [Epub ahead of print]. 10.1007/s13311-020-00871-0 - DOI - PMC - PubMed
    1. Hamani C., Dubiela F. P., Soares J. C. K., Shin D., Bittencourt S., Covolan L., et al. . (2010). Anterior thalamus deep brain stimulation at high current impairs memory in rats. Exp. Neurol. 225, 154–162. 10.1016/j.expneurol.2010.06.007 - DOI - PubMed
    1. Hamani C., Stone S. S., Garten A., Lozano A. M., Winocur G. (2011). Memory rescue and enhanced neurogenesis following electrical stimulation of the anterior thalamus in rats treated with corticosterone. Exp. Neurol. 232, 100–104. 10.1016/j.expneurol.2011.08.023 - DOI - PubMed
    1. Hubel D. H., Wiesel T. N. (1959). Receptive fields of single neurones in the cat’s striate cortex. J. Physiol. 148, 574–591. 10.1113/jphysiol.1959.sp006308 - DOI - PMC - PubMed

LinkOut - more resources