Brain-inspired meta-reinforcement learning cognitive control in conflictual inhibition decision-making task for artificial agents
- PMID: 35917665
- DOI: 10.1016/j.neunet.2022.06.020
Brain-inspired meta-reinforcement learning cognitive control in conflictual inhibition decision-making task for artificial agents
Abstract
Conflictual cues and unexpected changes in human real-case scenarios may be detrimental to the execution of tasks by artificial agents, thus affecting their performance. Meta-learning applied to reinforcement learning may enhance the design of control algorithms, where an outer learning system progressively adjusts the operation of an inner learning system, leading to practical benefits for the learning schema. Here, we developed a brain-inspired meta-learning framework for inhibition cognitive control that i) exploits the meta-learning principles in the neuromodulation theory proposed by Doya, ii) relies on a well-established neural architecture that contains distributed learning systems in the human brain, and iii) proposes optimization rules of meta-learning hyperparameters that mimic the dynamics of the major neurotransmitters in the brain. We tested an artificial agent in inhibiting the action command in two well-known tasks described in the literature: NoGo and Stop-Signal Paradigms. After a short learning phase, the artificial agent learned to react to the hold signal, and hence to successfully inhibit the motor command in both tasks, via the continuous adjustment of the learning hyperparameters. We found a significant increase in global accuracy, right inhibition, and a reduction in the latency time required to cancel the action process, i.e., the Stop-signal reaction time. We also performed a sensitivity analysis to evaluate the behavioral effects of the meta-parameters, focusing on the serotoninergic modulation of the dopamine release. We demonstrated that brain-inspired principles can be integrated into artificial agents to achieve more flexible behavior when conflictual inhibitory signals are present in the environment.
Keywords: Basal ganglia; Brain-inspired modeling; Inhibition cognitive control; Meta-learning; Prefrontal cortex.
Copyright © 2022 The Author(s). Published by Elsevier Ltd.. All rights reserved.
Conflict of interest statement
Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Similar articles
-
Reward-dependent learning in neuronal networks for planning and decision making.Prog Brain Res. 2000;126:217-29. doi: 10.1016/S0079-6123(00)26016-0. Prog Brain Res. 2000. PMID: 11105649 Review.
-
Errors in Action Timing and Inhibition Facilitate Learning by Tuning Distinct Mechanisms in the Underlying Decision Process.J Neurosci. 2019 Mar 20;39(12):2251-2264. doi: 10.1523/JNEUROSCI.1924-18.2019. Epub 2019 Jan 17. J Neurosci. 2019. PMID: 30655353 Free PMC article.
-
A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task.Neuroscience. 1999;91(3):871-90. doi: 10.1016/s0306-4522(98)00697-6. Neuroscience. 1999. PMID: 10391468
-
A Kernel Reinforcement Learning Decoding Framework Integrating Neural and Feedback Signals for Brain Control.Annu Int Conf IEEE Eng Med Biol Soc. 2023 Jul;2023:1-4. doi: 10.1109/EMBC40787.2023.10340203. Annu Int Conf IEEE Eng Med Biol Soc. 2023. PMID: 38083464
-
Meta-learning, social cognition and consciousness in brains and machines.Neural Netw. 2022 Jan;145:80-89. doi: 10.1016/j.neunet.2021.10.004. Epub 2021 Oct 18. Neural Netw. 2022. PMID: 34735893 Review.
Cited by
-
Distinctive properties of biological neural networks and recent advances in bottom-up approaches toward a better biologically plausible neural network.Front Comput Neurosci. 2023 Jun 28;17:1092185. doi: 10.3389/fncom.2023.1092185. eCollection 2023. Front Comput Neurosci. 2023. PMID: 37449083 Free PMC article. Review.
MeSH terms
Substances
LinkOut - more resources
Full Text Sources