乐胖代购免代理版

{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,7,29]],"date-time":"2024-07-29T11:56:28Z","timestamp":1722254188491},"reference-count":77,"publisher":"MIT Press","issue":"2","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,2,1]]},"abstract":"Abstract<\/jats:title>Surprise-based learning allows agents to rapidly adapt to nonstationary stochastic environments characterized by sudden changes. We show that exact Bayesian inference in a hierarchical model gives rise to a surprise-modulated trade-off between forgetting old observations and integrating them with the new ones. The modulation depends on a probability ratio, which we call the Bayes Factor Surprise, that tests the prior belief against the current belief. We demonstrate that in several existing approximate algorithms, the Bayes Factor Surprise modulates the rate of adaptation to new observations. We derive three novel surprise-based algorithms, one in the family of particle filters, one in the family of variational learning, and one in the family of message passing, that have constant scaling in observation sequence length and particularly simple update dynamics for any distribution in the exponential family. Empirical results show that these surprise-based algorithms estimate parameters better than alternative approximate approaches and reach levels of performance comparable to computationally more expensive algorithms. The Bayes Factor Surprise is related to but different from the Shannon Surprise. In two hypothetical experiments, we make testable predictions for physiological indicators that dissociate the Bayes Factor Surprise from the Shannon Surprise. The theoretical insight of casting various approaches as surprise-based learning, as well as the proposed online algorithms, may be applied to the analysis of animal and human behavior and to reinforcement learning in nonstationary environments.<\/jats:p>","DOI":"10.1162\/neco_a_01352","type":"journal-article","created":{"date-parts":[[2021,1,5]],"date-time":"2021-01-05T22:38:38Z","timestamp":1609886318000},"page":"269-340","source":"Crossref","is-referenced-by-count":18,"title":["Learning in Volatile Environments With the Bayes Factor Surprise"],"prefix":"10.1162","volume":"33","author":[{"given":"Vasiliki","family":"Liakoni","sequence":"first","affiliation":[{"name":"\u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, School of Computer and Communication Sciences and School of Life Sciences, 1015 Lausanne, Switzerland vasiliki.liakoni@epfl.ch"}]},{"given":"Alireza","family":"Modirshanechi","sequence":"additional","affiliation":[{"name":"\u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, School of Computer and Communication Sciences and School of Life Sciences, 1015 Lausanne, Switzerland alireza.modirshanechi@epfl.ch"}]},{"given":"Wulfram","family":"Gerstner","sequence":"additional","affiliation":[{"name":"\u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, School of Computer and Communication Sciences and School of Life Sciences, 1015 Lausanne, Switzerland wulfram.gerstner@epfl.ch"}]},{"given":"Johanni","family":"Brea","sequence":"additional","affiliation":[{"name":"\u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne, School of Computer and Communication Sciences and School of Life Sciences, 1015 Lausanne, Switzerland johanni.brea@epfl.ch"}]}],"member":"281","published-online":{"date-parts":[[2021,2,1]]},"reference":[{"key":"2021031822393537500_B1","unstructured":"Adams, R. P., & MacKay, D. J. (2007). Bayesian online changepoint detection. arXiv:0710.3742."},{"key":"2021031822393537500_B2","doi-asserted-by":"crossref","unstructured":"Aminikhanghahi, S., & Cook, D. J. (2017). A survey of methods for time series change point detection. Knowledge and Information Systems, 51(2), 339\u2013367.","DOI":"10.1007\/s10115-016-0987-z"},{"key":"2021031822393537500_B3","unstructured":"Barber, D. (2006). Expectation correction for smoothed inference in switching linear dynamical systems. Journal of Machine Learning Research, 7, 2515\u20132540."},{"key":"2021031822393537500_B4","doi-asserted-by":"crossref","unstructured":"Barber, D. (2012). Bayesian reasoning and machine learning. Cambridge: Cambridge University Press.","DOI":"10.1017\/CBO9780511804779"},{"key":"2021031822393537500_B5","unstructured":"Beal, M. J. (2003). Variational algorithms for approximate Bayesian inference. Ph.D. diss., University College London."},{"key":"2021031822393537500_B6","doi-asserted-by":"crossref","unstructured":"Behrens, T. E., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. (2007). Learning the value of information in an uncertain world. Nature Neuroscience, 10(9), 1214.","DOI":"10.1038\/nn1954"},{"key":"2021031822393537500_B7","doi-asserted-by":"crossref","unstructured":"Bogacz, R. (2017). A tutorial on the free-energy framework for modelling perception and learning. Journal of Mathematical Psychology, 76, 198\u2013211.","DOI":"10.1016\/j.jmp.2015.11.003"},{"key":"2021031822393537500_B8","unstructured":"Bogacz, R. (2019). Dopamine role in learning and action inference. bioRxiv:837641."},{"key":"2021031822393537500_B9","doi-asserted-by":"crossref","unstructured":"Boyd, S., & Vandenberghe, L. (2004). Convex optimization. Cambridge: Cambridge University Press.","DOI":"10.1017\/CBO9780511804441"},{"key":"2021031822393537500_B10","doi-asserted-by":"crossref","unstructured":"Brown, S. D., & Steyvers, M. (2009). Detecting and predicting changes. Cognitive Psychology, 5(1), 49\u201367.","DOI":"10.1016\/j.cogpsych.2008.09.002"},{"key":"2021031822393537500_B11","unstructured":"Cummings, R., Krehbiel, S., Mei, Y., Tuo, R., & Zhang, W. (2018). Differentially private change-point detection. In S.Bengio, H.Wallach, H.Larochelle, K.Grauman, N.Cesa-Bianchi, & R.Garnett (Eds.), Advances in neural information processing systems, 31 (pp. 10825\u201310834). Red Hook, NY: Curran."},{"key":"2021031822393537500_B12","doi-asserted-by":"crossref","unstructured":"d'Acremont, M., & Bossaerts, P. (2016). Neural mechanisms behind identification of leptokurtic noise and adaptive behavioral response. Cerebral Cortex, 26(4), 1818\u20131830.","DOI":"10.1093\/cercor\/bhw013"},{"key":"2021031822393537500_B13","unstructured":"Daw, N., & Courville, A. (2008). The pigeon as particle filter. In J. C.Platt, D.Koller, Y.Singer, & S. T.Roweis (Eds.), Advances in neural information processing systems, 20 (pp. 369\u2013376). Red Hook, NY: Curran."},{"key":"2021031822393537500_B14","doi-asserted-by":"crossref","unstructured":"Doucet, A., Godsill, S., & Andrieu, C. (2000). On sequential Monte Carlo sampling methods for Bayesian filtering. Statistics and Computing, 10(3), 197\u2013208.","DOI":"10.1023\/A:1008935410038"},{"key":"2021031822393537500_B15","unstructured":"Doucet, A., & Johansen, A. M. (2009). A tutorial on particle filtering and smoothing: Fifteen years later. In D.Crisan & B.Rozovskii (Eds.), The Oxford handbook of nonlinear filtering (pp. 656\u2013704), 3. Oxford: Oxford University Press."},{"key":"2021031822393537500_B16","doi-asserted-by":"crossref","unstructured":"Doucet, A., & Tadi\u0107, V. B. (2003). Parameter estimation in general state-space models using particle methods. Annals of the Institute of Statistical Mathematics, 55(2), 409\u2013422.","DOI":"10.1007\/BF02530508"},{"key":"2021031822393537500_B17","doi-asserted-by":"crossref","unstructured":"Efron, B., & Hastie, T. (2016). Computer age statistical inference. Cambridge: Cambridge University Press.","DOI":"10.1017\/CBO9781316576533"},{"key":"2021031822393537500_B18","doi-asserted-by":"crossref","unstructured":"Faraji, M., Preuschoff, K., & Gerstner, W. (2018). Balancing new against old information: The role of puzzlement surprise in learning. Neural Computation, 30(1), 34\u201383.","DOI":"10.1162\/neco_a_01025"},{"key":"2021031822393537500_B19","doi-asserted-by":"crossref","unstructured":"Fearnhead, P., & Liu, Z. (2007). On-line inference for multiple changepoint problems. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69(4), 589\u2013605.","DOI":"10.1111\/j.1467-9868.2007.00601.x"},{"key":"2021031822393537500_B20","unstructured":"Findling, C., Chopin, N., & Koechlin, E. (2019). Imprecise neural computations as source of human adaptive behavior in volatile environments. bioRxiv:799239."},{"key":"2021031822393537500_B21","doi-asserted-by":"crossref","unstructured":"Fox, E., Sudderth, E. B., Jordan, M. I., & Willsky, A. S. (2011). Bayesian nonparametric inference of switching dynamic linear models. IEEE Transactions on Signal Processing, 59(4), 1569\u20131585.","DOI":"10.1109\/TSP.2010.2102756"},{"key":"2021031822393537500_B22","doi-asserted-by":"crossref","unstructured":"Fr\u00e9maux, N., & Gerstner, W. (2016). Neuromodulated spike-timing-dependent plasticity, and theory of three-factor learning rules. Frontiers in Neural Circuits, 9, 85.","DOI":"10.3389\/fncir.2015.00085"},{"key":"2021031822393537500_B23","doi-asserted-by":"crossref","unstructured":"Friston, K. (2010). The free-energy principle: A unified brain theory?Nature Reviews Neuroscience, 1(2), 127.","DOI":"10.1038\/nrn2787"},{"key":"2021031822393537500_B24","doi-asserted-by":"crossref","unstructured":"Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., & Pezzulo, G. (2017). Active inference: A process theory. Neural Computation, 29(1), 1\u201349.","DOI":"10.1162\/NECO_a_00912"},{"key":"2021031822393537500_B25","unstructured":"George, C. P., & Doss, H. (2017). Principled selection of hyperparameters in the latent Dirichlet allocation model. Journal of Machine Learning Research, 18, 1\u201338."},{"key":"2021031822393537500_B26","doi-asserted-by":"crossref","unstructured":"Gershman, S. J. (2019). What does the free energy principle tell us about the brain? arXiv:1901\/07945.","DOI":"10.51628\/001c.10839"},{"key":"2021031822393537500_B27","doi-asserted-by":"crossref","unstructured":"Gershman, S. J., Monfils, M.-H., Norman, K. A., & Niv, Y. (2017). The computational nature of memory modification. Elife, 6, e23763.","DOI":"10.7554\/eLife.23763.019"},{"key":"2021031822393537500_B28","doi-asserted-by":"crossref","unstructured":"Gershman, S. J., Radulescu, A., Norman, K. A., & Niv, Y. (2014). Statistical computations underlying the dynamics of memory updating. PLOS Computational Biology, 10(11), e1003939.","DOI":"10.1371\/journal.pcbi.1003939"},{"key":"2021031822393537500_B29","doi-asserted-by":"crossref","unstructured":"Gerstner, W., Lehmann, M., Liakoni, V., Corneil, D., & Brea, J. (2018). Eligibility traces and plasticity on behavioral time scales: Experimental support of neo-Hebbian three-factor learning rules. Frontiers in Neural Circuits, 12.","DOI":"10.3389\/fncir.2018.00053"},{"key":"2021031822393537500_B30","doi-asserted-by":"crossref","unstructured":"Ghahramani, Z., & Hinton, G. E. (2000). Variational learning for switching state-space models. Neural Computation, 12(4), 831\u2013864.","DOI":"10.1162\/089976600300015619"},{"key":"2021031822393537500_B31","doi-asserted-by":"crossref","unstructured":"Glaze, C. M., Kable, J. W., & Gold, J. I. (2015). Normative evidence accumulation in unpredictable environments. Elife, 4, e08825.","DOI":"10.7554\/eLife.08825.019"},{"key":"2021031822393537500_B32","doi-asserted-by":"crossref","unstructured":"Gordon, N. J., Salmond, D. J., & Smith, A. F. (1993). Novel approach to nonlinear\/non-gaussian Bayesian state estimation. In IEE Proceedings (Radar and Signal Processing), 140, 107\u2013113.","DOI":"10.1049\/ip-f-2.1993.0015"},{"key":"2021031822393537500_B33","doi-asserted-by":"crossref","unstructured":"Heilbron, M., & Meyniel, F. (2019). Confidence resets reveal hierarchical adaptive learning in humans. PLOS Computational Biology, 15(4), e1006972.","DOI":"10.1371\/journal.pcbi.1006972"},{"key":"2021031822393537500_B34","unstructured":"Huang, Y., & Rao, R. P. (2014). Neurons as Monte Carlo samplers: Bayesian inference and learning in spiking networks. In Z.Ghahramani, M.Welling, C.Cortes, N.Lawrence, & K. Q.Weinberger (Eds.), Advances in neural information processing systems, 27 (pp. 1943\u20131951). Red Hook, NY: Curran."},{"key":"2021031822393537500_B35","doi-asserted-by":"crossref","unstructured":"Huettel, S. A., Mack, P. B., & McCarthy, G. (2002). Perceiving patterns in random series: Dynamic processing of sequence in prefrontal cortex. Nature Neuroscience, 5(5), 485\u2013490.","DOI":"10.1038\/nn841"},{"key":"2021031822393537500_B36","unstructured":"Itti, L., & Baldi, P. F. (2006). Bayesian surprise attracts human attention. In Y.Weiss, B.Sch\u00f6lkopf, & J.Platt (Eds.), Advances in neural information processing systems, 18 (pp. 547\u2013554). Cambridge, MA: MIT Press."},{"key":"2021031822393537500_B37","doi-asserted-by":"crossref","unstructured":"Joshi, S., & Gold, J. I. (2020). Pupil size as a window on neural substrates of cognition. Trends in Cognitive Sciences, 24.","DOI":"10.1016\/j.tics.2020.03.005"},{"key":"2021031822393537500_B38","doi-asserted-by":"crossref","unstructured":"Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 9(430), 773\u2013795.","DOI":"10.1080\/01621459.1995.10476572"},{"key":"2021031822393537500_B39","doi-asserted-by":"crossref","unstructured":"Konovalov, A., & Krajbich, I. (2018). Neurocomputational dynamics of sequence learning. Neuron, 98(6), 1282\u20131293.","DOI":"10.1016\/j.neuron.2018.05.013"},{"key":"2021031822393537500_B40","doi-asserted-by":"crossref","unstructured":"Kopp, B., & Lange, F. (2013). Electrophysiological indicators of surprise and entropy in dynamic task-switching environments. Frontiers in Human Neuroscience, 7, 300.","DOI":"10.3389\/fnhum.2013.00300"},{"key":"2021031822393537500_B41","doi-asserted-by":"crossref","unstructured":"Kutschireiter, A., Surace, S. C., Sprekeler, H., & Pfister, J.-P. (2017). Nonlinear Bayesian filtering and learning: A neuronal dynamics for perception. Scientific Reports, 7(1), 8722.","DOI":"10.1038\/s41598-017-06519-y"},{"key":"2021031822393537500_B42","doi-asserted-by":"crossref","unstructured":"Legenstein, R., & Maass, W. (2014). Ensembles of spiking neurons with noise support optimal probabilistic inference in a dynamically changing environment. PLOS Computational Biology, 10(10).","DOI":"10.1371\/journal.pcbi.1003859"},{"key":"2021031822393537500_B43","doi-asserted-by":"crossref","unstructured":"Lieder, F., Daunizeau, J., Garrido, M. I., Friston, K. J., & Stephan, K. E. (2013). Modelling trial-by-trial changes in the mismatch negativity. PLOS Computational Biology, 9(2).","DOI":"10.1371\/journal.pcbi.1002911"},{"key":"2021031822393537500_B44","unstructured":"Lin, K., Sharpnack, J. L., Rinaldo, A., & Tibshirani, R. J. (2017). A sharp error analysis for the fused lasso, with application to approximate changepoint screening. In I.Guyon, U. V.Luxburg, S.Bengio, H.Wallach, R.Fergus, S.Vishwanathan, & R.Garnett (Eds.), Advances in neural information processing systems, 30 (pp. 6884\u20136893). Red Hook, NY: Curran."},{"key":"2021031822393537500_B45","doi-asserted-by":"crossref","unstructured":"Lisman, J., Grace, A. A., & Duzel, E. (2011). A neohebbian framework for episodic memory; role of dopamine-dependent late LTP. Trends in Neurosciences, 34(10), 536\u2013547.","DOI":"10.1016\/j.tins.2011.07.006"},{"key":"2021031822393537500_B46","unstructured":"Liu, J., & West, M. (2001). Combined parameter and state estimation in simulation-based filtering. In A.Doucet, N.de Freitas, & N.Gordon (Eds.), Sequential Monte Carlo methods in practice (pp. 197\u2013223). Berlin: Springer."},{"key":"2021031822393537500_B47","unstructured":"Lomonaco, V., Desai, K., Culurciello, E., & Maltoni, D. (2019). Continual reinforcement learning in 3D non-stationary environments. arXiv:1905.1011."},{"key":"2021031822393537500_B48","doi-asserted-by":"crossref","unstructured":"Loued-Khenissi, L., Pfeuffer, A., Einh\u00e4user, W., & Preuschoff, K. (2020). Anterior insula reflects surprise in value-based decision-making and perception. NeuroImage, 116549.","DOI":"10.1016\/j.neuroimage.2020.116549"},{"key":"2021031822393537500_B49","doi-asserted-by":"crossref","unstructured":"Maheu, M., Dehaene, S., & Meyniel, F. (2019). Brain signatures of a multiscale process of sequence learning in humans. Elife, 8, e41541.","DOI":"10.7554\/eLife.41541.023"},{"key":"2021031822393537500_B50","doi-asserted-by":"crossref","unstructured":"Mars, R. B., Debener, S., Gladwin, T. E., Harrison, L. M., Haggard, P., Rothwell, J. C., & Bestmann, S. (2008). Trial-by-trial fluctuations in the event-related electroencephalogram reflect dynamic changes in the degree of surprise. Journal of Neuroscience, 28(47), 12539\u201312545.","DOI":"10.1523\/JNEUROSCI.2925-08.2008"},{"key":"2021031822393537500_B51","unstructured":"Masegosa, A., Nielsen, T. D., Langseth, H., Ramos-L\u00f3pez, D., Salmer\u00f3n, A., & Madsen, A. L. (2017). Bayesian models of data streams with hierarchical power priors. In Proceedings of the 34th International Conference on Machine Learning, vol. 70 (pp. 2334\u20132343). PMLR."},{"key":"2021031822393537500_B52","doi-asserted-by":"crossref","unstructured":"Mathys, C., Daunizeau, J., Friston, K. J., & Stephan, K. E. (2011). A Bayesian foundation for individual learning under uncertainty. Frontiers in Human Neuroscience, 5, 39.","DOI":"10.3389\/fnhum.2011.00039"},{"key":"2021031822393537500_B53","doi-asserted-by":"crossref","unstructured":"Meyniel, F., Maheu, M., & Dehaene, S. (2016). Human inferences about sequences: A minimal transition probability model. PLOS Computational Biology, 12(12), e1005260.","DOI":"10.1371\/journal.pcbi.1005260"},{"key":"2021031822393537500_B54","doi-asserted-by":"crossref","unstructured":"Modirshanechi, A., Kiani, M. M., & Aghajan, H. (2019). Trial-by-trial surprise-decoding model for visual and auditory binary oddball tasks. NeuroImage, 196, 302\u2013317.","DOI":"10.1016\/j.neuroimage.2019.04.028"},{"key":"2021031822393537500_B55","doi-asserted-by":"crossref","unstructured":"Musiolek, L., Blankenburg, F., Ostwald, D., & Rabovsky, M. (2019). Modeling the n400 brain potential as semantic Bayesian surprise. In Proceedings of the 2019 Conference on Cognitive Computational Neuroscience.https:\/\/ccneuro.org\/2019\/Papers\/AcceptedPapers.asp","DOI":"10.32470\/CCN.2019.1184-0"},{"key":"2021031822393537500_B56","unstructured":"Nagabandi, A., Clavera, I., Liu, S., Fearing, R. S., Abbeel, P., Levine, S., & Finn, C. (2018). Learning to adapt in dynamic, real-world environments through meta-reinforcement learning. arXiv:1803.11347."},{"key":"2021031822393537500_B57","doi-asserted-by":"crossref","unstructured":"Nassar, M. R., Bruckner, R., & Frank, M. J. (2019). Statistical context dictates the relationship between feedback-related EEG signals and learning. Elife, 8, e46975.","DOI":"10.7554\/eLife.46975.020"},{"key":"2021031822393537500_B58","doi-asserted-by":"crossref","unstructured":"Nassar, M. R., Rumsey, K. M., Wilson, R. C., Parikh, K., Heasly, B., & Gold, J. I. (2012). Rational regulation of learning dynamics by pupil-linked arousal systems. Nature Neuroscience, 15(7), 1040\u20131046.","DOI":"10.1038\/nn.3130"},{"key":"2021031822393537500_B59","doi-asserted-by":"crossref","unstructured":"Nassar, M. R., Wilson, R. C., Heasly, B., & Gold, J. I. (2010). An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment. Journal of Neuroscience, 30(37), 12366\u201312378.","DOI":"10.1523\/JNEUROSCI.0822-10.2010"},{"key":"2021031822393537500_B60","doi-asserted-by":"crossref","unstructured":"Ostwald, D., Spitzer, B., Guggenmos, M., Schmidt, T. T., Kiebel, S. J., & Blankenburg, F. (2012). Evidence for neural encoding of Bayesian surprise in human somatosensation. NeuroImage, 62(1), 177\u2013188.","DOI":"10.1016\/j.neuroimage.2012.04.050"},{"key":"2021031822393537500_B61","doi-asserted-by":"crossref","unstructured":"\u00d6zkan, E., \u0160m\u00eddl, V., Saha, S., Lundquist, C., & Gustafsson, F. (2013). Marginalized adaptive particle filtering for nonlinear models with unknown time-varying noise parameters. Automatica, 49(6), 1566\u20131575.","DOI":"10.1016\/j.automatica.2013.02.046"},{"key":"2021031822393537500_B62","unstructured":"Papoulis, A., & Saunders, H. (1989). Probability, random variables and stochastic processes. American Society of Mechanical Engineers Digital Collection."},{"key":"2021031822393537500_B63","unstructured":"Prat-Carrabin, A., Wilson, R. C., Cohen, J. D., & Da Silveira, R. A. (2020). Human inference in changing environments with temporal structure. bioRxiv:720516."},{"key":"2021031822393537500_B64","doi-asserted-by":"crossref","unstructured":"Preuschoff, K., t Hart, B. M., & Einhauser, W. (2011). Pupil dilation signals surprise: Evidence for noradrenaline's role in decision making. Frontiers in Neuroscience, 5, 115.","DOI":"10.3389\/fnins.2011.00115"},{"key":"2021031822393537500_B65","unstructured":"Ryali, C., Reddy, G., & Yu, A. J. (2018). Demystifying excessively volatile human learning: A Bayesian persistent prior and a neural approximation. In S.Bengio, H.Wallach, H.Larochelle, K.Grauman, N. Cesa-Bianchi, & R.Garnett, Advances in neural information processing systems, 31 (pp. 2781\u20132790). Red Hook, NY: Curran."},{"key":"2021031822393537500_B66","doi-asserted-by":"crossref","unstructured":"S\u00e4rkk\u00e4, S. (2013). Bayesian filtering and smoothing (Vol. 3). Cambridge: Cambridge University Press.","DOI":"10.1017\/CBO9781139344203"},{"key":"2021031822393537500_B67","doi-asserted-by":"crossref","unstructured":"Schmidhuber, J. (2010). Formal theory of creativity, fun, and intrinsic motivation (1990\u20132010). IEEE Transactions on Autonomous Mental Development, 2(3), 230\u2013247.","DOI":"10.1109\/TAMD.2010.2056368"},{"key":"2021031822393537500_B68","doi-asserted-by":"crossref","unstructured":"Schwartenbeck, P., FitzGerald, T., Dolan, R., & Friston, K. (2013). Exploration, novelty, surprise, and free energy minimization. Frontiers in Psychology, 4, 710.","DOI":"10.3389\/fpsyg.2013.00710"},{"key":"2021031822393537500_B69","doi-asserted-by":"crossref","unstructured":"Shannon, C. (1948). A mathematical theory of communication. Bell System Technical Journal, 20, 623\u2013656; 27, 379-423.","DOI":"10.1002\/j.1538-7305.1948.tb01338.x"},{"key":"2021031822393537500_B70","unstructured":"Shi, L., & Griffiths, T. L. (2009). Neural implementation of hierarchical Bayesian inference by importance sampling. In Y.Bengio, D,Schuurmans, J.Lafferty, C.Williams, & A.Culotta (Eds.), Advances in neural information processing systems, 22 (pp. 1669\u20131677). Red Hook, NY: Curran."},{"key":"2021031822393537500_B71","unstructured":"Storck, J., Hochreiter, S., & Schmidhuber, J. (1995). Reinforcement driven information acquisition in non-deterministic environments. In Proceedings of the International Conference on Artificial Neural Networks (Vol. 2, pp. 159\u2013164). Piscataway, NJ: IEEE."},{"key":"2021031822393537500_B72","unstructured":"Traor\u00e9, R., Caselles-Dupr\u00e9, H., Lesort, T., Sun, T., Cai, G., D\u00edaz-Rodr\u00edguez, N., & Filliat, D. (2019). Discorl: Continual reinforcement learning via policy distillation. arXiv:1907.05855."},{"key":"2021031822393537500_B73","doi-asserted-by":"crossref","unstructured":"Wilson, R. C., Nassar, M. R., & Gold, J. I. (2010). Bayesian online learning of the hazard rate in change-point problems. Neural Computation, 22(9), 2452\u20132476.","DOI":"10.1162\/NECO_a_00007"},{"key":"2021031822393537500_B74","doi-asserted-by":"crossref","unstructured":"Wilson, R. C., Nassar, M. R., & Gold, J. I. (2013). A mixture of delta-rules approximation to Bayesian inference in change-point problems. PLOS Computational Biology, 9(7), e1003150.","DOI":"10.1371\/journal.pcbi.1003150"},{"key":"2021031822393537500_B75","doi-asserted-by":"crossref","unstructured":"Yu, A. J. (2012). Change is in the eye of the beholder. Nature Neuroscience, 15(7), 933.","DOI":"10.1038\/nn.3150"},{"key":"2021031822393537500_B76","unstructured":"Yu, A. J., & Cohen, J. D. (2009). Sequential effects: Superstition or rational behavior? In D.Koller, D.Schuurmans, Y.Bengio, & L.Bottou (Eds.), Advances in neural information processing systems, 21 (pp. 1873\u20131880). Red Hook, NY: Curran."},{"key":"2021031822393537500_B77","doi-asserted-by":"crossref","unstructured":"Yu, A. J., & Dayan, P. (2005). Uncertainty, neuromodulation, and attention. Neuron, 46(4), 681\u2013692.","DOI":"10.1016\/j.neuron.2005.04.026"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/direct.mit.edu\/neco\/article-pdf\/33\/2\/269\/1896808\/neco_a_01352.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/direct.mit.edu\/neco\/article-pdf\/33\/2\/269\/1896808\/neco_a_01352.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,10]],"date-time":"2022-12-10T17:26:22Z","timestamp":1670693182000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/33\/2\/269\/95646\/Learning-in-Volatile-Environments-With-the-Bayes"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,1]]},"references-count":77,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,2,1]]},"published-print":{"date-parts":[[2021,2,1]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01352","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,2]]},"published":{"date-parts":[[2021,2,1]]}}}