Active inference and epistemic value - PubMed Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2015;6(4):187-214.
doi: 10.1080/17588928.2015.1020053. Epub 2015 Mar 13.

Active inference and epistemic value

Karl Friston  1 , Francesco Rigoli  1 , Dimitri Ognibene  2 , Christoph Mathys  1   3   4 , Thomas Fitzgerald  1 , Giovanni Pezzulo  5
Affiliations

Active inference and epistemic value

Karl Friston et al. Cogn Neurosci. 2015.

Erratum in

  • Corrigendum.
    [No authors listed] [No authors listed] Cogn Neurosci. 2016 Jan-Oct;7(1-4):223. doi: 10.1080/17588928.2016.1164961. Cogn Neurosci. 2016. PMID: 27153240 No abstract available.

Abstract

We offer a formal treatment of choice behavior based on the premise that agents minimize the expected free energy of future outcomes. Crucially, the negative free energy or quality of a policy can be decomposed into extrinsic and epistemic (or intrinsic) value. Minimizing expected free energy is therefore equivalent to maximizing extrinsic value or expected utility (defined in terms of prior preferences or goals), while maximizing information gain or intrinsic value (or reducing uncertainty about the causes of valuable outcomes). The resulting scheme resolves the exploration-exploitation dilemma: Epistemic value is maximized until there is no further information gain, after which exploitation is assured through maximization of extrinsic value. This is formally consistent with the Infomax principle, generalizing formulations of active vision based upon salience (Bayesian surprise) and optimal decisions based on expected utility and risk-sensitive (Kullback-Leibler) control. Furthermore, as with previous active inference formulations of discrete (Markovian) problems, ad hoc softmax parameters become the expected (Bayes-optimal) precision of beliefs about, or confidence in, policies. This article focuses on the basic theory, illustrating the ideas with simulations. A key aspect of these simulations is the similarity between precision updates and dopaminergic discharges observed in conditioning paradigms.

Keywords: Active inference; Agency; Bayesian inference; Bayesian surprise; Bounded rationality; Epistemic value; Exploitation; Exploration; Free energy; Information gain; Utility theory.

PubMed Disclaimer

Comment in

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

Publication types

LinkOut - more resources