Download:
Abstract:

Attributing outcomes to your own actions or to external causes is essential for appropriately learning which actions lead to reward and which actions do not. Our previous work showed that this type of credit assignment is best explained by a Bayesian reinforcement learning model which posits that beliefs about the causal structure of the environment modulate reward prediction errors (RPEs) during action value updating. In this study, we investigated the brain networks underlying reinforcement learning that are influenced by causal beliefs using functional magnetic resonance imaging while human participants (n = 31; 13 males, 18 females) completed a behavioral task that manipulated beliefs about causal structure. We found evidence that RPEs modulated by causal beliefs are represented in dorsal striatum, while standard (unmodulated) RPEs are represented in ventral striatum. Further analyses revealed that beliefs about causal structure are represented in anterior insula and inferior frontal gyrus. Finally, structural equation modeling revealed effective connectivity from anterior insula to dorsal striatum. Together, these results are consistent with a possible neural architecture in which causal beliefs in anterior insula are integrated with prediction error signals in dorsal striatum to update action values.

Figure 5: Reward prediction errors modulated by causal beliefs in dorsal striatum

Citation

Dorfman, H. M. *, Tomov, M. S. *, Cheung, B., Clarke, D., Gershman, S. J., Hughes, B. L. (2021). “Causal Inference Gates Corticostriatal Learning.” Journal of Neuroscience, 41(32), 6892-6904. https://doi.org/10.1523/JNEUROSCI.2796-20.2021.

@article{dorfman2021causal,
	author = {Dorfman, Hayley M. and Tomov, Momchil S. and Cheung, Bernice and Clarke, Dennis and Gershman, Samuel J. and Hughes, Brent L.},
	title = {Causal Inference Gates Corticostriatal Learning},
	volume = {41},
	number = {32},
	pages = {6892--6904},
	year = {2021},
	doi = {10.1523/JNEUROSCI.2796-20.2021},
	publisher = {Society for Neuroscience},
	issn = {0270-6474},
	URL = {https://www.jneurosci.org/content/41/32/6892},
	eprint = {https://www.jneurosci.org/content/41/32/6892.full.pdf},
	journal = {Journal of Neuroscience}
}