www.astralcodexten.com/p/the-beauty-of-settled-science
1 correction found
The brain is sculpted by reinforcement learning, and a key driver of this learning is a prediction-error-based algorithm located in the nucleus accumbens and implemented through dopamine.
The canonical dopamine reward-prediction-error signal is classically attributed to midbrain dopamine neurons, not to an algorithm "located in the nucleus accumbens." The nucleus accumbens is a major target of those neurons and participates in reward learning, but it is not where the standard teaching-signal account places the generator of the signal.
Full reasoning
This sentence mixes up an important anatomical distinction in the reward-learning literature.
Mainstream reviews describe reward-prediction-error (RPE) signals as being encoded by midbrain dopamine neurons, especially in the ventral tegmental area (VTA) and substantia nigra pars compacta (SNc). Those neurons then project to striatal targets including the nucleus accumbens (NAc).
Two representative sources say this directly:
- A Nature Reviews Neuroscience review states that dopamine neurons originating in the VTA and SNc project to the striatum, that the VTA projects preferentially to the nucleus accumbens, and that "these DA neurons encode reward prediction error (RPE)".
- An NCBI Bookshelf chapter on reward neuroanatomy says "Reward prediction and error detection signals are generated, in part, from the midbrain dopamine cells" and separately describes the ventral striatum / nucleus accumbens as a key target involved in reward processing.
So the nucleus accumbens is indeed deeply involved in reinforcement learning, and NAc activity can reflect prediction-error-related signals. But saying the algorithm is "located in the nucleus accumbens" is anatomically inaccurate relative to the standard account: the classic dopamine teaching signal is generated by midbrain dopamine neurons and broadcast to downstream regions such as the nucleus accumbens.
2 sources
- Striatal circuits for reward learning and decision-making - PMC
DA neurons that originate in the ventral tegmental area (VTA) and substantia nigra pars compacta (SNc) provide dense, topographic innervation to the striatum. The VTA projects preferentially to the nucleus accumbens (NAc) ... Seminal experiments demonstrated that these DA neurons encode reward prediction error (RPE).
- Neuroanatomy of Reward: A View from the Ventral Striatum - Neurobiology of Sensation and Reward - NCBI Bookshelf
Reward prediction and error detection signals are generated, in part, from the midbrain dopamine cells. ... Together, the frontal regions ... project primarily to the rostral striatum, including the nucleus accumbens ... collectively referred to as the VS.