Dopamine and Reward Prediction Error

I am back from the IBAGS conference and full of new information! I plan to blog about tons of amazing things over the next month or so, but today we'll start with some foundation building.

Dopamine nails (source)

The IBAGS (international basal ganglia society) meeting is all about the basal ganglia (which includes the striatum), and as you may know, dopamine is a super important molecule for the proper function of the striatum (it is the dopamine cells that die in Parkinson's Disease).

There were many fantastic talks during the IBAGS meeting and almost a third of them showed the exact same figure on one of their slides. So much so that everyone would start to laugh when someone showed it. And as you may have guessed, it is about dopamine. Here it is:

Schultz 1998 Figure 2

This figure is the basis for the belief that dopamine represents a 'reward prediction error'.  Let me explain. The scattered dots on the lower half of each panel represent action potentials from individual dopaminergic neurons. The x axis it time in seconds. The black columns above them are a histogram showing how much firing is going on at each point in time. When the black columns are tall, there was more dopamine neuron firing. You can see that the height of the black columns matches up with the density of the scattered dots below them.

During 'reward learning', an animal is trained to associate a stimulus (like a tone or a flash of light) with getting a reward (like a drop of water or juice). These three panels show how dopamine responds to this whole process. The first panel shows that when there is no stimulus (CS) and the reward is a surprise, the dopamine neurons respond very strongly to it. The second panel shows that when there is a stimulus that tells the animal that a reward is soon to come, the dopamine neurons respond to the stimulus, but not to the reward. Finally the third panel shows that when there is a stimulus the dopamine neurons respond to it, but if the reward (R) never comes, the dopamine neurons actually stop firing when the reward should have happened.

What is so fascinating about this is that it shows dopamine neurons do not just fire in response to reward, they encode the actual reward with respect to the expected reward. In the author's words:

"Dopamine neurons report rewards relative to their prediction rather than signaling primary rewards unconditionally (Fig. 2). The dopamine response is positive (activation) when primary rewards occur without being predicted. The response is nil when rewards occur as predicted. The response is negative (depression) when predicted rewards are omitted. Thus dopamine neurons report primary rewards according to the difference between the occurrence and the prediction of reward, which can be termed an error in the prediction of reward..." Schultz 1998

This finding is so important to researchers now because it shows that dopamine neurons can encode learning rules. Dopamine neurons constantly and dynamically tell the rest of the brain which stimuli lead to reward, and which stimuli don't. The implications here for pathological learning are huge as well. Mis-signalling in dopamine neurons could lead to an inability to tell what is rewarding and what is not.

© TheCellularScale



Schultz W (1998). Predictive reward signal of dopamine neurons. Journal of neurophysiology, 80 (1), 1-27 PMID: 9658025