DeepMind's AI researchers are discovering things about the human brain

(Deepmind)

DeepMind's recent research on how artificial neural networks behave may shed light on how the human brain works, too.

In a paper published on January 15, 2020, DeepMind researchers argue that the brain calculates risk and reward in a far more complicated way than previously thought.

Neural activity measured in mice, they argue, increasingly mirrors their more complicated and successful AI training algorithms.

From their blog post:

We found that dopamine neurons in the brain were each tuned to different levels of pessimism or optimism. If they were a choir, they wouldn’t all be singing the same note, but harmonizing – each with a consistent vocal register, like bass and soprano singers. In artificial reinforcement learning systems, this diverse tuning creates a richer training signal that greatly speeds learning in neural networks, and we speculate that the brain might use it for the same reason.

They write that AI research and neuroscience have long treated the brain as a simple, short-term reward calculator. In AI research, however, developers have used more advanced forms of reward and punishment signalling to speed up the time it takes to train AIs in gaming, autonomous driving and medical research. Artificial neurons, they've found, can work together to map out many possible outcomes.

Now, they say, they've found some evidence that the brain's neurons already do that naturally.

Their new model, based on research done on mice at Harvard, suggests that dopamine helps the brain estimate multiple outcomes to many possible events far in advance.

"The brain represents possible future rewards not as a single mean," the paper reads, "but instead as a probability distribution."

This finding may have serious ramifications in AI research, which has long sought to mimic the inner workings on the mind.

Artificial neural nets that estimate both positive and negative outcomes (known as distributional reinforcement learning) do far better on average than those that only calculate positive outcomes. So what about the brain? Finding evidence that the mind works like this too suggests researchers are on the right path.

Dopamine neurons, they write, aren't nearly as simple as reward estimators:

The existence of distributional reinforcement learning in the brain has interesting implications both for AI and neuroscience. Firstly, this discovery validates distributional reinforcement learning – it gives us increased confidence that AI research is on the right track, since this algorithm is already being used in the most intelligent entity we're aware of: the brain.

Learn more with Brilliant. Get 20% off today.