When you walk into a new café, you very quickly pick things up. After a few visits, you know not only where the counter is, but also how the queue usually forms, which door actually opens, and what your favourite drink costs. You also learn that if you smile and order clearly, you’re more likely to get what you want, fast. In other words, your brain is constantly learning how the world is structured and what to do to get good outcomes. How the brain builds this picture, how it learns and adapts, are questions that draw many people to research.

Currently, neuroscience frames learning as arising from two systems, working in parallel. Reinforcement learning (RL): did my action pay off? And statistical learning (SL): what tends to follow what?

Reinforcement learning was first described in the 1950’s and has dominated neuroscientific studies of learning since. Researchers have uncovered some of the key parts of the machinery and algorithms that drive how we learn when there is a reward at stake or an objective to achieve. 

Statistical learning was described at around the same time as the method by which babies learn the structure of language. Though it is now understood to be involved far beyond language, like figuring out the layout of that new café, it is much less well studied, especially in systems neuroscience.

A new review from Dr Athena Akrami, Group Leader at the Sainsbury Wellcome Centre, Dr Ambra Ferrari, University of Trento, and Dr Floris P de Lange, Radboud University Nijmegen, explores the differences and similarities between RL and SL, and future directions for research. 

It’s all learning

Some researchers argue that learning is learning, see no difference between RL and SL, and propose one mechanism underlying both systems. At the other end of the spectrum, others believe these are completely separate systems in the brain.

For Dr Akrami and colleagues, the two systems cannot be the same – primarily because the aim of each is so different. 

“In statistical learning, the objective is to build models of the world, even if there is no task at hand. Our brains know that down the line, it’s going to be useful to know the structure and regularities of our environments. But reinforcement learning can be thought of as a control problem - we want to find a solution for a specific challenge. We may build a model, but it is always in service of a specific action and goal,” explains Dr Akrami.

These different aims shape what is learnt, too. Back in our café, if you are strictly focused on your goal of getting your coffee, you prioritise information linked to that reward (Reinforcement Learning). You learn exactly where to stand and what to say. But while you wait, your brain is still absorbing the background details: the layout of the tables, the rhythm of the espresso machine, the fact that the front door always sticks (Statistical Learning). Structure is extracted, whether it is immediately relevant to getting your coffee or not.

Statistical learning (SL) and Reinforcement Learning (RL) have different goals: in SL, observers estimate and predict relationships between environmental variables (‘how the world is structured’), without any reward signal or explicit instruction; in RL, agents develop an optimal decision-making policy for reward maximization (‘what to do’)

New paradigms and future research

The reason statistical learning is less well studied is partly that there haven’t been experimental paradigms available. But new research is changing that, with a recent pre-print from Dr Akrami’s group describing a statistical learning task for mice.

The team used the task to show that the hippocampus plays a key role in SL.

“It’s amazing to think that our brains are constantly absorbing patterns from the world around us, quietly building models that guide perception and understanding long before we ever act,” says Dr Akrami.

There isn’t, as yet, a mathematical theory or framework that explains SL. At the moment, a lot of the learning that is described in neuroscience is based on updating error predictions. We make a prediction, and if the world is different from our expectation, an error signal then updates our predictions. Whether the same mechanism is involved in statistical learning remains unknown. 

The role of dopamine, which is crucial for reinforcement learning, also hasn’t yet been explored in statistical learning. Could dopamine carry “teaching signals” even when there is no explicit reward, for example during passive exposure to structured stimuli? Or does SL rely on different neuromodulatory systems altogether?

Another key question is about the links between the two systems of learning. How do reinforcement systems in the brain tap into the rich knowledge that has been acquired by statistical learning? Does SL provide the world model that RL then uses for planning and decision-making, or do they sometimes compete?

“When I started my lab, a part of it was trying to figure out ways to study statistical learning. But I have always had in the back of my mind the question of how statistical learning and reinforcement learning interact. Understanding the differences and similarities is a very timely topic,” says Dr Akrami.

Ultimately, unlocking how these two engines of learning communicate could shift our entire understanding of the brain. It moves us closer to answering that fundamental question: how do we so seamlessly turn our quiet observations of the world into the dynamic actions that shape our lives? The new review maps out what we currently know about each of these engines and outlines a research agenda for uncovering how they work together to support flexible, intelligent behaviour.

Read more

Read the review in Current Opinion in Neurobiology: Where learning paths meet: Convergence and divergence of statistical and reinforcement learning