Pac-Man, released in 1980, is regarded as one of the greatest and most influential video games of all time. But what can this retro arcade game tell us about the architecture of the human brain? 

Many studies treat decision-making as a sequence of discrete, independent choices. Dr Tianming Yang of the Chinese Academy of Sciences sees it as something more sophisticated: a structured, hierarchical process governed by underlying principles. 

By studying both humans and macaques playing Pac-Man, he has revealed that both species ‘speak’ a language of problem-solving, though with different levels of complexity. 

He has found that in humans, this problem-solving ability sits in the right brain hemisphere, in a network that strikingly mirrors the left hemisphere’s language centres. His findings suggest that the brain encodes problem-solving strategies using the same computational principles found in language processing.

Dr Yang recently spoke at SWC, and in this Q&A, he discusses his findings.

How did you end up using Pac-Man to study the brain? What was the thinking behind that choice?

In our field, many researchers have been relying on simple behaviour to study intelligence, but I don't think that's sufficient. We know we are capable of solving highly complex problems, and if we only examine the brain through simple decisions, we’ll never truly understand its complexity. 

We wanted a task that's complex and yet intuitive for animals to understand, and Pac-Man is ideal for this.  

In the Pac-Man game, there’s a lot happening simultaneously. You have to collect rewards, you have to avoid danger, all those things. It's very complicated. But the core concepts of seeking rewards and escaping threats are behaviours animals are well adapted to. So we decided to use Pac-Man.

Video of Pac Man game, from the e-Life publication "Monkey plays Pac-Man with compositional strategies and hierarchical decision-making", authored by Qianli Yang, Zhongqiao Lin, Wenyi Zhang, Jianshu Li, Xiyuan Chen, Jiaqi Zhang and Tianming Yang. The video shows a monkey's moving trajectory, actual and predicted actions, and labeled strategies in this example segment of the game. Video shared under the Creative Commons Licence 4.0 

You describe humans and monkeys as solving problems at different levels of complexity. What does that look like when you're watching them play?

If you watch a monkey play, you wouldn’t necessarily realise that's a monkey, or even a bad player; it's clearly an intelligent being playing the game. 

However, detailed analyses reveal some subtle differences between monkeys and humans. The monkey's gameplay is simpler, repetitive and more reactive than humans. Monkeys typically only plan their games about two steps ahead. That's their limit. 

Humans, on the other hand, plan on a longer horizon. They envision how the game will unfold in the long run so that they can plan accordingly. That's the biggest difference. Humans plan further ahead, allowing them to take more advantage of the game mechanics for higher scores. 

The nice thing about Pac-Man is that there isn’t just one way to win. It can be approached simply or with complex strategies, which makes it a perfect tool for us to compare monkeys and humans. 

Your study suggests that the brain organises problem-solving in the same way that it organises language. What's that connection that you've found?

It's not exactly the same – but it is similar. Similar in the sense it involves the same prefrontal and temporal circuitry that are typically associated with language processing. 

However, while about 90% of people use the left side of their brain to process language, we found that for the language of problem solving, as we call it, people use the right hemisphere. This suggests to us that these corresponding brain areas in the left and right brain are performing very similar computations, just for different purposes: language on the left and abstract problem-solving on the right. 

We’ve seen this kind of left-right division in other cognitive functions before, but this is the first time it’s been observed for problem solving.

What surprised you when you started to see the data? 

It's a developing story. 

When we first designed the experiment, we were actually exploring different questions. The first surprising finding is that we could describe both monkey and human gameplay as a sequence of strategies. We essentially found the basic unit of thinking, much like the words in a language. Once you have the words, you can start building sentences. Similarly, by identifying these basic strategies used by both humans and monkeys, we could begin to understand how they combine them to build more sophisticated solutions. 

So finding those basic units was our first major breakthrough. Then, using more advanced analyses on these basic units, we found that the right side of the brain handles this very language-like processing. That was another great surprise for us. We initially thought the brain’s language circuitry would be involved, but it turns out the brain has a specialised circuitry dedicated to problem-solving, which mirrors the language circuitry on the left.

What does that tell us about human intelligence?

It tells us two things. First, there might not be a fundamental difference between monkeys and humans - the basic brain machinery is the same. The underlying structure of our problem-solving is quite similar. 

However, there is a massive difference in complexity. Monkeys are playing at a simpler level and the learning process is extremely slow. They probably will never catch up to humans, even with a lifetime of training. Humans learn very fast. Even a human novice who has only played the game for a couple of hours can become much better than a monkey that has been trained extensively for years.

Where could this research go - are there practical applications, or is it about understanding how our brains work?

Right now, our primary focus is on understanding how the brain works. However, we are also considering how this research could influence AI research. Large language models are very popular right now and seem capable of almost anything. At their core, large language models are highly complex neural networks. What we are learning from the brain is that, while it is composed of neural networks, it also carries out symbolic computation, reasoning and making decisions based on symbols and concepts. This approach hasn't been fully explored in current large language model development. We hope our study might inspire AI scientists to integrate symbolic reasoning into modern models to create more human-like AI.

What's the next piece of the puzzle?

Our next experiment is to go back to monkeys to understand what neurons in the brain are doing for this game. Right now, what we are learning from the brain is based on human functional MRI experiments. As we know, fMRI has relatively low spatial resolution. It can tell us what, in general, a brain area is doing, but not the activity or exact computations of neurons within that area. To get that level of detail, you need electrophysiology. That's the advantage of the monkey experiments. We can perform these precise recordings in monkeys to answer those deeper questions.

Biography  

Tianming Yang, Ph.D., is Senior Investigator at the Center for Excellence in Brain Science and Intelligence Technology (Institute of Neuroscience), Shanghai, Chinese Academy of Sciences, where he has been conducting research on how neural circuits support complex decision making and cognition since 2013; he earned his Ph.D. in Neuroscience from Baylor College of Medicine under Dr. John H. R. Maunsell, and has held research positions as a staff scientist at the National Institute of Mental Health (2008–2013) under Dr. Elisabeth A. Murray, following his postdoctoral study at the University of Washington and Howard Hughes Medical Institute (2003–2008) with Dr. Michael N. Shadlen.