Our brains are bombarded with input from multiple senses every day and it is vital to combine this information so that we can navigate our environment.
If you have ever tried to identify an object in a mystery touch and feel box using only your hands, you’ll know how challenging it is without vision!

Integration of touch and vision in the cortex

17 April 2024
Scientists discover multisensory integration occurs in the primary visual cortex in mice

Researchers have revealed how information from multiple senses is combined early in the cortex at the level of primary sensory areas. This multisensory integration was traditionally thought to be carried out by higher brain areas, with the primary sensory areas believed to only process single senses. These findings further challenge the long-held view that primary sensory cortical areas work in complete isolation.

“Multimodal integration is essential for exploring the world. Our brains are bombarded with input from multiple senses every day and it is vital to combine this information so that we can navigate our environment. We found that this integration is already occurring in primary (low order) cortical areas, explained Dr Simon Weiler, Research Fellow in the Margrie Lab at SWC and lead author on the paper.

The study, published in Nature Communications, is the result of a collaboration between scientists at the Sainsbury Wellcome Centre at UCL, Jena University Hospital and Max Planck Institute for Biological Intelligence. The team around Manuel Teichert (Jena) used electrophysiological recordings to listen to brain areas in mice during visual and whisker stimulation, which is the equivalent of touch in mice. They discovered a novel multisensory zone in the primary visual cortex of mice that integrates tactile and visual inputs from a shared sensory space in front of the animal.

Video showing the shared sensory space in front of a mouse where tactile and visual inputs overlap.
© Sainsbury Wellcome Centre


“If you have ever tried to identify an object in a mystery touch and feel box using only your hands, you’ll know how challenging it is without vision! Likewise, the combination of touch with the visualisation of an object can provide significantly more information about the visual object. Combining information from multiple senses is key in humans. It’s also vital in other animals, which use integrated information to explore objects in their environment and catch prey. We wanted to understand how the brain achieves this,” explained Professor Troy Margrie, Associate Director at SWC and joint last author on the paper.

To explore this question, the researchers chose to study two of the most prominent sensory modalities in mice: vision and tactile stimulation through whiskers.

“We knew that in different anatomical areas of the visual cortex, different areas of the visual field are represented. The position of the whiskers cover areas of visual space directly in front and below the eyes. Mice can see most of their own whiskers, so the visual system and whisker system share a common physical space in front of the animal,” explained Dr Weiler.

The team used a number of techniques starting with stereo photogrammetry to precisely map how individual whiskers are represented in 3D space during rest, retraction and protraction. During protraction, when mice are using their whiskers to explore objects, more whiskers come into the visual space. Through another technique called intrinsic signal imaging, the team found that once the whiskers are in the visual space, the visual cortex is suppressed, meaning there is an inhibition of the visual system.

To further confirm this finding, the researchers used c-fos labelling to indirectly measure neuronal activity. The team also recorded from the visual cortex using in vivo electrophysiology and observed that when they stimulated the whiskers, inhibition of the visual cortex occurred.

To understand how the inhibition occurs, the scientists used whole brain retrograde viral labelling of cells, along with serial two-photon imaging, and found neurons directly projecting from the primary barrel cortex (a posterior region of the somatosensory cortex that represents the whiskers in the mouse) to the anterior primary visual cortex.

Summary scheme showing projection from primary somatosensory cortex to primary visual cortex.
© Sainsbury Wellcome Centre

To confirm the function of the anatomical connection between primary barrel and visual cortex, the team used optogenetics to activate the projection neurons and found that this led to strong inhibition in the visual cortex.

The next question for the researchers is to understand the purpose of this inhibition. They are currently testing two different theories. The first is that during a mouse’s interaction with an object, vision is suppressed so that all attention is on the tactile input from the whiskers, which are providing the more important information in that moment.

The alternative explanation is that there is selective inhibition of certain visual features only. This would increase the signal to noise ratio for processing of visual information by shutting down the noise from the visual scene, so the animal can focus on the important object. Thereby vision of the object would actually be sharpened through inhibition, because non-specific input from the visual scene is suppressed.

This research was supported by the Interdisciplinary Centre for Clinical Research (IZKF; Advance medical scientist - Program 11, M.T.) and by funding from the Foundation “Else Kröner-Fresenius-Stiftung” within the Else Kröner Graduate School for Medical Students “Jena School for Aging Medicine” (JSAM); The Wellcome Trust (214333/Z/18/Z; 090843/F/09/Z) and Humboldt Foundation; the German Research Foundation (HO 2156/5-1, HO 2156/6-1); the Max Planck Society; the German Federal Ministry for Economic Affairs and Climate Action (BMWK) within the Promotion of Joint Industrial Research Programme (IGF) due to a decision of the German Bundestag as part of the research project (IGF 22462 BR) by the Association for Research in Precision Mechanics, Optics and Medical Technology (F.O.M.) under the auspices of the German Federation of Industrial Research Associations (AiF); and the German Research Foundation (FOR3004; GE 2519/8-1; GE 2519/9-1).


Read the full paper in Nature Communications: ‘A primary sensory cortical interareal feedforward inhibitory circuit for tacto-visual integration’ DOI: 10.1038/s41467-024-47459-2

Media contact:

For more information or to speak to the researchers involved, please contact:

April Cashin-Garbutt
Head of Research Communications and Engagement, Sainsbury Wellcome Centre
E: a.cashin-garbutt@ucl.ac.uk T: +44 (0)20 3108 8028

About the Sainsbury Wellcome Centre

The Sainsbury Wellcome Centre (SWC) brings together world-leading neuroscientists to generate theories about how neural circuits in the brain give rise to the fundamental processes underpinning behaviour, including perception, memory, expectation, decisions, cognition, volition and action. Funded by the Gatsby Charitable Foundation and Wellcome, SWC is located within UCL and is closely associated with the Faculties of Life Sciences and Brain Sciences.