The where and when of multisensory decision dynamics published in Nature Communications

In a new study published in Nature Communications, Dr. Delis collaborated with researchers at the University of Glasgow and Bielefeld University to study the neural mechanisms underlying the processing of incoming information from different senses. They showed that decision processes in the human brain drive improvements in perception and behaviour when multisensory information is available.

Recent studies on how the brain combines multisensory information focused on when and where in the brain this information converges. However, when taking the stance of an observer making an everyday decision – such as crossing a busy street– it is still unclear what neural processes underpin the well-known behavioural improvements.

To address this question, the research team employed a well-established visual object categorization task in which early sensory and post-sensory decision processes can be clearly dissociated. Human participants were presented with noisy images and asked to report whether these images showed a face or a car while the participants’ brain activity was recorded using electroencephalography (EEG). In a subset of the trials, the presented visual information was combined with auditory information in the form of face (speech) or car/street sounds. Using single-trial analysis of the electroencephalogram (EEG), the researchers identified well-established neural representations of early sensory and post-sensory (decision) processes in the human brain. Capitalizing on these distinct neural representations, they then showed that acoustic evidence enhances perceptual accuracy via the amplification of post-sensory decision, not early sensory, processes.

To strengthen the mechanistic interpretation of these results, Dr. Delis and his collaborators, for the first time, exploited a neurally-informed model of decision-making behaviour to link single trial behaviour and neural responses, in a clearly interpretable mechanistic context of decision-making. This model perspective confirmed that the multisensory perceptual benefits are best explained by an enhancement of post-sensory decision processes rather than changes in early sensory encoding.