by 
Scientists at the Ivcher Institute used machine learning to study how people perceive water temperature based on its sound, confirming that this sensory ability, which likely develops through life experience, can be accurately analyzed and classified by algorithms. Source: SciTechDaily.com
Researchers have shown that humans can sense water temperature through sound. This ability likely evolved through exposure to sound and suggests the potential for new sensory maps of the brain and expanded perceptual abilities.
Researchers at the Ivcher Institute for Brain, Cognition, and Technology (BCT Institute) at Reichman University (IDC Herzliya) have discovered a previously largely overlooked perceptual ability. machine learning to shed light on the dynamics of cross-modal perception – the interaction between different sensory modalities. In a recent study, the team investigated whether people can detect the thermal properties of water, such as its temperature, from its sound and whether this perception is conscious.
The team used the principles of multisensory integration (the way the brain combines information from different sensory modalities to form a unified perception of the environment) and explored the potential of multisensory thermal perception. They also used a pre-trained deep neural network (DNN) and a classification algorithm (a support vector machine) to investigate whether machine learning can successfully and consistently classify audio recordings of poured water at different temperatures and create a mapping of these thermal properties that is physically encoded in the sound.
A questionnaire investigated people’s views on cross-modal perception. Participants’ perceptual ability was tested in an online 2AFC task using recordings taken in an ambisonic facility. The recordings were also used to train a machine learning model to perform the same classification. Image credit: Ivcher Institute for Brain Cognition and Technology, Reichman University
“Temperature perception is quite unique compared to other sensory perceptions,” says Dr. Adi Snir, postdoctoral fellow at the BCT Institute and co-author of the study. “For seeing and hearing, we have specialised sensory organs such as eyes and ears, but for temperature perception, we rely on specialised receptors in the skin that respond to different temperature ranges that we perceive as hot and cold. However, in the animal kingdom, we know that snakes, for example, can actually ‘see’ body heat, which allows them to identify prey.”
The question of whether the multisensory perception of temperatures is transferable to humans has been asked before. “Previous studies have investigated this at the behavioral level,” explains Prof. Amir Amedi, founding director of the BCT Institute. “These studies have shown that people can hear a difference between the pouring of hot and cold liquids, but not how or why this is possible,” he explains.
Investigation of multisensory perception
The researchers first wanted to reproduce previous results and confirm this surprising perceptual ability, as well as clarify whether this ability is innate or acquired, a question that has long been the subject of fierce debate. “We also wanted to investigate whether or not people are aware of these differences in sound properties of thermal differences,” says Snir, “and also find out which properties of the sounds themselves allow for differentiation in perception,” he adds.
To achieve this, the team used a pre-trained deep neural network (DNN) to characterize recordings of water pours of different temperatures, a machine learning algorithm to classify the thermal properties of the water, and a computer-based analysis of the acoustic features of each recording. “We found that participants were consistently able to identify water temperature from sound, even when they did not believe they were able to do so. This suggests that this is likely an implicit ability acquired through lifelong exposure to acoustic stimuli,” explains Amedi. “At the same time, the machine learning model trained on recordings of hot and cold water showed high accuracy in classifying sounds.”
The study’s results show that humans have the ability to learn complex sensory mappings from everyday experiences and that machine learning can help explain subtle perceptual phenomena. “The next step is to investigate whether humans develop new sensory mappings in the brain for this experience, just as they do for sight, touch and hearing,” Amedi explains. “In theory, Elon Musk’s recent claims that Neuralink produces superhuman abilities could become a reality if this method is combined with brain stimulation,” he adds.
Reference: “Hearing temperatures: employing machine learning for elucidating the cross-modal perception of thermal properties through auditory” by Mohr Wenger, Amber Maimon, Or Yizhar, Adi Snir, Yonatan Sasson and Amir Amedi, June 28, 2024, Limits of psychology.
DOI: 10.3389/fpsyg.2024.1353490
