Computer model mimics human audiovisual perception
A new computer model developed at the University of Liverpool can combine sight and sound in a way that closely resembles how humans do it. This model is inspired by biology and could be useful for artificial intelligence and machine perception.The model is based on a brain function first found in insects, which helps them detect movement. Dr. Cesare Parise, Senior Psychology Lecturer, has adapted this idea to create a system that can process real-life audiovisual signals—like videos and sounds—rather than relying on abstract parameters used in older models. His paper is published in the journal eLife.
When we watch someone speak, our brains automatically match what we see with what we hear. This can lead to illusions, such as the McGurk effect, where mismatched sounds and lip movements create a new perception, or the ventriloquist illusion, where a voice seems to come from a puppet instead of the performer. This latest work asks how does the brain know when sound and vision match?Previous models tried to explain this but were limited because they didn't work directly with real audiovisual signals. Dr. Parise, from the Institute of Population Health, University of Liverpool, explains, "Despite decades of research in audiovisual perception, we still did not have a model that could solve a task as simple as taking a video as input and telling whether the audio would be perceived as in sync.
"This limitation reveals a deeper issue: Without being stimulus-computable, perceptual models can capture many aspects of perception in theory, but can't perform even the most straightforward real-world test."
Computer Scientists Awards
For Enquiries: info@computerscientist.net
Website: computerscientists.net
Nominate Now: https://computerscientists.net/award-nomination/?ecategory=Awards&rcategory=Awarde
#sciencefather #researchawards #scientists #researchers #AudiovisualPerception #AIModels #CognitiveComputing #ArtificialIntelligence #Neuroscience #HumanPerception #MachineLearning #AIResearch #ComputationalNeuroscience #BrainInspiredAI #DeepLearning #SensoryProcessing #AIInnovation #CognitiveScience #AudioVisualAI #HumanComputerInteraction #PerceptualComputing #VisionAndSound #AIAlgorithms #MultimodalAI #DataScience #TechInnovation #IntelligentSystems #AIExperiments #AIinNeuroscience
When we watch someone speak, our brains automatically match what we see with what we hear. This can lead to illusions, such as the McGurk effect, where mismatched sounds and lip movements create a new perception, or the ventriloquist illusion, where a voice seems to come from a puppet instead of the performer. This latest work asks how does the brain know when sound and vision match?Previous models tried to explain this but were limited because they didn't work directly with real audiovisual signals. Dr. Parise, from the Institute of Population Health, University of Liverpool, explains, "Despite decades of research in audiovisual perception, we still did not have a model that could solve a task as simple as taking a video as input and telling whether the audio would be perceived as in sync.
"This limitation reveals a deeper issue: Without being stimulus-computable, perceptual models can capture many aspects of perception in theory, but can't perform even the most straightforward real-world test."
Computer Scientists Awards
For Enquiries: info@computerscientist.net
Website: computerscientists.net
Nominate Now: https://computerscientists.net/award-nomination/?ecategory=Awards&rcategory=Awarde
#sciencefather #researchawards #scientists #researchers #AudiovisualPerception #AIModels #CognitiveComputing #ArtificialIntelligence #Neuroscience #HumanPerception #MachineLearning #AIResearch #ComputationalNeuroscience #BrainInspiredAI #DeepLearning #SensoryProcessing #AIInnovation #CognitiveScience #AudioVisualAI #HumanComputerInteraction #PerceptualComputing #VisionAndSound #AIAlgorithms #MultimodalAI #DataScience #TechInnovation #IntelligentSystems #AIExperiments #AIinNeuroscience
Comments
Post a Comment