New Baycrest research reveals that the brain remembers what we see and what we hear in different ways. Visual memories tend to retain fine details, while auditory memories are more likely to preserve the overall meaning of an experience. The findings shed new light on how the brain reconstructs memories and could help inform future approaches to learning, aging and rehabilitation.
“For years, research has shown that we remember what we see and what we hear differently, but this study is among the first to show the brain mechanisms behind that difference,” says Dr. Lei Zhang, Postdoctoral Fellow at Baycrest’s Rotman Research Institute and lead author of the study, which was conducted with co-senior authors Dr. Claude Alain and Dr. Bradley Buchsbaum. “Our research opens the door to understanding not just how memory works, but how we might better support it across different contexts, from learning to aging.”
The study, titled “Different Reliance on Sensory Reinstatement and Internally Transformed Representations during Vivid Retrieval of Visual and Auditory Episodes,” was published recently in The Journal of Neuroscience.
The study found that while both visual and auditory memories rely on a process known as “reinstatement” (in which the brain reactivates patterns from the original experience), the brain emphasizes different kinds of information depending on what is being remembered. Visual memories tend to preserve more detailed traces of the original experience, while auditory memories rely more heavily on reconstructed meaning-based information, or “gist.”
Main study findings:
• Both visual and auditory memories reactivate brain activity patterns linked to the original experience, with stronger reactivation associated with more vivid recall.
• Visual memories preserve more perceptual detail, while auditory memories rely more on reconstructed meaning-based information.
• The brain uses overlapping but differently weighted systems to process memories of sight and sound.
Functional magnetic resonance imaging (fMRI) was used to examine how 25 young adults learned and later recalled naturalistic soundscapes and video clips while their brain activity was recorded. Participants were presented with short audio and video clips arranged in pairs, allowing one clip to later serve as a cue to prompt recall of the other. During memory testing, participants were shown the cue clip and asked to mentally reconstruct its paired match. After each trial, participants rated how vivid their memory was.
Researchers analyzed participants’ brain activity during both learning and recall to compare how visual and auditory memories are formed and reconstructed. The analysis showed that both types of memories reactivated patterns in higher-order sensory regions of the brain when participants recalled past experiences. In other words, the brain partially “replayed” elements of the original experience during retrieval for both sight and sound.
However, the strength and nature of this replay differed across sensory systems. Visual memories were more closely tied to detailed perceptual information, while auditory memories relied more heavily on internally reconstructed meaning-based representations. The findings suggest the brain uses a shared memory system for both sight and sound, but prioritizes different forms of recall depending on the sensory experience.
This is one of the first studies to directly compare how the brain reconstructs memories across sight and sound at the neural level.
“The findings may eventually help researchers better understand how memory changes with aging, how people learn most effectively and how rehabilitation strategies can be tailored for individuals with memory impairment,” says Dr. Alain, Senior Scientist at Baycrest’s Rotman Research Institute. “The study also opens the door to future research exploring whether auditory memory’s emphasis on meaning may offer advantages for long-term retention.”
This study was funded by the Canadian Institutes of Health Research (CIHR) and the Natural Sciences and Engineering Research Council of Canada (NSERC).
About Baycrest
Baycrest is an internationally recognized academic health sciences organization based in Toronto, focused on aging and brain health.
Baycrest brings together a post-acute care hospital, long-term care, senior living, memory care, research and education, supporting older adults, caregivers and others on the journey of aging. This integrated ecosystem allows care, discovery and learning to inform one another, improving lives today while shaping how aging is understood and supported tomorrow.
Guided by a vision of a world where every older person lives with purpose, fulfilment and dignity, Baycrest translates knowledge into practice, advances specialized care and shares its expertise with health systems, partners and communities in Canada and around the world.
The future of aging is here.
For more information, visit: baycrest.org.
Media contact
Natasha Nacevski-Laird
Media Relations Specialist, Baycrest
[email protected]