Credit: Photo by Adrian Korte on Unsplash

by Ruairi J Mackenzie

In a pair of new studies, researchers have shown how our brains continue to activate in response to music, even when listening to gaps between notes and when imagining music without sound.

When we listen to music, we might feel our skin tingle, a rush of nostalgia or perhaps a sudden desire to stick our fingers in our ears. All these reactions are mediated via our brain, which has its own suite of responses to musical stimuli.

Great expectations
Studying how the brain responds to different passages of music is a tough task for brain researchers, as many imaging techniques, such as magnetic resonance imaging (MRI), cannot record quickly enough to capture the rapid changes in activity produced by our brain in response to music. But over time, scientists have identified that our brain’s expectations and predictions about music play a key role in how we enjoy and feel about it. Our brains show activity before a note is played, and if the resulting sound fits into an expected music trajectory, it will produce less of a neural response than jarring or unorthodox ones. Guilhem Marion, a École normale supérieure of Paris (ENS) researcher and study co-author, explained in an interview with Technology Networks how this process of balancing expectations and reality is commonplace in the brain. “Predictive signals, according to the predictive coding framework, are supposed to cancel actual sensory signals coming, for instance, from the ear. Think when someone is touching your arm without you notifying and you touching your own arm: you will feel it more when someone else is touching you, that’s because you have not been able to send an accurate predictive signal,” said Marion. 
Researchers at the ENS, part of PSL University, have gone a step further into the exploration of our brain’s response to music. In a new body of work, they ask  an unusual question that takes full advantage of the brain’s predictive abilities: what is our response to music that can’t be heard?

In a pair of studies published in the Journal of Neuroscience, ENS researchers, including Marion and Dr. Giovanni DiLiberto used electroencephalography (EEG), an electrophysiological technique that measures electrical signals from the brain via electrodes placed on the surface of the scalp, to measure brain signals taken from 21 trained musicians. The musicians were recorded when listening to periods of silence within melodies taken from the works of Johann Sebastian Bach and when imagining Bach’s melodies while sitting in silence.

All in the imagination

The latter technique is called musical imagery and has been studied in detail previously. Marion and DiLiberto wanted to look specifically at how brain activity during audible music varied from that produced by imagined music and whether the predictions our brain makes about a heard melody are preserved when there is no melody to hear.

The researchers knew that time (specifically keeping their participants in time with music while imagining it) would be key to the success of the study, and so they wired up their volunteers with tactile metronomes that vibrated at a steady beat to keep their imagined music at a steady pace. In the first paper, the researchers played 4 different Bach melodies 11 times each to their participants, who were allowed to read the sheet music. Then, they were placed in the same scenario but with the music switched off.

The EEG recordings from the study showed that, when imagining music, volunteers’ brain activity was slightly delayed and had reversed polarity – peaking and troughing in reverse – when compared to the same activity in response to heard music. This finding was no suprise to Marion, who pointed out that this was in line with theories about how the brain makes predictions: “[Imagery signals’] nature would be to suppress the sensory signal. In other words, summing two signals of an inverse polarity would result in a diminished, almost null, signal which is the main primary role of a predictive signal.”

These differences, however, happened in such a predictable fashion that the researchers were able to identify the heard melody just by analyzing the EEG signals produced during the imagery sessions.

The researchers estimated expectations by building a statistical model that was fed with a huge variety of Western music. This model enabled the researchers to assess how well their participants’ brains were predicting the notes they heard or imagined. Their analysis showed that, regardless of whether music was actually audible, their volunteers’ brains predicted it in a very similar way.

A Bach-breaking study

The researchers’ companion study looked at a more natural form of silence – the rests and pauses written into Bach’s melodies. Previous studies on this topic tended to artificially cut out notes from a melody, leaving unexpected silence. But Marion and DiLiberto wanted to study the structured silences that naturally crop up in musical structures.

Studying these silences during both heard and imagined music, the researchers noted the resulting brain activity was extremely similar – written silences produced the same inverted polarity in brain activity as imagining music did.

These similarities allowed the team to produce a unifying theory on how we predict music: the brain produces a signal prior to hearing a note that is then subtracted from the activity produced when the note is actually heard. In the absence of the note, such as when a pause occurs in music or when one is simply imagining the music, subtraction isn’t possible. This, the authors outline in their discussion, explains why the polarity of the signal is reversed in these moments of silence.

The authors therefore conclude that our enjoyment of music is more complex than just a kneejerk reaction to sound. It  is a constant processing that happens even when there is nothing to hear but the whirr of our brains’ prediction engine.

References: 

Marion G, Liberto GMD, Shamma SA. The Music of Silence. Part I: Responses to Musical Imagery Encode Melodic Expectations and Acoustics. J. Neurosci. 2021;49. doi: 10.1523/JNEUROSCI.0183-21.2021

Liberto GMD, Marion G, Shamma SA. The Music of Silence. Part II: Music Listening

Induces Imagery Responses. J. Neurosci. 2021;49. doi: 10.1523/JNEUROSCI.0184-21.2021

Via