Facing the Music: Reproducibility Studies to Guarantee Scientific Validity

Over the past 10 – 15 years, researchers have become increasingly aware that some key findings across many areas of science have not stood the test of time.  

Dubbed the “reproducibility crisis,” professionals in these fields came to understand that the results from many important scientific studies could not be recreated to verify findings. To do his part in addressing the reproducibility crisis, psychology professor Andrew Oxenham is undertaking a large reproducibility study specifically examining the perceptual and neural effects associated with musical training.  

Spotting the Need

Oxenham first arrived at the Department of Psychology at the University of Minnesota in 2006, but his earlier work in the music industry introduced him to research on the perception of sound. Specifically, he became interested in how people hear, including people with normal hearing, people with some sort of hearing loss, and people with cochlear implants. 

“Hearing loss is a big issue; it affects many people in our society and its prevalence gets a lot higher with age.... It’s particularly important for a society that’s aging like ours is,” Oxenham explains. “I thought it would be good to take the knowledge that I gained about hearing and try to apply it to improving the lives of people with hearing loss.” 

Studies over the past 10 years generated significant media interest by reporting that individuals who received musical training actually have better speech perception in comparison to those who had not received musical training. “The idea is that having grown up taking music lessons might help not just in becoming a better musician, but also in understanding speech in background noise,” Oxenham says. “And the particularly interesting part of that proposal was that having good training might protect you from some of the effects of aging in understanding speech in noise.” 

With the help of a grant from the National Science Foundation, Oxenham and his team at the University of Minnesota have begun the first ever multi-site auditory research study to verify these potentially influential claims.

Establishing Reproducibility

To reproduce the findings of the musical training study, Oxenham and his team practice several research techniques. First, these reproducibility tests take place in more labs than just his own at the U. His multi-site study includes labs at other universities, including Purdue University, University of Western Ontario, Boston University, University of Rochester (including the Eastman School of Music), and Carnegie Mellon University. The more sites working on the research, the more examples can be shown to confirm or disprove the original claims. Oxenham’s research also incorporates more participants. 

According to Oxenham, the original studies were conducted with small pools of around 20 – 30 participants each. “We're going to be testing over 300 participants with varying degrees of musical training to see whether we can reproduce these claims,” he says of his own project. By including more participants and participants of varying musical abilities, the results of the study can be applicable to more people. 

While many of the experiments have been done before, they have not been done in the same group of subjects. Replicating the results in one large group will, therefore, help establish the legitimacy of the earlier findings. By establishing legitimacy, these findings become more usable. 

In Oxenham’s case, usable findings mean potentially implementing musical training early on for many children, so that those who receive the training might strengthen their hearing and speech perception for the future. 

Research for All 

“Science, in general, relies on us being able to trust the results we read about in journals. Basically, money is being wasted if we're reporting results that can't be reproduced. And it's not something that's specific to psychology,” Oxenham says. In an effort to increase reproducibility efforts and increase credibility, Oxenham and scientists in similar fields participate in the Open Science Movement. The movement encourages researchers to submit raw data and hypotheses to be published prior to actual testing, which helps maintain a record of scientific methods. 

“We're working on understanding hearing with a long-term goal of improving [it],” Oxenham says. These reproducibility studies don’t stop at proving the validity of a study. Research like Oxenham’s means better scientific practices and stronger results for all. 

This story was written by an undergraduate student in CLAgency. Meet the team.