mPerf: Measuring Workplace Performance Is Just the Beginning

As long as humans have existed, they have evaluated each other. Up until the turn of the 20th century, evaluations were by observation and word of mouth. During the 20th century, the questionnaire method and standardized assessments emerged and reached their apogee. Standardization allowed evaluators to make meaningful and quantitative comparisons. 

Now, a new assessment technology is being invented. “In my opinion, this research is the start of the next development in the evolution of how people can be evaluated and measured,” says Deniz Ones, who holds the Hellervik Chair of Industrial Psychology and is a Distinguished McKnight Professor at the University of Minnesota. 

mPerf

Ones thinks that mPerf, short for Mobile Performance, is a project that has the potential to revolutionize workplace evaluations. The government-funded research represents the collaboration of six Universities: the University of Memphis, the lead; the University of Minnesota; Cornell University; Ohio State; the University of Massachusetts, Amherst; and the University of California, Los Angeles. The purpose of their work is to develop technologies that can persistently and unobtrusively measure and predict work performance. 

“My colleagues on the project are mostly engineers, data scientists, and computer scientists. I’m the only work psychologist in the crowd,” Ones explains.

Ones is currently overseeing a field study, gathering data from 400 employees across several organizations and workplaces. Participants in the study wear a set of sensors, and the subjects’ phones are one of the largest contributors to the data. 

“This is a non-invasive study. We’re not recording conversations or going through internet history and the like,” says Ones. They use the cellular devices and other sensors in different ways. For example, the phone’s microphone can report the amount of decibels in the room. “This might tell us if one, two, a crowd, or a stadium of people are in the room,” says Ones. Mobile phones can also detect ambient light. This reports whether the phone has been sitting in complete darkness, versus in open air, versus artificial indoor light. 

“We, of course, collect these data with full consent of the participants. We also have rigorous privacy, confidentiality, and anonymity protections in place,” says Ones. 

Subjects also wear wrist sensors. The sensors can measure the number of steps someone has walked, and various activities of subjects such as how much time is spent typing, and the position of the arms. Additionally, subjects are offered to wear chest belts for a week of the study. The belt can measure the subjects’ respiration and can be used to infer conversational patterns. 

“We have several hundred sensor-captured variables that yield constantly streaming, incoming data.” From all of the sensors, the mPerf team is getting about 1 terabyte of data per study participant. Overall, 30-40 TB of data are generated per week, when all 400 subjects are participating. 

“Let me put that into perspective,” Ones says, “the Library of Congress is estimated to add the same amount of data to their digital holdings at the rate that we do. But we only have 400 subjects.” 

Ones’ Intellectual Family

Ones is a curious, life-long learner. While there are a plethora of companies and organizations that do research where she could easily work at, she prefers a university setting. 

“I love working with PhD students. I have had more than two dozen of them that have graduated with their PhDs and have gone on to make names and reputations and discoveries of their own . . . It is a satisfaction that is incomparable. To know that their knowledge that started with me, has a life of its own and has grown and will continue to grow to be bigger and better. It is a powerful multiplier effect,” Ones says. 
 
Once a student works with Ones at the PhD level, they become part of what she calls her “intellectual family.” “My students call me for career advice, research advice, to collaborate with each other. . . When projects get too big or unmanageable, we lend each other a helping hand,” Ones says. 

Ones’ intellectual family played a large role in subject recruitment and management for the mPerf study. Getting 400 employed adults in high complexity jobs from various locations around the country to participate in a study for 10 weeks is difficult. “I picked up the phone and called my intellectual family. We made it happen,” Ones says.

Implications of the Research

The first phase of research—the 400 subject data collection—will conclude this summer. The plan then is fully compiling and analyzing the data. The project’s funder, the Intelligence Advanced Research Projects Activity (IARPA), is currently examining the first set of results from the project. “With this research, we’ll tell them what the sensors we utilized can measure, provide them the algorithms to measure and predict work-relevant psychological variables as well as work behaviors,” Ones says. 

In addition to workplace performance, another goal is to see how the sensor data relates to the psychological characteristics of the person. “It can potentially report how conscientious someone is, their anxiety levels . . . I don’t expect everything to be measurable accurately by the wearable sensors we are using yet. As long as progress is made, this project will have pushed the envelope in the science of assessment and evaluation. If at the end of the study, I can present half a dozen psychological variables that we can measure or predict well using the sensors, I’ll be happy.”

“This is a visionary project. It can provide seeds of knowledge and wisdom for all psychological assessments, at work, but I believe also in educational contexts,” Ones says. “It is not inconceivable that not too many years in the future there will come a day when high school students applying to colleges won’t have to take the SAT or the ACT. They’ll be asked to download some apps on their mobile devices and link their wearable sensors and let colleges collect data for a couple of months.”

Once the mPerf project is completed, the information will be made public. “I am a believer [in the] free sharing [of] knowledge, especially knowledge that has been created and data that has been gathered using government resources,” Ones says. This way, other researchers can then build on the current project and the technology can progress. 

“This is just the first opening of the floodgates. We are forging a new path. Pioneering research is not easy or for the faint of heart, but utterly worth the trouble,” Ones says. 

This story was written by an undergraduate student account executive in CLAgency. Meet the team.