Recommendation Letters and Bias in Admissions

During the first week of the trial in the lawsuit charging that Harvard University discriminates against Asian American applicants, university officials offered a number of reasons to explain why the admission rate for Asian American applicants (who, on average, have slightly better academic credentials) is a bit lower than that of white applicants.

If teachers and counselors favor some groups over others, should colleges be relying on their letters? William Fitzsimmons, dean of admissions at Harvard, said that one reason was that white applicants receive "somewhat stronger" recommendations from their teachers and guidance counselors than do Asian American applicants. A spokeswoman for Harvard, asked if there had been a study behind that view, said that the testimony was "based on the dean's experience in the past 32 years."

Leaving aside the question of whether Harvard discriminates, which the trial will adjudicate, the testimony raises a larger question: Should colleges be using letters of recommendation even if patterns indicate that applicants from some groups consistently get better letters than do others? In the case of Harvard, for example, the favored group (white people) received more enthusiastic letters than did Asians despite not being the best in terms of academics.

Some experts on college admissions believe that letters can add bias and should not be used as they are, while others say that the evidence of bias isn't clear. Notably, some of those experts also say that there is minimal evidence that the letters help colleges make better decisions.

Letters Matter

Each year, the National Association for College Admission Counseling does a survey of colleges on their admissions practices, and the data show that letters matter.

According to the most recent survey, 15 percent of colleges report that the counselor recommendation has "considerable importance," while 46 percent say these letters have "moderate importance." For teacher recommendations, 11 percent of colleges report that they are of "considerable importance" while 46 percent say they are of "moderate importance."

By way of comparison, letters are in the view of colleges surveyed far less important than are the top factors cited by colleges: grades in college preparatory courses, grades generally, the strength of the curriculum and standardized test scores.

But the letters are seen by colleges as more important than such factors as class rank, extracurricular activities and work experience.

Are Letters Fair?

Letters are the norm in competitive college admissions, with counselor letters required even at places that don't require teacher recommendations.

Jon Boeckenstedt, associate vice president for enrollment management and marketing at DePaul University, wrote an essay for The Washington Post in 2016 in which he argued that colleges should cut back on their reliance on letters. (DePaul requires counselor letters but not teacher letters.)

In his essay, Boeckenstedt noted that some teachers are more skilled at letter writing than others (or more connected or have more time) and he noted that some privileged students may be more likely than others to have access to good letter writers.

"Imagine you’re 17 again, and you have to ask teachers to write a letter for you. What do you know about how well a particular teacher writes?" Boeckenstedt wrote. "Do you know what a 'good' letter contains? Do you even understand that the spectrum of writing ability almost certainly varies widely among teachers, as it does with any group of professionals? In all probability, you just have to trust yourself, flying blind in a process you get to go through just once."

He added, "This seems grossly unfair: the letter has virtually nothing to do with the student’s performance, and a lot to do with the teacher’s ability to turn a phrase, note interesting character traits, structure a cogent series of paragraphs … In short, it’s as much about the teacher as the student … It can also be about how much time a teacher has to complete the task, and the extent to which she sees it as a function of her duties. Who is, on average, going to write the better, more complete, and more nuanced letter?

"A teacher from a small college prep school where it’s widely understood that giving students every advantage in the college admissions process is a part of the job? Or someone in a large, public, under-resourced school where the range of abilities in each class is wider, and the number of students to get to know greater, and the teaching load is probably higher? Who is more likely to be able to go to the conferences and the workshops where this specific skill is taught and honed, often under the tutelage of the ones who eventually read the letters?"

Boeckenstedt said last week that he still fears that letters are unfair to many students. "The problem, of course, is that the admissions process is filled with bias, as are most of our human interactions, I suppose," he said. "Some high schools are -- by design -- much better at positioning their students for admission to college, and it’s very hard if not impossible to strip off the advantages some students have to see the essence of the student."

One of the few studies to try to document possible bias in letters of recommendation for college admissions was done by examining letters submitted on behalf of applicants to a competitive public university in the Southeast. The study examined applicants' academic qualifications to control for them, and be sure to be comparing letters of those of comparable academic qualifications. The study -- published in School Counselor -- found that certain positive words about applicants were less likely to be used when describing underrepresented male applicants (but notably not for underrepresented female applicants.)

The study cautioned against reading too much into the study, and urged that similar research be done at a number of institutions.

Many studies have found signs of potential bias in letters of recommendation in higher education, but without a focus on undergraduate admissions. A study published this year, for example, found that letters on behalf of women seeking positions in academe were more likely to include words and phrases raising a doubt than were letters about men.

Evidence of Bias

While high schools are full of teachers and counselors deeply committed to helping students from all racial and ethnic groups, there also have been studies finding bias that could hurt some students.

A study released last year in the journal Education Next, based on data from a longitudinal database of the U.S. Department of Education’s National Center for Education Statistics, found that high school teachers expect 58 percent of white high school students, but just 37 percent of black high school students, to go on to obtain a four-year college degree (and then perhaps a graduate education).

Given that many black children grow up in low-income neighborhoods and attend schools that lack resources, the gap in expectations would not by itself indicate a racial gap in the fairness of teacher perceptions of students.

But the database used for analysis has the views of two teachers for every student, and demographic data on the teachers as well as the students. And here the researchers focused on gaps in the expectations of black and white teachers of the same black and white students. When teachers of different races evaluated the same black student, white teachers were nine percentage points less likely than their black colleagues to expect that student to earn a college degree. This gap was more pronounced for black male students than for black female students.

This year, a study in the journal Sociology of Race and Ethnicity was based on letters sent from fictional black students to admissions officers at predominantly white institutions. The study found that those letters from students who described a desire to be involved with black activism were less likely than others to receive encouragement that the college would be "a good fit" for them.

Ted Thornhill, assistant professor of sociology at Florida Gulf Coast University and author of the study, said via email that his research made him think that the gaps found at Harvard in letters on behalf of Asian and white students "could be a function of the perceptions of the letter writer vis-à-vis the student’s racial salience. Stated differently, teachers and guidance counselors are racial actors who will have their perceptions and evaluations of students, who are also racial actors, influenced by this fact."

He urged colleges to study this issue, rather than assuming that letters of recommendation are a bias-free factor to use in admission. He said that there could well be "racialized patterns whereby those letters written for white students are in some way more favorable than those written for students of color."

Are Letters Valuable?

Given that letters are widely used, and some question their fairness, is there evidence that they add in a significant way (as many college admissions officials assert) to admissions officials' ability to make good decisions?

Nathan R. Kuncel, professor of psychology at the University of Minnesota-Twin Cities, was co-author of a study of all existing research on letters of recommendation in college and graduate school admission, published in 2014 in The International Journal of Selection and Assessment.

He said that there is only "limited" evidence that letters help identify students who can succeed, and he argued that there is a need for much more research on the validity of letters, and issues of potential bias. (Similarly, he said that there is very limited evidence that admissions essays, also highly valued by admissions officers, contribute to better decisions.)

Kuncel said that patterns such as the one identified at Harvard don't provide conclusive evidence of bias. But he said such patterns should encourage colleges to study why one group is getting better recommendations.

Colleges that rely on letters can increase their effectiveness or decrease the chances of bias, he said, by asking for a specific focus on actions, rather than more general rhetoric about how wonderful a student is.

"Behaviorally anchored ratings" are best, Kuncel said. "Does the student participate a lot in class? Does the student bring in extra materials to share? Do the students collaborate and listen to others?" Writing on such themes, he said, is much better than "impressions such as 'this is a good kid.'"