WORKING PAPER: “Varying impacts of letters of recommendation on college admissions: Approximate balancing weights for subgroup effects in observational studies” Eli Ben-Michael, Avi Feller, and Jesse Rothstein (August 2021)
WORKING PAPER: “Qualitative information in undergraduate admissions: A pilot study of letters of recommendation” Jesse Rothstein (September 2021)
In the admissions cycle that began in November 2016, UC Berkeley carried out the second year of a pilot experiment with letters of recommendation. Unlike other highly selective universities, Berkeley has never previously asked applicants to submit letters from teachers and guidance counselors. This may limit the information available for use in holistic review, and some at Berkeley think that as the university gets more selective it is getting harder to make informed decisions with the evidence available. Others, however, are concerned that students from disadvantaged backgrounds may not have access to adults who can write strong letters, and that the use of letters will further disadvantage these students.
In the pilot experiment, a subset of applicants was invited to submit letters of recommendation if they wished. Any submitted letters were incorporated into the “second read” evaluations of their applications. CPL evaluated the impact of this on the outcomes of applicants from four groups underrepresented among successful applicants to Berkeley: students from families with low incomes, students whose parents did not attend college, students from low-scoring high schools, and students from underrepresented racial and ethnic groups. CPL used a variety of methods, including a within-subject design that compares application scores when readers had access to letters with scores from a parallel process that suppressed the letters and a regression discontinuity design that exploits sharp distinctions between otherwise similar students in the selection of students to be invited to submit letters.
CPL found that the option to submit letters of recommendation led to a more diverse class of admitted students overall, though many students did not submit letters when given the opportunity to do so. There was little evidence that letters submitted on behalf of underrepresented applicants were distinctive in their language or style. Application readers evaluated underrepresented applicants’ portfolios more positively when letters were available, though this effect diminished as the letters grew stronger. The effect was to raise the admissions chances of underrepresented applicants, concentrated among those in the middle of the distribution of traditional academic credentials rather than among those with the strongest credentials.
Visit the Letters of Recommendation project page for more information.