iewpoint | Admissions
Confessions of an Application Reader
Lifting the Veil on the Holistic Process at the University of California, Berkeley
A HIGHLY qualified student, with a 3.95 unweighted grade point average and 2300 on the SAT, was not among the top-ranked engineering applicants to the University of California, Berkeley. He had perfect 800s on his subject tests in math and chemistry, a score of 5 on five Advanced Placement exams, musical talent and, in one of two personal statements, had written a loving tribute to his parents, who had emigrated from India.
Why was he not top-ranked by the “world’s premier public university,” as Berkeley calls itself? Perhaps others had perfect grades and scores? They did indeed. Were they ranked higher? Not necessarily. What kind of student was ranked higher? Every case is different.
The reason our budding engineer was a 2 on a 1-to-5 scale (1 being highest) has to do with Berkeley’s holistic, or comprehensive, review, an admissions policy adopted by most selective colleges and universities. In holistic review, institutions look beyond grades and scores to determine academic potential, drive and leadership abilities. Apparently, our Indian-American student needed more extracurricular activities and engineering awards to be ranked a 1.
Now consider a second engineering applicant, a Mexican-American student with a moving, well-written essay but a 3.4 G.P.A. and SATs below 1800. His school offered no A.P. He competed in track when not at his after-school job, working the fields with his parents. His score? 2.5.
Both students were among “typical” applicants used as norms to train application readers like myself. And their different credentials yet remarkably close rankings illustrate the challenges, the ambiguities and the agenda of admissions at a major public research university in a post-affirmative-action world.
WHILE teaching ethics at the University of San Francisco, I signed on as an “external reader” at Berkeley for the fall 2011 admissions cycle. I was one of about 70 outside readers — some high school counselors, some private admissions consultants — who helped rank the nearly 53,000 applications that year, giving each about eight minutes of attention. An applicant scoring a 4 or 5 was probably going to be disappointed; a 3 might be deferred to a January entry; students with a 1, 2 or 2.5 went to the top of the pile, but that didn’t mean they were in. Berkeley might accept 21 percent of freshman applicants over all but only 12 percent in engineering.
My job was to help sort the pool.
We were to assess each piece of information — grades, courses, standardized test scores, activities, leadership potential and character — in an additive fashion, looking for ways to advance the student to the next level, as opposed to counting any factor as a negative.
External readers are only the first read. Every one of our applications was scored by an experienced lead reader before being passed on to an inner committee of admissions officers for the selection phase. My new position required two days of intensive training at the Berkeley Alumni House as well as eight three-hour norming sessions. There, we practiced ranking under the supervision of lead readers and admissions officers to ensure our decisions conformed to the criteria outlined by the admissions office, with the intent of giving applicants as close to equal treatment as possible.
The process, however, turned out very differently.
In principle, a broader examination of candidates is a great idea; some might say it is an ethical imperative to look at the “bigger picture” of an applicant’s life, as our mission was described. Considering the bigger picture has aided Berkeley’s pursuit of diversity after Proposition 209, which in 1996 amended California’s constitution to prohibit consideration of race, ethnicity or gender in admissions to public institutions. In Fisher v. the University of Texas, the Supreme Court, too, endorsed race-neutral processes aimed at promoting educational diversity and, on throwing the case back to lower courts, challenged public institutions to justify race as a factor in the holistic process.
In practice, holistic admissions raises many questions about who gets selected, how and why.
I could see the fundamental unevenness in this process both in the norming Webinars and when alone in a dark room at home with my Berkeley-issued netbook, reading assigned applications away from enormously curious family members. First and foremost, the process is confusingly subjective, despite all the objective criteria I was trained to examine.
In norming sessions, I remember how lead readers would raise a candidate’s ranking because he or she “helped build the class.” I never quite grasped how to build a class of freshmen from California — the priority, it was explained in the first day’s pep talk — while seeming to prize the high-paying out-of-state students who are so attractive during times of a growing budget gap. (A special team handled international applications.)
In one norming session, puzzled readers questioned why a student who resembled a throng of applicants and had only a 3.5 G.P.A. should rank so highly. Could it be because he was a nonresident and had wealthy parents? (He had taken one of the expensive volunteer trips to Africa that we were told should not impress us.)
Income, an optional item on the application, would appear on the very first screen we saw, along with applicant name, address and family information. We also saw the high school’s state performance ranking. All this can be revealing.
Admissions officials were careful not to mention gender, ethnicity and race during our training sessions. Norming examples were our guide.
Privately, I asked an officer point-blank: “What are we doing about race?”
She nodded sympathetically at my confusion but warned that it would be illegal to consider: we’re looking at — again, that phrase — the “bigger picture” of the applicant’s life.
After the next training session, when I asked about an Asian student who I thought was a 2 but had only received a 3, the officer noted: “Oh, you’ll get a lot of them.” She said the same when I asked why a low-income student with top grades and scores, and who had served in the Israeli army, was a 3.
Which them? I had wondered. Did she mean I’d see a lot of 4.0 G.P.A.’s, or a lot of applicants whose bigger picture would fail to advance them, or a lot of Jewish and Asian applicants (Berkeley is 43 percent Asian, 11 percent Latino and 3 percent black)?
The idea behind multiple readers is to prevent any single reader from making an outlier decision. And some of the rankings I gave actual applicants were overturned up the reading hierarchy. I received an e-mail from the assistant director suggesting I was not with the program: “You’ve got 15 outlier, which is quite a lot. Mainly you gave 4’s and the final scores were 2’s and 2.5’s.” As I continued reading, I should keep an eye on the “percentile report on the e-viewer” and adjust my rankings accordingly.
In a second e-mail, I was told I needed more 1’s and referrals. A referral is a flag that a student’s grades and scores do not make the cut but the application merits a special read because of “stressors” — socioeconomic disadvantages that admissions offices can use to increase diversity.
Officially, like all readers, I was to exclude minority background from my consideration. I was simply to notice whether the student came from a non-English-speaking household. I was not told what to do with this information — except that it may be a stressor if the personal statement revealed the student was having trouble adjusting to coursework in English. In such a case, I could refer the applicant for a special read.
Why did I hear so many times from the assistant director? I think I got lost in the unspoken directives. Some things can’t be spelled out, but they have to be known. Application readers must simply pick it up by osmosis, so that the process of detecting objective factors of disadvantage becomes tricky.
It’s an extreme version of the American non-conversation about race.
I scoured applications for stressors.
To better understand stressors, I was trained to look for the “helpful” personal statement that elevates a candidate. Here I encountered through-the-looking-glass moments: an inspiring account of achievements may be less “helpful” than a report of the hardships that prevented the student from achieving better grades, test scores and honors.
Should I value consistent excellence or better results at the end of a personal struggle? I applied both, depending on race. An underrepresented minority could be the phoenix, I decided.
We were not to hold a lack of Advanced Placement courses against applicants. Highest attention was to be paid to the unweighted G.P.A., as schools in low-income neighborhoods may not offer A.P. courses, which are given more weight in G.P.A. calculation. Yet readers also want to know if a student has taken challenging courses, and will consider A.P.’s along with key college-prep subjects, known as a-g courses, required by the U.C. system.
Even such objective information was open to interpretation. During training Webinars, we argued over transcripts. I scribbled this exchange in my notes:
A reader ranks an applicant low because she sees an “overcount” in the student’s a-g courses. She thinks the courses were miscounted or perhaps counted higher than they should have been.
Another reader sees an undercount and charges the first reader with “trying to cut this girl down.”
The lead reader corrects: “We’re not here to cut down a student.” We’re here to find factors that advance the student to a higher ranking.
Another reader thinks the student is “good” but we have so many of “these kids.” She doesn’t see any leadership beyond the student’s own projects.
Listening to these conversations, I had to wonder exactly how elite institutions define leadership. I was supposed to find this major criterion holistically in the application. Some students took leadership courses. Most often, it was demonstrated in extracurricular activities.
Surely Berkeley seeks the class president, the organizer of a volunteer effort, the team captain. But there are so many other types of contributions to evaluate. Is the kindergarten aide or soup kitchen volunteer not a leader?
And what about “blue noise,” what the admissions pros called the blank blue screen when there were no activities listed? In my application pile, many students from immigrant households had excellent grades and test scores but few activities. I commented in my notes: “Good student, but not many interests or activities? Why? Busy working parents? And/or not able to afford, or get to, activities?”
IN personal statements, we had been told to read for the “authentic” voice over students whose writing bragged of volunteer trips to exotic places or anything that “smacks of privilege.”
Fortunately, that authentic voice articulated itself abundantly. Many essays lucidly expressed a sense of self and character — no small task in a sea of applicants. Less happily, many betrayed the handiwork of pricey application packagers, whose cloying, pompous style was instantly detectable, as were canny attempts to catch some sympathy with a personal story of generalized misery. The torrent of woe could make a reader numb: not another student suffering from parents’ divorce, a learning difference, a rare disease, even dandruff!
As I developed the hard eye of a slush pile reader at a popular-fiction agency, I asked my lead readers whether some of these stressors might even be credible. I was told not to second-guess the essays but simply to pick the most worthy candidate. Still, I couldn’t help but ask questions that were not part of my reader job.
The assistant director’s words — look for “evidence a student can succeed at Berkeley” — echoed in my ears when I wanted to give a disadvantaged applicant a leg up in the world. I wanted to help. Surely, if these students got to Berkeley they would be exposed to all sorts of test-taking and studying techniques.
But would they be able to compete with the engineering applicant with the 3.95 G.P.A. and 2300 SATs? Does Berkeley have sufficient support services to bridge gaps and ensure success? Could this student with a story full of stressors and remedial-level writing skills survive in a college writing course?
I wanted every freshman walking through Sather Gate to succeed.
Underrepresented minorities still lag behind: about 92 percent of whites and Asians at Berkeley graduate within six years, compared with 81 percent of Hispanics and 71 percent of blacks. A study of the University of California system shows that 17 percent of underrepresented minority students who express interest in the sciences graduate with a science degree within five years, compared with 31 percent of white students.
When the invitation came to sign up for the next application cycle, I wavered. My job as an application reader — evaluating the potential success of so many hopeful students — had been one of the most serious endeavors of my academic career. But the opaque and secretive nature of the process had made me queasy. Wouldn’t better disclosure of how decisions are made help families better position their children? Does Proposition 209 serve merely to push race underground? Can the playing field of admissions ever be level?
For me, the process presented simply too many moral dilemmas. In the end, I chose not to participate again.
Ruth A. Starkman teaches writing and ethics at Stanford and, from 1992 to 1996, taught writing at the University of California, Berkeley.
BERKELEY ON BERKELEY ADMISSIONS
“In general, we have an incredibly successful story to tell about our process,” said Amy Jarich, who has been director of admissions at the University of California, Berkeley, since September.
In an interview, Ms. Jarich responded to some of the issues raised by Ruth A. Starkman in her essay on the training of outside application readers and Berkeley’s admissions process — a process Ms. Jarich calls transparent. (Freshman selection criteria and reports on comprehensive review can be found on Berkeley’s Web site.)
“The training process is tried and true,” she said. “We try to do consistent training that helps people understand the policies and also the practice.”
On the application examples used in training, she said, “we intentionally pick the trickiest cases to norm with, aimed at generating discussion,” after which many new readers have to adjust their scoring.
Noting that reading applications is “an art,” she said that Proposition 209 was a challenge that created the need for readers to separate out in their minds race, ethnicity and gender. “Other factors, like reported family income, do not make the decision for us, but they do inform us as we read in context.”
“We’re very sensitive to the fact that we want to pull in a socioeconomically diverse group,” she said, naming several programs in place to help students graduate.
To further diversify, the chancellor has set a goal that 20 percent of students come from outside California, she said. Calling the in-state/out-of-state argument “so political,” she added: “It’s hard to close your mind to it, but in the review process it’s not a factor.” Nor are candidates compared, she said. “Nobody should say we have too many of one and not enough of another.”
“The student reports to us their G.P.A., and shows us every strength and every marker,” she said. Readers in the application-review stage should not consider “anything that’s out of that student’s control.”
