kitchen table math, the sequel: Gladwell on Kaplan

Sunday, July 18, 2010

Gladwell on Kaplan

From the moment he set up shop on Avenue K, Stanley Kaplan was a pariah in the educational world. Once, in 1956, he went to a meeting for parents and teachers at a local high school to discuss the upcoming S.A.T., and one of the teachers leading the meeting pointed his finger at Kaplan and shouted, “I refuse to continue until THAT MAN leaves the room.” When Kaplan claimed that his students routinely improved their scores by a hundred points or more, he was denounced by the testing establishment as a “quack” and “the cram king” and a “snake oil salesman.” At the Educational Testing Service, “it was a cherished assumption that the S.A.T. was uncoachable,” Nicholas Lemann writes in his history of the S.A.T., “The Big Test”: The whole idea of psychometrics was that mental tests are a measurement of a psychical property of the brain, analogous to taking a blood sample. By definition, the test-taker could not affect the result. More particularly, E.T.S.’s main point of pride about the S.A.T. was its extremely high test-retest reliability, one of the best that any standardized test had ever achieved. . . . So confident of the S.A.T.’s reliability was E.T.S. that the basic technique it developed for catching cheaters was simply to compare first and second scores, and to mount an investigation in the case of any very large increase. E.T.S. was sure that substantially increasing one’s score could be accomplished only by nefarious means.

But Kaplan wasn’t cheating. His great contribution was to prove that the S.A.T. was eminently coachable—that whatever it was that the test was measuring was less like a blood sample than like a heart rate, a vital sign that could be altered through the right exercises. In those days, for instance, the test was a secret. Students walking in to take the S.A.T. were often in a state of terrified ignorance about what to expect. (It wasn’t until the early eighties that the E.T.S. was forced to release copies of old test questions to the public.) So Kaplan would have “Thank Goodness It’s Over” pizza parties after each S.A.T. As his students talked about the questions they had faced, he and his staff would listen and take notes, trying to get a sense of how better to structure their coaching. “Every night I stayed up past midnight writing new questions and study materials,” he writes. “I spent hours trying to understand the design of the test, trying to think like the test makers, anticipating the types of questions my students would face.” His notes were typed up the next day, cranked out on a Gestetner machine, hung to dry in the office, then snatched off the line and given to waiting students. If students knew what the S.A.T. was like, he reasoned, they would be more confident. They could skip the instructions and save time. They could learn how to pace themselves. They would guess more intelligently....The S.A.T. was a test devised by a particular institution, by a particular kind of person, operating from a particular mind-set. It had an ideology, and Kaplan realized that anyone who understood that ideology would have a tremendous advantage.

Examined Life
Malcolm Gladwell
The New Yorker
December 17, 2001


Anonymous said...

Back around 1981, when I was taking the SAT, I didn't know anyone who ever took an SAT prep class. I seem to recall that they were looked down upon and thought of as needed only by the students who REALLY needed help. This sounds consistent with the story above. Nowadays everyone seems to take them.

cranberry said...

The trouble is, I believe college admissions offices now expect everyone to have taken a prep class. So, there seems to be a rule-of-thumb adjustment applied to SAT scores, depending upon the applicant's demographics. A doctor's child from Scarsdale? Must have had SAT prep (even if he hasn't.)

Lisa said...

When I took the SAT in 1981 no one even told me that test prep existed. My dd took it cold too in 2009. No one seems to have told the rural public schools that it exists now. (She did fine by the way).

Bostonian said...

For a contrary view about the effectiveness of SAT prep courses, please see my comments in Did you celebrate National SAT Day?

Crimson Wife said...

When I graduated high school in an affluent Boston suburb in '95, taking a Kaplan or Princeton Review course was considered a thing for dummies. I did study for the SAT but it was by memorizing vocabulary and working through the College Board's old exams. By the time my youngest brother graduated in '03, it was nearly universal to take a course or hire an individual tutor.

The other big change was the use of paid admissions consultants. I never heard of such a thing but my brother used one as did pretty much all the other kids aiming for selective colleges. The only help I got on my application was critiques of my essay by several teachers, my guidance counselor, and the editor of the paper where my mom worked.

Catherine Johnson said...

wow -

they all used paid admissions consultants?


Catherine Johnson said...

I've just re-skimmed Bostonian's comment about test prep.

Now that I'm immersed in this realm, my sense is that 'real' test prep raises scores by a lot; an average increase of 375 is cited in Hack the SAT.

We're talking about a school year's worth of serious preparation, which means constant practice.

Crimson Wife said...

The kids who were planning on going to UMass didn't hire admissions consultants but the ones in my brother's class who were aiming for a "name brand" college almost universally did. My parents were very skeptical about it at first because I'd gotten accepted to Stanford doing the application myself. But then they started talking to other parents and found out that it was indeed the new norm. So at that point, it would've put my brother at a major disadvantage NOT to hire one since basically all his peers were taking advantage of the packaging.

The really insane thing is that there are now admissions consultants for KINDERGARTEN in certain cities. I was given the card of one by the wife of one of my DH's colleagues when he was considering taking a position in the firm's Manhattan office.

lgm said...

>>We're talking about a school year's worth of serious preparation, which means constant practice.

This indicates that mastery did not occur when the student took the course originally.

Catherine Johnson said...

well....I wonder about that (mastery occurring originally)

First of all, I assume all of these kids are attending private school. (Cost of one year high-end SAT prep is $45K) So I'm guessing they have more mastery than many public school kids.

Second, the SAT test is HARD. I score 790 - 800 on verbal reasoning samples, but I have to read closely and think to do it. I assume the same is true for math.

Third, these kids are prepping math, reading, AND writing ---- it's not 9 months devoted to math alone (or to reading alone...)

All of the commentary I've seen thus far seems to agree that a student can be very good at math and still need to prep for SAT math.

Anonymous said...

I like to think of myself as smart (not anywhere close to genius-level, but above average) and I managed 750 math, 760 verbal on my first (and only) SAT back in the late 80's - without any other prep than a few of the ETS practice tests.

I'm sure it helped a lot that I was in the "accelerated" math sequence, taking Calculus in 12th grade, and had been a voracious reader since elementary school. But it demonstrates that mastery of math (calculus wasn't covered that I remember) and a large internalized vocabulary will do wonders for SAT scores.

I suspect a lot of even the one-year Kaplan programs is trying to get kids to mastery - unconscious knowledge - of math and vocabulary that they supposedly have been learning for their entire previous educational "careers". So in that sense, I think it is great that Kaplan manages to give their studends these gains, although it is unfortunate that their services are necessary.