Gladwell on Kaplan

From the moment he set up shop on Avenue K, Stanley Kaplan was a pariah in the educational world. Once, in 1956, he went to a meeting for parents and teachers at a local high school to discuss the upcoming S.A.T., and one of the teachers leading the meeting pointed his finger at Kaplan and shouted, “I refuse to continue until THAT MAN leaves the room.” When Kaplan claimed that his students routinely improved their scores by a hundred points or more, he was denounced by the testing establishment as a “quack” and “the cram king” and a “snake oil salesman.” At the Educational Testing Service, “it was a cherished assumption that the S.A.T. was uncoachable,” Nicholas Lemann writes in his history of the S.A.T., “The Big Test”: The whole idea of psychometrics was that mental tests are a measurement of a psychical property of the brain, analogous to taking a blood sample. By definition, the test-taker could not affect the result. More particularly, E.T.S.’s main point of pride about the S.A.T. was its extremely high test-retest reliability, one of the best that any standardized test had ever achieved. . . . So confident of the S.A.T.’s reliability was E.T.S. that the basic technique it developed for catching cheaters was simply to compare first and second scores, and to mount an investigation in the case of any very large increase. E.T.S. was sure that substantially increasing one’s score could be accomplished only by nefarious means.

But Kaplan wasn’t cheating. His great contribution was to prove that the S.A.T. was eminently coachable—that whatever it was that the test was measuring was less like a blood sample than like a heart rate, a vital sign that could be altered through the right exercises. In those days, for instance, the test was a secret. Students walking in to take the S.A.T. were often in a state of terrified ignorance about what to expect. (It wasn’t until the early eighties that the E.T.S. was forced to release copies of old test questions to the public.) So Kaplan would have “Thank Goodness It’s Over” pizza parties after each S.A.T. As his students talked about the questions they had faced, he and his staff would listen and take notes, trying to get a sense of how better to structure their coaching. “Every night I stayed up past midnight writing new questions and study materials,” he writes. “I spent hours trying to understand the design of the test, trying to think like the test makers, anticipating the types of questions my students would face.” His notes were typed up the next day, cranked out on a Gestetner machine, hung to dry in the office, then snatched off the line and given to waiting students. If students knew what the S.A.T. was like, he reasoned, they would be more confident. They could skip the instructions and save time. They could learn how to pace themselves. They would guess more intelligently....The S.A.T. was a test devised by a particular institution, by a particular kind of person, operating from a particular mind-set. It had an ideology, and Kaplan realized that anyone who understood that ideology would have a tremendous advantage.

Examined Life
Malcolm Gladwell
The New Yorker
December 17, 2001