kitchen table math, the sequel: guessing hurts

Sunday, February 27, 2011

guessing hurts

C. and I finished all the Critical Reading passages of Test 2 in the "Blue Book" today.

He scored 700.

We figured that if he had skipped the questions he didn't know instead of answering and getting them  wrong, he would have scored a 720 or 730. (C. remembers 730; I recall 720.)

That's an important difference because 30-points can be make-or-break in college admissions.

Speaking of which, various metanalyses of SAT coaching put the gain from test prep in the vicinity of 30 points:
Does test preparation help improve student performance on the SAT and ACT? For students that have taken the test before and would like to boost their scores, coaching seems to help, but by a rather small amount. After controlling for group differences, the average coaching boost on the math section of the SAT is 14 to 15 points. The boost is smaller on the verbal section of the test, just 6 to 8 points. The combined effect of coaching on the SAT for the NELS sample is about 20 points.

The Effect of Admissions Test Preparation: Evidence from NELS:88 (pdf file)
Derek Briggs
May 1, 2007
Is this a wisdom of the crowd moment?

I've always assumed parents were being more-or-less ripped-off by the test prep companies. But if 30 points matter to colleges, and test prep companies raise scores by 30 points -- and those 30 points matter most to selective colleges -- then parents are right.


The Official SAT Study Guide, 2nd edition

The Official SAT Study Guide, 2nd edition


Crimson Wife said...

I keep hearing these claims of only minimal increase as the result of test prep, but it doesn't square with my own personal experience nor that of my DH, my brother, and many other folks I know. I went up 200 points from the score projected by my PSAT to my 2nd attempt at the SAT my senior year. My DH went up 150 points, and my brother went up by 100 points.

Kaplan and Princeton Review wouldn't stay in business if the SAT wasn't easy to prep.

Jo in OKC said...

I think improving 30 points can matter, but it depends on where your original score was.

I think that there's a point for each school where an improvement no longer helps.

For example, I think there's essentially no difference for admissions in a 2370 and a 2400, no matter which college you're applying to.

John said...

Forgive my ignorance - I know what a SAT test is but being a European - I have never sat one nor am I likely to.
Can I just check - they penalise for wrong answers? That seems bizarre.
I remember getting -20 out of 10 in my first French dictation and it put me off French for many many years.
What is the advantage of penalising wrong answers?

Jo in OKC said...

John --

You get 1 point for a correct answer, 0 points for a blank answer, and -1/4 points for a wrong answer.

The point is supposedly to keep people from benefitting by random guesses.

Lisa said...

Which leads me to the question: If you're aiming for a state school where getting in isn't going to be that hard, is it better to rock the SAT or have stellar grades? I'm thinking in terms of scholarship money.

Anonymous said...

I'm with Jo on this.

Also, the thing is, while you may need a decent score to avoid immediate rejection (depending on who you are), even having perfect scores is no guarantee of admission at a selective school. Having reasonably good scores and really strong other factors is better than having perfect scores and less good other factors (such as teacher recommendations, something special in extracurriculars, lots-o-money).

I also think the USA Today methodology of polling admissions officers may not lead to accurate results. Looking at the stats on who is admitted is more meaningful.
(If you can get the info, I've seen some of it for the local ivy.)

Catherine Johnson said...

is it better to rock the SAT or have stellar grades? I'm thinking in terms of scholarship money

I **assume** SATs trump grades for scholarships in state schools as well as in private schools, but I don't know.

For private schools, SATs are key to merit aid.

ChemProf said...

Merit aid is, in my admittedly specific experience, a matter of exciting admissions officers. So SATs are important at selective schools where everyone's grades are comparable. Your essays and letters are important, to get someone in admissions arguing for your student. For the same reason, contact with admissions officers is important.

In a state school, there may be more merit aid that is given "by the numbers," but since there usually isn't much merit aid at those institutions, I'd still guess that your essay and contact with admissions are important.

Catherine Johnson said...

What I've seen & been told (will have to see if I can dig up real sources on this -- ) .... is that high SAT scores will get a lot of students merit aid **if** they are willing to go to a less selective college.

In other words, less selective colleges will 'pay for' higher SAT scores to pull up their mean SATs.

That may be changing with the move to SAT-optional admissions -- although I think the logic of 'buying' higher scores would still hold.

Catherine Johnson said...

I don't know whether we've talked about it here, but there seems to be pressure on colleges to go SAT-optional in order to raise their mean SAT score.

When a college doesn't require SAT scores, students with the lowest SAT scores tend not to send them in.

Once some schools do that, there's pressure on other schools to follow suit.

SteveH said...

" that high SAT scores will get a lot of students merit aid **if** they are willing to go to a less selective college."

I know examples of this for both SAT and ACT. In addition, one school was trying to get my niece to retake the ACT to see if she could raise her already high score of 34. They dangled more money in front of her if she did. That was pretty obvious. That turned her off completely.

SteveH said...

Does any analysis ever talk about what, exactly, they mean by test prep? When I refer to test prep, I take a much longer approach. There are two parts; one has to do with making sure I provide my son with the education that he doesn't get in school. The second is to help him learn how the tests work and how to prepare for them.

He recently took the AMC/10 test and didn't do very well even though he has gotten A's in all of his math classes. There is something missing. Part of it has to do with speed, part has to do with learning more things, and part of it has to do with practicing test questions. This includes knowing what tricks they will throw at you and seeing more variations of problems.

Most discussions of test prep seem to focus on courses that don't last very long. Of course they won't help very much. I taught a SSAT course where the students came twice a week for one hour after school. This lasted for 6 weeks. I knew that the help would be minimal because their effort was minimal.

Doing a lot of reading and studying a lot of vocabulary is a nice long term goal, but I think it's very important to coordinate or calibrate that with some test taking. Are you missing a lot of words because you've never seen them before, or are you having trouble just knowing what they want? You can generally study a lot of math, or you can use test questions to help guide that learning.