kitchen table math, the sequel: bubble this, part 2

Sunday, November 20, 2011

bubble this, part 2

wow

I just checked my October SAT scores and discovered that, as Debbie predicted, I apparently failed to answer an entire chunk of questions without knowing it:


Moreover, as Debbie also predicted, the questions I didn't answer were (C. says) on the 2nd Writing Section, which is the last section of the entire 11-Section test.*


Debbie really has become the expert on the actual, real-world SAT: what it's like to study for it, what it's like to take it, and even what your score means before the College Board tells you what your score means. (I didn't believe her! Every time the subject of my Writing score came up, Debbie would say: "Section 10. You didn't answer all the questions." I thought that was nuts! Of course I answered all the questions! I obsessively checked and double-checked to make sure I answered all the questions! Wrong.)

For the record: omitting four questions while checking and double-checking to make sure you haven't omitted any questions is not a "careless" error.

Omitting four questions while checking and double-checking to make sure you haven't omitted any questions is a maxed-out biology error. Omitting 4 questions in the final section of the test is the same phenomenon as a runner losing her stride at the end of a real marathon going out way too fast from the gate and falling apart at the end of a real marathon, which Ed says happened to the front-runner in the women's field this year.

By the way, in the run-up to the test, it occurred to me that I needed to start running again. I'm thinking aerobic conditioning would have to help. But I didn't do it.

For parents whose kids will be taking the SAT soon, the bubbling issue is major: it's the reason kids need effective test prep. The SAT is too long and has too many problems jammed into too little time and space** to be considered a test of content knowledge. The SAT tests test prep as well as content knowledge.

High school students need effective test prep for the same reason runners need effective coaching: they need to learn strategy and pacing, especially on the math sections.

I'm a 10
May 4, 2010: mandatory bubbling session
March 8, 2011: bubble this
April 13, 2011: progress report, part 2
May 7, 2011: the return of bubbling errors

* 10 sections that count towards your score plus 1 experimental section that does not count
** Too little space: test font size is too small, white space is too small, desktops in testing centers are too small... I may have left something out.

23 comments:

LynnG said...

Stamina is so important. Building the strength to keep your mental focus is a huge part of the SAT.

I may be going far afield here, but maybe mental stamina is also a predictor of college readiness. As I struggle with an unfocused high school student here at home, I'm seeing the negative results of a kid that simply can't stay on task -- she's too easily distracted and when she's tired, her homework and tests suffer.

College readiness might be about both content knowledge as well as the ability to stay focused and on task for extended periods of time.

Since I'm observing an "n" of 2 kids, I realize my sample size is rather low. But there is something to be said for college kids needing to have mental stamina to succeed.

ChemProf said...

Stamina is definitely an issue for college students. I know that if my final is the first one of finals week that the scores will be higher than if they take it at the end of the week. This is especially an issue in first semester Gen Chem, when most of them have never taken a three hour exam (other than the SAT).

Honestly, I think that is one thing that private schools do much better than the mediocre publics we pull from (and that distinguishes top publics from the other). Students have bigger tests that are worth more, and that puts them in a much better position in college.

SteveH said...

College requires a different type of stamina. Perhaps it's more like character. It's not based on the short time frame mental stamina required by the SAT. Besides, the SAT really relates more to preparation. At the lower test score range, it relates to a good preparation in basic math. At the upper range, it's directly related to preparation for the questions on the SAT. Now that I'm looking at the questions in detail, I clearly see the specialized skills and preparation my son needs. Preparation will reduce the stress and the stamina required. It's the same as training for a sport. You could have lots of potential, but you are competing against peers who are training like crazy. The goal of the SAT is to create questions that separate kids at all levels. If you don't prepare, you are at a disadvantage.

One can claim that this sort of long term preparation is a better indicator of college readiness. One could even claim that that is more of what they are measuring than math. OK. We can play that game, but don't tell me that it has a lot to do with general mathematical maturity.

So, instead of allowing my son to continue to study Fourier Analysis using GeoGebra, I have to have him learn to quickly find the hidden equilateral triangle and immediately tell me the height in radical form. I have to have him learn how the SAT hides the answer. They MUST think that it tests aptitude, but it really tests preparation. Bubbling is just one more skill to learn. It should not be taken for granted.

ChemProf said...

It isn't necessarily different, SteveH. At many places, the freshman science finals are 3-4 hours of bubbling in multiple choice answers. One of my colleagues gives a four hour final, and specializes in horrible multiple choice with five or six possibilities and lots of "A. I and II are true, B. I, II, and IV are true" etc. And students may need to go from his physics final on Tuesday to a Linear Algebra final on Wednesday. Short term mental stamina absolutely matters when finals are 20-40% of your total grade.

ChrisA said...

I have to say that I'm quite happy that our family does not get so obsessed with the SAT. Out older daughter, who is what I would categorize as a standard/average student did just fine. Our younger daughter who is most definitely much more academically oriented will go through about the same level of SAT preperation as the older one did.

When the student loan bubble pops, getting into exclusive schools may not be quite so hard and perhaps higher education won't be quite so expensive. I would recommend a trip to Instapundit and search on the phrase "higher education bubble".


Regarding the NY City Marathon, which I watched a decent portion of the women's race including final few miles, if you call going out way to fast "losing your stride" so be it. I call it (very) bad strategy.

SteveH said...

"When the student loan bubble pops, getting into exclusive schools may not be quite so hard ..."

I've been waiting for the college cost bubble to burst for 30 years. Waiting for the burst is not one of my financial plans.

Also, lower cost doesn't mean that it will be easier to get into an exclusive school. The SAT is designed to separate kids so that exclusive colleges can use it as a big factor of admission. Most of us oldsters (proudly or naively)never prepared one bit for the SAT. You do that now only at your own risk in terms of both acceptance and money. ACT and SAT scores can be traded for money.



"...our family does not get so obsessed with the SAT."

You have to look closely. My job as a parent is to keep the SAT from becoming an obsession for my son. He knows it's important, but my goal is to maximize his score with the least amount of effort on his part. I'm not trying to get him to be something that he isn't. That's perhaps where the word obsession might be used.

Can we pass a law that says that nobody is allowed to prepare for the SAT? I would like that. In lieu of that, a little bit of specific SAT preparation may pay off big in terms of acceptance and money. Unfortunately, you are competing against others who stretch "little bit" to an extreme. You may not like the game (I don't), but you ignore it at your own risk. You could call me obsessed about learning the game, but never my son.

SteveH said...

"At many places, the freshman science finals are 3-4 hours of bubbling in multiple choice answers."


But I don't think that ETS carefully chose that skill as a college readiness indicator. Maybe I'm wrong. Perhaps it's a way to justify how the test has evolved, since they apparently don't like to call it an "aptitude" test.

Is college readiness an aptitude or the ability to work hard? Is it defined carefully by the SAT, or is the SAT just a way to separate students?


My general advice to college students is to NEVER skip a class and ALWAYS do every homework as carefully as you can. You will do fine on the tests. I do, however, remember tests where you look at the first big question and decide to do it later; look at the second question and decide to do it later; repeat process. You can hear the groans. When you get the test back, the first question is what is the average and median.

ChemProf said...

"But I don't think that ETS carefully chose that skill as a college readiness indicator."

ETS never carefully chooses anything, as Catherine has pointed out, but one reason that colleges continue to use the SAT as a readiness indicator is that mental stamina is a real concern (especially for first year college students).

Also, a thought on the higher education bubble from the inside -- it is real, I think, and we will see some deflation sometime, but if I had high school or middle school aged kids, I wouldn't count on it mattering to them. I would however look at targeting schools, either state schools or second tier privates, to maximize my financial aid and let them graduate debt-free regardless of their major. The danger right now, I think, is for students graduating with lots of debt.

Catherine Johnson said...

I may be going far afield here, but maybe mental stamina is also a predictor of college readiness.

Hey Lynn!

The mental stamina required by the SAT is completely different from the mental stamina required by college IN MY EXPERIENCE (which may or may not generalize to most college students in most majors....)

Taking the SAT really is akin to running a marathon. (I've never run a marathon, but Debbie has, and we've talked about this at some length.)

Debbie came up with a great expression last night: after you finish the SAT you are in an "altered state." You're not just exhausted, you're....

I don't have a word for it.

Here's an example.

When I came out of the test -- and let me add that both the reading and the writing sections are "easy" for me -- I felt exhausted and slightly 'dazed.' I went out to the parking lot, got in my car, and....immediately felt anxious (I'm not an anxious person) about all the milling students behind me.

Could I see them all?

Would I back into one?

etc.

I felt **anxious.**

That is not a normal emotion for me, backing out of a parking spot in a crowded parking lot. (Normally I feel 'hyperalert' in that situation, which is completely different.)

I started backing out, and of course there were zillions of cars trying to get back and forth by me, and suddenly a police car honked at me. I was blocking the police car's path.

I was flummoxed.

I hadn't seen the police car there, I didn't know what they were trying to tell me (apart from "don't do that'), I didn't know if I was backing out in the wrong direction, etc.

I don't recall ever experiencing anything like that mental state in college.

Also, I've made my living as a writer, often doing reading & writing that was fantastically difficult for me to do. (Difficult for me in that I was trying to grapple with material I'd never been taught, didn't understand, etc., etc.)

I used to say that Animals in Translation "almost killed me," and it did.

Reading over-my-head studies, writing under huge, huge deadline pressure (my fault, but still) ---- these experiences have never, ever produced the kind of 'altered state' the SAT produced.

NEVER.

Catherine Johnson said...

btw, Advantage, the high-end tutoring firm in NYC, gives clients over a dozen full-length, proctored SAT tests before they take the real thing.

Apparently they actually rent out schools to give the trial tests in.

Catherine Johnson said...

chemprof wrote: Stamina is definitely an issue for college students. I know that if my final is the first one of finals week that the scores will be higher than if they take it at the end of the week.

Oh, yes!

I say that my class is "fading in the stretch."

I definitely see that.

Catherine Johnson said...

Chris wrote:

if you call going out way to fast "losing your stride" so be it. I call it (very) bad strategy.

I should correct that line in the post.

I know almost nothing about running---

It's the same issue with the SAT (SAT math in particular), and people like Phillip Keller constantly warn students against it.

high-end tutors teach pacing and a strategy of **not** going out too fast in the beginning -- which has to be practiced.

You can't just decide "I'm not going to go out too fast in the beginning"; you have to actually develop a sense of how long 45 seconds actually is, and you have to maintain that sense of how long 45 seconds is under the stress of actual testing conditions.

College Board doesn't allow you to use stopwatches that make sounds (and I didn't look into products that might work).

What you need is a double-faced watch thingie that tells you a) how much time you have left and b) how many seconds you've devoted to YOUR CURRENT PROBLEM.

Catherine Johnson said...

As to spending too much time on the SAT ----- NO KIDDING.

I now think everyone should take the ACT.

What I don't know is whether & how taking ONLY the ACT affects the intangibles of college admissions & aid.....

We never thought about it one way or another. The SAT was there; everyone else was taking it (and preparing for it); so Chris took it & prepared for it - and I decided to take it, too.

We've never even talked to his guidance counselor at school....

Catherine Johnson said...

ACT and SAT scores can be traded for money.

ABSOLUTELY.

Chris gets ZILLIONS of solicitations in the mail, some of which tell him he's already definitely going to be getting money if he agrees to come to the school.

His scores- and it's pretty much only his scores - guarantee that if we wanted to, we could have him apply to several 2nd-tier schools & then negotiate the offers back and forth.

We're not doing that **only** because we have free tuition at NYU.

Otherwise, we would be taking those scores to the bank.

Catherine Johnson said...

The SAT wasn't a mental marathon before 2006 (at least, not insofar as I know).

Since the mid-200s they added the 25-minute essay and there's been an arms race on the math sections.

I think I've mentioned that I've been told that the most recent SAT tests are significantly harder even than the Blue Book tests.

As far as I can tell, we've got a major arms race going with Chinese nationals. They are entering US colleges at higher numbers (I'll find that stuff and post), and they are focused exclusively on the SAT (it appears to me).

The College Board has a relationship with the Chinese govt.

Our principal just returned from an 8-day junket to China paid for by China and the College Board.

My feeling is: if that's the way it is, let China have the SAT.

Americans can take the ACT.

Catherine Johnson said...

Chinese Language & Culture Initiatives

Catherine Johnson said...

So, instead of allowing my son to continue to study Fourier Analysis using GeoGebra, I have to have him learn to quickly find the hidden equilateral triangle and immediately tell me the height in radical form. I have to have him learn how the SAT hides the answer. They MUST think that it tests aptitude, but it really tests preparation. Bubbling is just one more skill to learn. It should not be taken for granted.

I think that the time and energy that has to go into SAT prep (or had to in our case) is very, very costly, not just in terms of money, but in terms of time and learning.

C. needed to spend last summer re-taking precalculus (or as much of precalculus as a summer allows).

Instead, he spent his time on middle school math (not a dead loss at all, but still), formal grammar, and learning to write an SAT essay.

Catherine Johnson said...

free advice on bubbling:

(worth a heck of a lot more than you all are paying! that's a JOKE)

I taught myself a 'checking obsession' for correct bubbling, the same way I taught myself, years ago, a 'saving tic' for the computer.

Saving tic: when I'm writing anything on the computer, my fingers automatically hit "Control-Save" without my having to make a conscious decision to hit Control-Save.

It's a tic; it's automatic & automated.

Since I early on discovered that bubbling errors are a major peril with the SAT, I also taught myself an obsessive Bubbling-Check process.

After I bubbled in an answer, I would obsessive check and re-check to make sure I bubbled in the RIGHT answer on the RIGHT line in the RIGHT bubble (the answer sheet always has far more bubbles than you need).

That became so obsessive that I then had to exercise glucose-burning WILLPOWER to tear myself away from checking and rechecking while taking the actual test.

Come to find out, I needed to develop a second CHECKING OBSESSION, which is checking to see that the correct number of bubbles have in fact been bubbled in.

All of this is NUTS.

It is RIDICULOUS.

If you're shooting for high scores, it is also necessary.

Catherine Johnson said...

arrghh---I haven't gotten to this yet...Steve sent me a chart of raw scores in the 700s compared to raw scores in the 600s.

One careless error (or four) when you're shooting for an 800 kills you.

One careless error when you're shooting for the 600s doesn't.

Catherine Johnson said...

Debbie just showed me the Oct 2011 test (I haven't got my copy yet).

Sure enough.

I omitted all 4 questions on the final page of the exam, which means I answered them in the test booklet but not on the bubble sheet.

Jen said...

** My job as a parent is to keep the SAT from becoming an obsession for my son. He knows it's important, but my goal is to maximize his score with the least amount of effort on his part.**

This is it exactly -- though I find that HS students are rarely (though not never) the obsessed ones, it's their parents.

The SAT is a hurdle, just like writing a good essay and responding well in interviews and being able to talk about your activities in a mature and understandable way.

Making your kid think that it's a hurdle that has much to do with anything else is a disservice. Teach them what they need to know, make them do practice tests and correct their answers and understand their mistakes. Send them off to the test well-rested and with snacks (and something caffeine-y as long as it isn't going to make them jittery or crazy).

But then, Steve H just said the same thing only far more concisely!

jaylen watkins said...

Stamina is very essential in our day to day life activity. For working professionals and Home maker as well.


Sample Evaluations

SteveH said...

There are different levels and types of stamina, but the SAT tests only one. It's not the primary type required to be successful in college or in a career. The ETS didn't sit around and try to figure out a stamina variable to add that directly reflected on overall success in college. There might be some sort of correlation, but the main purpose they use it for is separation of students, especially at the top end. Many students can do well enough on the SAT to get into college. They need little stamina for the test. They just need to work on their math, critical reading, and grammar.

With more people preparing so much for the test, ETS has to do something to separate students. Does the stamina required at a particular SAT level correlate to the stamina required for the competition at the college one gets into? No. It's a different kind of stamina. As I mentioned in another post, it has more to do with internal drive and character. You have to go to class and work long hours on homework. The exams will take care of themselves.

It would be better to say that instead of stamina, the ETS is testing your preparation skills. You have to prepare methodically, long and hard for the test. If you are so smart that you need little test prep, that's perhaps not a good sign. What happens when you hit an academic wall in college? Do you have the character and will power to do the work? This is the important kind of stamina.

It would be better to say that the ETS is not testing your stamina on the test, but your stamina for preparation. If you prepare properly, you will do well on the test ... but not necessarily.

Some musicians are enormously successful at home in practice. They can record amazing performances. However, when it comes to a live performance, something always goes wrong. No matter how hard they work at home, it doesn't help. Playing in a lot of concerts and competitions will help, but for some people, that is a huge struggle. It ruins their career chances. The ability to do well under intense pressure is not just a learned skill. I mentioned to Catherine once that you have to be a perfectionist with preparation, but not worry about the performance. That is a very difficult thing to do.

College is not a performance or an athletic contest, but the SAT is. At the upper end, the goal is not to test for college readiness. The goal is to separate students. It's a college acceptance and money contest.