kitchen table math, the sequel: Steve H on SAT math and math prep

Sunday, October 23, 2011

Steve H on SAT math and math prep

from the comments:
SAT math tries to trick students. You could say that the tricks relate directly to whether or not they really understand math. However, when you add in the time constraints, it really relates to preparation. Is preparation the same as mastery? Yes. Mastery of the test. Is this equivalent to mastery of math or whether you will do well in college math? Not necessarily. There are better ways of determining that than with the limited material included on the SAT. Why not just require students to take the Achievement Test? Look at the AP Calculus grade.

What is it about SAT-Math that is so important? They are trying to test something other than just math knowledge. They think that these tricky questions reflect on how well you think on your feet, but what it really does is test preparation and whether you have seen these questions before. The questions don't reflect on whether you have a wide body of knowledge and skills in math.

They create problems where you have to "see" the shortcut. You get problems with hidden 3-4-5 triangles. Add a time constraint and then what do you call those problems? It's not just about math knowledge and skills. The problem has to do with trying to determine the difference between aptitude and preparation. The tricks may have some basis in meaningful math, but that's not what they are trying to test.

It reminds me of questions companies like to ask at job interviews, like "Why is a manhole cover round", and "How many golf courses are there in the US.?" Preparation can make you look like you have a great aptitude. Preparation is directly related to math knowledge, and that is important, but identifying aptitude is an arms race for something like the SAT. That's causing the tricky problems, not any desire to test a breadth and depth of math knowledge.

In Dick Feynman's books, he talks about how he spent a lot of time in high school learning about all sorts of trick, lateral thinking problems. He would challenge people to ask him questions. There is nothing like preparation to make you look like a genius, although he really didn't need help with that. It really annoyed some of his colleagues.

My son will get to calculus in his junior year and he always gets A's. He still has to prepare for SAT-Math. He can't let others, with specific SAT-Math preparation, seem like they have a better aptitude than him.
and:
They try to trick students in most questions....What bothers me the most are the shortcut problems where using standard math techniques cause you to take too much time. This is supposed to identify aptitude, but it really tests preparation for the test.

There are also the problems where using a brute force or direct counting technique works better than any applied math technique. In some cases, there is no math to apply. One question on a sample PSAT test asked for the number of positive integers less than 1000 which don't have a '7' as one of the digits. (notice - "don't have" and positive integer) This simply checks how well you work under time pressure. Nobody expects you to apply any fancy math to this problem. One of the answers was the "have" solution. This tests preparation and practice, not aptitude or math ability. There may be a correlation between the test and aptitude or math ability, but not to the resolution colleges use it to select students. At the top levels, it correlates to preparation. That's not necessarily a bad thing, but there are better ways of figuring that out.

11 comments:

Debbie Stier said...

"One question on a sample PSAT test asked for the number of positive integers less than 1000 which don't have a '7' as one of the digits. (notice - "don't have" and positive integer) This simply checks how well you work under time pressure. Nobody expects you to apply any fancy math to this problem. "

I KNOW THIS QUESTION!! (pretty sure I got it wrong, UNDER TYPE CONSTRAINTS).

And I feel like it's the archetypal SAT math trick. And there are gillions of them.

For me, they are fun. But the stakes are only as high as the embarrassment I may cause myself by publicly reporting it.

Anonymous said...

Debbie, that question tests your ability to
a) read the question
b) understand place value
b2) understand place value well enough that you really understand how to count

Why is that a math trick?

Anonymous said...

Sorry, i should say: Steve and Debbie, why is that a trick?

SteveH said...

I didn't say that this particular problem was a "trick", although some of the answers try to trick you into thinking you're done when you really need to stop and think if you got everything.

I was talking about "brute force". This is not math, in spite of any talk of place value. They are trying to test something other than math. If they wanted to test place value and counting, there are better ways of doing this.

Getting this problem correct has little to do with an understanding of place value. My son took into account the 700's, but forgot the 70's. He came up with a result and found it as one of the choices. It was a stupid mistake. If he had seen a problem like this before, he would have gotten it correct. Can you say that because he got this problem wrong he has no understanding of place value or that it is an indicator of aptitude?

Their goal is to test aptitude, so they don't want to ask direct questions that test knowledge and understanding. This isn't about place value. My whole point is that there is an arms race between a test for aptitude and preparation. In an attempt to test for aptitude within a limited domain of material, they have to resort to what many call tricks. Even though you can work out some connection to math, that's not what they are testing. Since people can see the old tests and prepare, it forces them to come up with more creative and different problems to test aptitude.

As others have said, SAT-Math is not so much a math test as much as it is a SAT test.

Anonymous said...

Why is it a trick? Because one of the most common ways that as student will attack this is to start by listing the numbers that DO contain a 7. And many of those students will go from 7, 17, 27, ...67, 77, 88 ... completely missing 70, 71, 72...

Now, no one thinks that a functioning high school student does not know that the number 72 contains a 7. But ETS knows that many of them will zip right past it.

Why else is it a trick? Because you can use the counting principle, if you realize that each digit has 9 possibilities. You can even let the leading digits be zero, generating the numbers with fewer digits. But at the end, you have to realize that you just counted zero and didn't mean to.

Why do they include these? I believe it is that the SAT secretly DOES believe that it is an IQ test. And I think the colleges believe it too.

SteveH said...

Most all tests include what some people call tricks, and in another thread, I said that I was not bothered much by the 'a' problem. I didn't like the "r^2" problem because of the time limitation. I tie speed with preparation more than aptitude.

For many of the SAT problems, (below a score of 600, maybe), I'm not too sympathetic about the "trick" issue. I think the problem arises because of the limited domain of the test. ETS does not want to see a truncated bell curve at the top end. They have to do something to separate students. That is their main goal. They might say that this is an IQ issue, but I think they know that it's a race between preparation and coming up with new spin of the same material. Deep down, it's not about aptitude or IQ. It's about making sure that the number of 800's don't go up.

Michael Weiss said...

In my humble opinion, the brute force approach to this problem is simply the wrong solution. I agree with Anonymous: This problem should be solved with elementary combinatorics. Each of the three digits can be anything but a 7, so there are 9^3 possible choices. One of those choices is 000, however, which needs to be struck, so the answer is 728, not 729. If a student forgot to eliminate 000, and both 728 and 729 appear as solutions in the list, I would think a reasonably-skilled test-taker would see it as a red flag and go back and re-read the question.

SteveH said...

Do all SAT problems simply apply elementary mathematical formulas or ideas? What exactly is the tipping point for "trick" or "brute-force", or does the SAT NEVER resort to those things? How about the special case problems where some equations drop out and you have to "see" the short-cut ... in 1 minute? Do these problems not exist, or are you just questioning the boundary?

I see it as a fuzzy boundary that is open to argument, but I don't take the position that, at the upper end, the SAT-Math test is not tricky. Preparation is a valuable skill, but in the case of the SAT, it's preparation for taking the SAT test, not general math preparation.

Bonnie said...

This is a great question. This really tests the kind of thinking that is helpful in computer science, where approaches based on combinatorics are very important.

Debbie Stier said...

I shouldn't have said that it was an SAT math "trick" -- because now that I've thought about it more carefully, I take it out of the SAT Math "trick" category (but I do have a whole stack of others that I stick IN that category).

I put it in the SAT Math "puzzler" category. Are you good at puzzling FAST, and under pressure -- that's what this feels like to me. That's what a lot of the SAT math section feels like to me.

And apparently I'm not great at puzzling FAST.

I remember this question, and I understood what they were asking, but I forgot to minus out the 700s and I got it wrong. Had there been a little more time (just a little), I think I would have gotten it right. Dare I say, this one would have been on the easier end, given that I did understand what they were asking.

But I was racing as fast as I could through the section so I that I could finish, and I bumbled it (as I often do).

@anonomyous RE: "I believe it is that the SAT secretly DOES believe that it is an IQ test. And I think the colleges believe it too."

I don't know what the colleges believe, but I can tell you that I did recently take an IQ test, and there was definite overlap between the two tests. I'd say they were first cousins.

Debbie Stier said...

@SteveH re It's about making sure that the number of 800's don't go up.

I agree! It seems to me (mostly in the math) that they layer on the twists and turns to shake a few kids (and me) loose along the way.

Hairpin turn -- ok, we'll shave some points off here....
(thank kind of thing)