Posts
I should have made clear in the post I wrote about the post Debbie wrote* that this question is not "tricky" for me. This question is easy for me, and the fact that this question is easy for me tells you nothing about whether I have mastery of quadratic equations and their graphical representations. At this point, I do not.
This question is easy for me because I have some basic understanding of shifts, because I have memorized the rules about shifts listed in all of the SAT test prep books (and most notably in Phillip Keller's book), and because I see the intended trick of the question the minute I look at it.
The fact that this question is easy for me probably does tell you I am pushing 700 on SAT math, which I am. I scored 680 on the real test, putting me at the 90th percentile for all test-takers; on sample sections my range is well into the 700s. I don't know the exact percentage of test-takers in the 90th percentile who get this question right and find it easy, but it's going to be very high. The top 10% of test takers is the group for whom this question is not tricky.
This question is tricky for test-takers scoring in the 500s, and the College Board knows it. They tell us they know it; they're not keeping it a secret. When the College Board assigns a "Medium" level of difficulty to an item, they are telling us that x number of kids scoring in the 500s will reliably get the question wrong, and x number of kids scoring in the 500s will reliably get the question right. That is the meaning of the words "medium level difficulty."
Kids scoring in the 500s are the ones you can depend upon to see a in f(x) = a - x2 and think, in a certain percentage of cases, a-the-coefficient-of-x2-in-ax2+bx+c=0. Those are the kids getting tripped up by this question.
Test-takers scoring in the mid-500s do not have mastery of quadratic equations and their graphical representations, and neither they nor anyone else is claiming they do. So when a designated percentage of kids scoring in the 500s get this question wrong, we learn nothing about them we did not already know. By the same token, when a designated percentage of kids scoring in the 500s get this question right, we also learn nothing about them. As a general rule, 500-scoring kids do not have mastery of quadratic equations and their graphical representations. I think that is a safe assumption to make.
Ditto for me. When I easily get this question right -- and, again, I stress the word easily -- no one knows anything about my level of mastery of quadratic equations and their graphical representations. I do not have mastery at present, yet for me this question is so easy it's a gimme. In Steve H's words, I have (near)-mastery of the test, not the math.
So why is this question -- or, rather, questions just like it**-- on the test?
Questions like this are on the test because they have appeared in the experimental sections of previously-administered SAT tests and have been found to reliably produce a certain level of error. The College Board needs test items that produce reliable levels of test-taker error in order to keep raw scores stable from test to test. They can't have everyone getting all the questions right; they can't have everyone getting all the questions wrong; and, in the big, mooshy middle, they need items that reliably produce a certain percentage of right and wrong answers.
Hence the a-question.
** I don't know whether items in Online Course have been tested in Experimental sections. As I understand it, all items in the tests actually administered to real test-takers have indeed been tested in experimental sections taken by other test-takers.
53 comments:
1) Well said
2) This is "it" in a nutshell, I think: "Questions like this are on the test because they have appeared in the experimental sections of previously-administered SAT tests and have been found to reliably produce a certain level of error. The College Board needs test items that produce reliable levels of test-taker error in order to keep raw scores stable from test to test. They can't have everyone getting all the questions right; they can't have everyone getting all the questions wrong; and, in the big, mooshy middle, they need items that reliably produce a certain percentage of right and wrong answers."
Certainly, it is good test design to have questions that reliably produce predictable percentages of right and wrong answers, but that is not all the test designers look for. The probability of getting a question correct should also correlate well with the construct being measured. Since there is no magical way to measure "mathematical ability/achievement", this usually means that the question should correlate well with the score obtained from using other questions.
SAT test designers do more than that—they need to have questions that have different difficulties, so that information is obtained at all levels. A "medium difficulty" question is intended to separate students around 500 or 600—students at the high end should consistently get it right and students at the low end should do not much better than chance.
A question can be considered a trick question if it is more likely that a student will put in a particular wrong answer than the right one. I'm not convinced that the distractor here is that strong, though I'm sure that College Board looks for that effect when they are calibrating new questions.
"The probability of getting a question correct should also correlate well with the construct being measured."
What is the construct being measured? How close is the correlation? Is this some sort of absolute scale of what people know in mathematics? Can a college look at a score and predict what level of math that student can get to? Are students accepted or rejected based on that information? No.
When I look at the test, I know I can answer all of the questions. I just get really annoyed at the time factor and the special case solutions. When I rush, I pick some of the wrong answers. Is that because I don't know the material well enough or that I don't have aptitude or that I can't handle math at a tier 1 school? If I prepare specifically for SAT questions, my score will go up. Will this mean that I am better at math? Not really. It means that I'm better at solving the questions on the test.
If the SAT were used only as a general factor in student acceptance, it might not be so bad, but how many wrong answers separate a 700 from a 750? How is that difference used? How can one claim that the SAT has that level of resolution compared to math content or even aptitude?
What this means is that I have to have my son prepare specifically for SAT math questions because he is competing with others who are. This is a contest where there are winners and losers. The SAT defines who they are. There may be a general correlation to content, but their real job is to separate students.
My son does very well in math and loves to explore on his own. He sits in front of GeoGebra for hours. However, I really should have him doing middle school math competition questions. That, apparently, is the game that defines mathematical potential.
SteveH: something like the difference between chess (or go) and the lightning (high speed) versions?
I think your view of math is more like an architect's or engineer's, and mine is more like a plumber's. School math depends on having a big bag of tricks and knowing how and when to use each one.
-- Felix the Cat :)
Could this be a question of vocabulary? When you say a question is "tricky," I assume you're asserting that it misleads the test-taker. I would assert that it's not tricky.
f(x) = a-x^2 accompanying graph shows curve
From the graph, the test takers should know that a must be positive. When x = 0, the line intercepts a positive value on the y axis. Back in the stone age, when I took the SAT, I know I would have looked at the graph, chosen the right answer, and moved on to the next question. There's no need to get tangled up in the further question of what type of equation/curve might this be. When f(0) = a, a is positive.
The College Board isn't trying to trip anyone up. If a student gets this wrong, he doesn't understand what a function is. He can't read a graph at a basic level. This isn't a hard question. The College Board isn't creating a bell curve. If a test is sufficiently challenging, the bell curve emerges, for reasons which are not related to the College Board.
The kids scoring in the 500s who can't answer this question are not being misled. The test is revealing that they don't get it. I don't think one needs to drag ax^2 + bx + c into the discussion. If they can't get this question right, they don't understand equations. Quadratic equations are only a subset of the vast areas they don't understand.
Test-takers scoring in the mid-500s do not have mastery of quadratic equations and their graphical representations, and neither they nor anyone else is claiming they do. So when a designated percentage of kids scoring in the 500s get this question wrong, we learn nothing about them we did not already know. By the same token, when a designated percentage of kids scoring in the 500s get this question right, we also learn nothing about them. As a general rule, 500-scoring kids do not have mastery of quadratic equations and their graphical representations. I think that is a safe assumption to make.
But we learn which kids in the population score in the mid-500s, even if they get As in their classes. The SAT allows colleges to calibrate high schools' grading and rigor. If a kid gets As in her advanced math classes, but can't break 550 on the SAT (with superscoring), that tells you a great deal about her preparation.
How fast do you have to be with your bag of tricks? If you know some tricks, does that automatically mean that you are fast when confronted with a new trick? Does it reflect aptitude or preparation?
Back when I taught math, I could always add more questions to a test. I could always add in trickier problems. I could always force the grades down to the point where I saw the complete bell curve. However, I tried to do that without resorting to speed and tricks because they are crude tools for judging mathematical ability. They reflect preparation for the contest. SAT is a high stakes contest that is an arms race between ETS and those who prepare. How many tricks does one have to have in his/her bag for such low level material? Does this bag of tricks translate into a bag of tricks when you get to differential equations?
Do I really need to remember all of the angle relationships between circles and intersecting lines to get a problem done on time? Is that a necessary thing for a good mathematician, or is it just a good thing to win the SAT game? My whole career has focused on geometric modeling and I don't remember all of those relationships. My bag of tricks includes a six book series called "Graphics Gems" consisting of thousands of pages of geometric tricks. It's on the shelf right behind me.
This is a philosophical argument between rule makers and rule beaters. The rule makers claim over and over that the rule reflects or correlates with reality. The rule beaters know better. It's a game with winners and losers and there are benefits within the error of that correlation. If the stakes are high enough, the rewards go to the rule beaters. ETS expects you to prepare. They know it's a game. Woe to anyone who does not specifically prepare for the game.
All tests include what some would call tricks. I've used them myself, but there are tricks and there are tricks. There is also a fine line when it comes to a test with limited time. The "a" trick doesn't bother me much, but the SAT is full of these questions at all levels.
At a low SAT score level, time is perhaps not a problem because the student will not move on until a problem is completely thought out. Then again, the student might completely freak out because of the time and make more mistakes than usual. At a high SAT score level, and with limited material to test, the tricks become trickier. You have to know more esoteric relationships to get done on time. Time becomes critical. This requires specific preparation for the test.
SteveH, has your child taken the SAT yet? Before you fill up his free time with SAT prep, you should sign him up to try the SAT. I suspect he will score highly enough to alleviate any worries. Remember his score will increase from this year's baseline.
If he's taken the PSAT, the score should give you a guide to his probable percentile within his age group.
In applying to colleges, extracurriculars are given great weight. Too much time devoted to prep classes will cut into the time available for other activities. If you're aiming at the Ivy League, etc., high SAT scores won't differentiate him from the rest of the applicants.
700 versus 750 on the math SAT won't make much of a difference, except at "mathy" places like CalTech or MIT. Harvard's 25th/75th percentile numbers for math are 700 and 790 (reading: 690/800; writing: 710/800). They could fill the class with 800 math SAT students, but they don't.
Caltech is different -- their math 25th percentile is 770, while their writing and critical reading numbers are 710 and 700 respectively. But even at MIT, a 740 in math puts you in the 25th percentile (and their reading and writing 25th scores are lower).
In general, if he is scoring in the 700's, he's competitive and should think about which schools to target and about the rest of his application.
He has taken the PSAT in Oct. (as a sophomore) but we don't know the score yet. I gave him a timed practice test (with very little prep) and he got a 210. He got a 70 on math, but he would have gotten 75 with more preparation specifically geared to the test. He got a 62 on critical reading, but that was because he didn't get to 6 questions. He got a 78 on writing. I don't know why he found that section so easy.
I've given him a number of SAT questions and I see exactly where the tricks come into play. Call them special skills if you want. I don't agree that there is little practical difference between a score of 750 versus 700. There is a point of diminishing returns when it comes to specific SAT math prep, but we haven't really begun yet. That point may come fairly quickly, but I don't think so. I think it will take a number of real timed practice tests and careful reviews. It will also take a careful examination of special cases and the memorization of specific formulas.
I know he can handle any math at any college. I just have to make sure that his SAT score reflects that. Then again, he might get a better boost (for a boy) to work on his critical reading score. It's a very competitive game where a little bit more here and there can make a big difference.
SteveH wrote:
When I look at the test, I know I can answer all of the questions. I just get really annoyed at the time factor and the special case solutions. When I rush, I pick some of the wrong answers.
This is the issue I think we parents need to look at.
We have two data points: SteveH makes mistakes AND math/physics majors at NYU and Columbia have enough difficulties with SAT math that they aren't automatic picks for jobs tutoring the test.
Actually, I think you should throw in the 3rd data point, which is that there are more than a few questions on the SAT that people like me, who do not have mastery, get right thanks to SAT test prep.
Oh! Wait!
Here's another issue that concerns me.
Do you all remember Beilock's research finding that people with high working memory (which is pretty close to the same thing as high IQ) have a much more significant choking problem in high stakes situations than people with lower working memory? I **think** Beilock says that the SAT specifically handicaps high-IQ students, but I'll check.
There are enough funky things going on with this test that it concerns me .... especially when you take into account the fact that wealthy parents here are spending at least 25K per child for one year of SAT tutoring. (I think that that's the bottom of the range, too.)
There is no question in my mind that high-end, aggressive, EXTREMELY expensive SAT tutoring can raise scores far above where they would be with garden-variety Kaplan-course test prep.
Chemprof pointed out that colleges are looking for math/science students with scores of 650 and that strikes me as right and reasonable. In that case (chemprof should correct me if I'm wrong), you're using SAT scores as a check on grades and courses taken in high school, and you're working with a score range very close to the one the College Board gives out on the sample tests.
BUT...I'm not so sure admissions offices use scores in this way, and at least in the colleges I'm familiar with the actual professors have essentially no input into admissions or admissions criteria at all.
Admissions offices have to be concerned with US News ranks.
Merit aid decisions are based on SAT scores, another issue.
The kids scoring in the 500s who can't answer this question are not being misled.
The question is: how many kids scoring in the 500s would get this question right if College Board had used a different letter?
That's the issue.
The "a" isn't there by accident; the College Board doesn't do things by accident that I can see.
The "a" is there to produce error via the mechanism of associative interference.
I would be less bothered by this question if the test didn't have the severe time constraints it does; I **suspect** associative interference is much more likely to come into play when working memory is degraded (which it likely is for most test takers).
I guess the other question is: should we assume that students who get this question right with a different letter used for "a" are actually wrong?
Is it right to assume such students have no understanding of functions at all?
I'm sure that would be the case for some, but I don't think we can assume it's the case for all.
We're talking about a high-stakes test that is way too long, that is administered in far too little time, and for which students are allowed no scratch paper.
Given the test conditions, I wouldn't assume that a student who got this question right when it has a different variable understands nothing about functions.
It just occurred to me: I should say something about the "phenomenology of error."
I just remembered a trick I fell for a couple of months ago. C fell for it, too.
The trick was a drawing that automatically made both of us think of the gazillions find-the-hidden-right-triangle problems we'd done.
We both failed to see that, according to the terms of the problem, the triangle we were looking at was equilateral.
That was the mistake the College Board was trying to get us to make, and we made it.
When I realized my mistake, I was horrified. I know what an equilateral triangle is, ***and*** there was nothing in the problem that required any sort of deep understanding of triangles or cubes in order to get the problem right.
I'm certain mathematicians understand equilateral triangles better than I do, but the question was trivial, not deep.
And I fell for the trick.
SAT math makes you feel like an idiot, and after awhile you get tired of feeling like an idiot. At least, I do. Sure, it's a game; it's a video game. You're trying to beat the test.
But it's not a game of my choosing, and that gets old.
The fact that my kid's fate hangs on the results of this test also gets old.
But we learn which kids in the population score in the mid-500s, even if they get As in their classes.
Yes, definitely.
But we can learn the same thing from the ACT, which doesn't write "trick" questions.
This is a philosophical argument between rule makers and rule beaters.
I love it!
The kids I know who have super-high SAT math scores all did test prep, and the test prep was pretty extensive (and expensive), by which I mean the parents hired private tutors.
These are kids scoring anywhere from 710 to 760.
They had LOTS of tutoring.
I've talked to only one boy who told me vaguely that his/her mom bought a copy of Kaplan or Princeton Review and he did a few problems and then blew out the test --- but I'm a bit skeptical of his/her account.
One thing you see in competitive high schools is students telling each other they 'never' study and don't do test prep, etc. when the opposite is true.
The question is: how many kids scoring in the 500s would get this question right if College Board had used a different letter?
If you substituted a greek letter, say, for a? As many, if not more, in my estimation. The Greek letter would throw them for a loop.
You're supposing that they're getting it wrong because they go off on a wild goose chase triggered by associating "a" with the "a" of quadratic equations. I'm supposing that they're getting it wrong because they don't understand basic math. When people point fingers at memorizing math functions without understanding, this is exactly what they're talking about.
The trick was a drawing that automatically made both of us think of the gazillions find-the-hidden-right-triangle problems we'd done.
You can call me overly progressive, but this, and the example of confusing the equation's simple function with quadratic equations, both sound like examples of the negative side of "drill and kill."
There is an intelligence side to it--can you quickly size up a question, and answer it as easily as possible? Overly preparing for the SAT could lead to lower scores, because the weak math student may have *too many* tools at his disposal.
There is no question in my mind that high-end, aggressive, EXTREMELY expensive SAT tutoring can raise scores far above where they would be with garden-variety Kaplan-course test prep.
If so, there would be a study? Somewhere? I know parents believe this. I just don't think there's a correlation between the amount paid a tutor, and the test score. $25K per child for one year of SAT tutoring is nuts.
@SteveH, I would worry more about the critical reading than the math score. I think it is harder to raise the CR score than the math score. It's ironic that the best way to raise vocabulary is to read books, but very few teens take the time to read anymore.
"I would worry more about the critical reading than the math score."
I am. Part of it has to do with more practice with speed. He never had to do this although his comprehension is very good at a slightly lower speed. This annoys me because either speed is important or it isn't. His schools NEVER practiced speed and comprehension together on a short, technical reading selection.
Second, he reads a lot but I am surprised at how many vocabulary words he doesn't know. Reading more won't fix that except in a brute force way. Just the other day, I found that he didn't know what candor meant. He needs to study word lists.
710-760 is not a "superhigh" SAT math score. According to http://media.collegeboard.com/digitalServices/pdf/SAT-Mathemathics_Percentile_Ranks_2011.pdf
6% of college-bound seniors get 710 or above (8% of males, 4% of females).
If you had said 780-800 (top 1%, top 2% males), then "superhigh" might be an appropriate adjective.
Test prep is not essential for a score over 710: my son got 720 at the end of 6th grade with no test prep and very little formal math training.
I think that the hardest part of the SAT is the writing section (which also has the most questionable grading standards). I'll be curious to see how my son does on that, now that he knows what a 5-paragraph essay is. Because he has trouble with writing assignments in school, we will do some test prep for the SAT writing section—mainly doing some timed essay writing on SAT prompts, to keep him from freezing.
I work with geometry all day, but I am surprised at how SAT problems hide isosceles, equilateral, 45-45-90, and 30-60-90 triangles. I find it extraordinarily annoying. With practice, I get tricked less often, but what does that say about my mathematical ability? Does it mean that I am now a better mathematician?
As for SAT prep, there are two types; one to really teach more math (that's good), and one to teach the tricks of the SAT trade, so to speak. The second is not so trivial, but there is a point of diminishing returns. I would never dream of spending what some parents spend.
The issue any student faces is how much (and what type of) preparation is necessary. Others at your level are going to prepare, so you have to do at least the same if you want to stay equal. I don't think this requires a lot of money, but it does require a specialized plan.
I work with geometry all day, but I am surprised at how SAT problems hide isosceles, equilateral, 45-45-90, and 30-60-90 triangles. I find it extraordinarily annoying.
These are the ones that drive me nuts.
They're built on optical illusions, essentially; if you have a normal visual apparatus, you can't see the hidden figure.
Period.
You have to infer the hidden figure and then hold that invisible inferred figure in working memory while also manipulating the other elements of the problem, and you have to do this without enough scratch paper to allow you to re-draw the picture yourself.
If so, there would be a study? Somewhere?
Nope, and there's not going to be. (Although I saw what I think was an informal study of high-end tutoring in...Chance Magazine? Does that sound right? I couldn't get hold of the article, but I recall that it was written by a math professor -- I THINK -- who questioned the College Board data showing that tutoring doesn't work. I'm going to see if I can get hold of that article or at least the abstract....)
I can tell you that we paid for private tutoring of the writing section -- the writing section! with 2 writers in the house! -- and that produced a score gain of 50 points in approximately....two months of once a week tutoring, I think it was.
That's me; you know my name; you can see my picture next to my comment. I'm not spreading an urban legend; I'm telling you that my husband and I paid what I consider a very high hourly fee to an SAT tutor and the result was a 50-point gain.
College Board publishes data and reports implying that my experience essentially never happens.
But it does.
My friends have all done the same.
And, btw, what I see over and over again is that math and writing are tutorable while reading is not. It's VERY hard to budge reading; off the top of my head, I don't think I know any student whose reading scores rose significantly.
Another factoid: I personally tutored a score gain of 70 points in C's math.
And DS knows a gazillion people who have paid Advantage fees. (I think the highest fee is in the neighborhood of $400/hour.)
What I've learned is that it's a mistake to assume the rich are stupid and/or irrational.
Before I dealt with the SAT directly, I thought: "A fool and his money are soon parted."
Now I'm singing a different song!
gasstationwithoutpumps said...
Test prep is not essential for a score over 710: my son got 720 at the end of 6th grade with no test prep and very little formal math training.
Congratulations on your son, who is clearly very smart, and of whom you're clearly (justifiably) proud. But it's inappropriate to hold such an outlier up as evidence that SAT math is easy. In 2010, a 720 on the math section fell in the 95th percentile.
So pick a metric by which you'd be nowhere near the 95th percentile (How fast can you run a mile? How far can you drive a golf ball? Can you play the cello? What about your Halo skills?) and imagine how difficult it would be to reach that threshold without a specialized natural ability.
Manny Ramirez never had to think about hitting when he approached the plate. It was always see ball, hit ball. Does that mean it should be that easy for everyone? Could he just as easily score a 720 on the SAT math section?
Can your son hit a hanging curve ball 400 feet?
Just the other day, I found that he didn't know what candor meant. He needs to study word lists.
If you can get him to do the Vocabulary Workshop books, you should, I think.
They're the best I've seen.
I'm not sure how well word lists work per se ----- I can't quite suss that out.
VW is fantastic because you learn the words in several different ways: dictionary synonym; homonym; analogies; sentence completion; text.
LOVE that series, and I wish I'd insisted C. do it.
Just the other day, I found that he didn't know what candor meant.
The correct spelling, of course, is "Kandor", and it is the name of the miniature city kept in a bottle in Superman's Fortress of Solitude.
... What? Why are you looking at me like that?
If you had said 780-800 (top 1%, top 2% males), then "superhigh" might be an appropriate adjective.
OH C'MON!
The correct spelling, of course, is "Kandor", and it is the name of the miniature city kept in a bottle in Superman's Fortress of Solitude.
You crack me up!
I wonder if DS will mind if I tell the Obama-nonplussed story.
Turns out neither our president nor our friend DS knows the CORRECT definition of NONPLUSSED!!!!!
What is the world coming to?
did I spell that right?
HI - I'M POSTING FOR DEBBIE, WHOSE COMMENT GOT EATEN BY BLOGGER (MUST GET DISQUS!):
Where to begin.....where to begin....
@SteveH "How fast do you have to be with your bag of tricks? If you know some tricks, does that automatically mean that you are fast when confronted with a new trick? Does it reflect aptitude or preparation?"
Exactly!!!
And your Critical Reading issue with the timing is an "exactly!!!" from me too. I'm a very good reader -- I spent decades in book publishing and read many books per week. I still read at least a few books per week -- but I find the timing issue in the Critical Reading section very challenging. When I took the 1997 test last week, there were 5 extra minutes, and it felt like HEAVEN.
Also, Catherine is right about the Vocabulary Workshop books. The BEST I've found, hands down. Both of my kids use them in school, and I've seen their vocab improve a a result.
@Catherine - if that "a" had been a "d" or frankly, any other letter in the alphabet, I wouldn't have been so tricked -- OR, had I had more than 1 minute per question -- OR, had I not been HOURS into a test that's mentally fatiguing beyond anyone who hasn't taken this marathon test recently 's wildest imagination under "test conditions" (i.e. PRESSURE) -- I might have been able to work my way through it. Who knows.
And re that big spend tutoring issue -- first of all, "Advantage" (pronounced in my head with "advantaged" accent) STARTS at $390 for 100 minute session for a newbie tutor. It goes up to $990 (I think that was it. Would have to check my notes to confirm, but it's within $10 of that) for the "experienced" tutors. According to the woman I spoke with at "Advantage" -- the "average" kid spends a year being tutored.
The kids who write to me looking for "tips" for the SAT can't afford a $69 College Board online course. That's just not right. And the schools ARE NOT teaching this as a matter of course. They just aren't.
I've raised my score in reading from a 680 to a 740. Blood sweat and tears + @PWN + @Erica Meltzer.
I think I may have gotten to the bottom of "my issue" last week in a session with @pwnthesat -- which is that the PRECISE definitions of the SIMPLE words int the Critical Reading passages/questions/answers are what give me the most trouble. Not the big words. Think "excavate" -- and be sure to know what the dictionary says and not how it's used in the vernacular.
That said, if I had 5 more minutes on that section I could do a lot better (I'm pretty sure). It's the whole rush rush rushing aspect. What's the big rush? (They need to shave a few scores down.)
And @PWN the SAT re "Congratulations on your son, who is clearly very smart, and of whom you're clearly (justifiably) proud. But it's inappropriate to hold such an outlier up as evidence that SAT math is easy. In 2010, a 720 on the math section fell in the 95th percentile"
AMEN Thanks for your comment.
I'm familiar with Kandor. Also Zan Zee who I believe lived in Kandor. There was a time when I knew more about the history of Krypton than I did the U.S. I imagine my knowledge of Krypton history still exceeds that of U.S. history. Is SAT offering a subject test on Krypton History?
To be clear, the 650 math SAT thing is not something that admissions uses. However, I've seen a couple of studies (no refs, sorry) showing that student completion of a STEM major rises sharply at that value. So when I am involved in giving merit aid, I definitely use it. Further, if you look at higher ranked schools (especially highly ranked state universities), it is pretty clear that their cutoff is near that same value.
The comment that sci/eng grads overcomplicate SAT problems doesn't tell the whole story.
Consider this toy example problem as a metaphor:
Find the product of the following numbers:
1.01,0.40,0.16,1.00,1.21,0.02,0.00,2.10,6.04
Scientists and engineers compute products all the time. They compute products the way products are computed: by multiplication. Nature doesn't care whether the numbers are easy or hard for you to multiply. They are what they are, and you find the product by doing the multiplication.
Those whose math training comes from contest training or SAT prep would look at the above problem and start looking for "the shortcut." Why? Because multiplication usually has a shortcut? No, multiplication doesn't, but contest problems that appear to require a lot of multiplication usually do.
This isn't knowledge of math; it's knowledge of psychology. While the sci/eng grads were studying the behavior of differential equations, SAT tutors were studying the behavior of contest writers. For example, contest problem writers don't like long calculations, so if you see something that appears to require a long calculation, it's some sort of special case with a hidden shortcut that will save you most of the work.
The sci or eng grad knows the math of multiplication by zero as well as the SAT tutor, but he might not spot the 0.00 carefully camouflaged in the list (again, this toy problem is just a metaphor to illustrate the difference in thinking styles), because he would have no reason to look for it. Why would he? He's used to solving real sci/eng math problems, and nature, in general, gives you general cases, not special cases. In most real sci/eng problems, you actually have to do the work.
The SAT tutor then thinks the sci/eng grad "overcomplicates" the problem by doing the work instead of finding and using the shortcut.
(cont'd)
The SAT tutor takes the time to memorize lists of perfect squares and perfect cubes, because if you recognize 27, 64, and 216 in a problem, it usually means that the test writer has created a problem that requires cube roots. Test writer behaviorology.
Scientists and engineers get numbers like 23.7 and 133.4 to work with and have to reason their way through it with no expectation of finding any shortcuts or of being able to tell if they did the right thing by getting a simple answer. If they take a cube root, it's because that's what they had to do, not because they recognized some special numbers.
This creates a situation where real scientists or engineers, well-trained in math, might take too long reasoning their way through the SAT problems and applying general solutions to get a high score, while test writer behavior specialists know how to recognize clues, find the shortcuts, and use special techniques.
Of course, only some SAT problems are of this nature. Both math contestants and engineers are likely to respond similarly to the f(x) = a-x^2 problem, because it's just a typical parabola. There's no trick to look for, just familiarity with graphed functions. A typical SAT question such as "Company A stock is worth $100/share this year, up 20% from last year. What was its price last year?" would not trick most engineers or math contestants into choosing $80, because the general solution to the problem of backing out N% growth is to divide the current value by 1+N%. This is just understanding percentage growth and doing the work of applying the general solution.
So, yes, people with superior math skills but no knowledge of test writer behavioral heuristics may appear somewhat mathematically inferior on the SAT to those with test prep training.
And some problems really are "tricky" in the sense that doing well on them may depend heavily on non-mathematical knowledge.
And, yes, the SAT is still a pretty good measure of mathematical maturity, because you can get a very good score based purely on automatized mathematical reasoning, and a great score if you keep in mind a few simple testing heuristics that wouldn't take someone with strong math skills to learn.
And I suppose I should also say that using the distractor answer of $80 to the question of "$100 is 20% more than what?" might also be called "tricky," but it is of a different nature, because your susceptibility depends entirely on your familiarity with math, not on any non-mathematical knowledge. This means it's doing its job, measuring your math ability.
And, may I add, that if I'd had more time, I'd have written a shorter letter.
Glen: well said.
"...and a great score if you keep in mind a few simple testing heuristics that wouldn't take someone with strong math skills [long?] to learn."
Long? That's the question, isn't it? What is the separation between mathematical knowledge and tricks? (special cases?) It's not just learning heuristics. There are special facts and formulas to learn. There are skills to develop. That's what I'm dealing with now for my son. It's easy to know that you should look for the hidden isosceles triangles, but it's another thing to be able to do it on a problem that takes 1.25 minutes. Maybe it's some other sort of thing, like a hidden congruent triangle. Fifteen seconds go by and you start to panic. It's a skill that has to be developed.
This reminds me of the daily word scrambles I like to do. I've improved so that most of the time I can see the word within a few seconds. I've learned the tricks. However, some scrambles defy me completely. I just don't see the word. This skill has taken a lot of time to develop even though I know what to do.
Also, in terms of real mathematical knowledge, the goal has to be on what is covered by the SAT. You can't use just a general approach to learing math. You have to know exactly what material is covered by the SAT. Some questions are not really math.
If July 4 is a Friday, then what day of the week is Aug. 6th?
This really isn't difficult. What's difficult is if you have never done it before and have to do it in 1.25 minutes. Does this test aptitude? No. It tests preparation.
In a nice world where the stakes and competition are low, "long" is not very long. I could argue that the game is not worth the effort, but this is not calibrated very well. What is the point of diminishing returns? When is enough, enough? I have a feel for this point for my son, and it will be interesting to find out how long is long for him. Right now, it doesn't look short by any means.
There is also the ability to supersize your SAT scores at some colleges. They are playing their own marketing game. My niece got a 34 on the ACT, but one college tried to get her to take the test again in the hopes of getting more scholarship money. You are not just competing for acceptance; you are competing for money.
You would think that SAT Math Achievement scores would trump SAT-Math, but that doesn't seem to be the case. I assume those scores are not part of any college ranking formula.
I know that colleges don't select students based only on their SAT scores. In fact, top colleges could have higher average SAT scores if they wanted. This should help one from getting weird about SAT test prep, but what is that weird cutoff point? You can replace it with all of the other weird stuff students do to pad their resumes. My son has done volunteer work and now I'm trying to make sure we write it all down so it can be submitted to something - I don't know what. I HATE that. Everything has to be a tangible bullet on your resume.
You can under-prepare and be stressed out (or pissed off), or you can over-prepare and be stressed out. I knew that I should have pushed my son to do math contests. I should be pushing him to prepare for the AMC tests. I don't do that. Last night, while playing with GeoGebra, he wanted to know if he could find a function that had a particular curvature definition. I started to talk to him about differential equations. That won't get him into a good college. That's not the game. We have to play "Find the hidden congruent triangle."
"HI - I'M POSTING FOR DEBBIE, WHOSE COMMENT GOT EATEN BY BLOGGER (MUST GET DISQUS!):"
Yes, it's behaving badly. I always copy my comments to the clipboard just in case.
Debbie said:
"-- which is that the PRECISE definitions of the SIMPLE words int the Critical Reading passages/questions/answers are what give me the most trouble. Not the big words."
I have this same trouble. I know many words from reading, but what I think is the definition is not the simple dictionary definition. Testers know this. Some words almost always carry a particular spin. I like to do the Reader's Digest Word Power test, and they are really good at finding those words.
"Those whose math training comes from contest training or SAT prep would look at the above problem and start looking for "the shortcut." Why? Because multiplication usually has a shortcut? No, multiplication doesn't, but contest problems that appear to require a lot of multiplication usually do."
I suppose it depends on the source of the numbers. I did a quick look through the list looking for two specific things:
(a) A zero, and
(b) Pairs that multiplied out to 1
I did no test prep 20+ years ago when I took the SAT (much less common back then, I understand) and didn't prepare for any math competitions, so it isn't like I learned this behavior from either of those two sources.
One way of looking at this is that this is a test-specific strategy.
Another way of looking at this is that I spent 5 seconds considering the problem before I started cranking out an answer.
Thinking is good, though, as long as you don't spend too much time getting oriented. This strategy is more effective if one was adding a series of decimals, because you'd be more likely to find a better order to do the addition.
Is rewarding (or screening for, or however you want to describe this) this behavior a bad thing compared to just rewarding the kids who can crank out the answers the fastest?
It really depends on which of these approaches (crank or quick overview) correlates better with whatever the school is trying to find. My guess is that these sort of trick questions do a better job than simple brute-force-crank problems. Could we have better tests than this? Probably. But what? And how much better would they be at sorting the kids the way that the schools want. One could imagine that the questions individually look really bad, but collectively correlate quite well with whatever the schools want. If so, then the test is a good one.
-Mark Roulo
The vocabulary thing is HUGE. I can't tell you how many students I've had who had trouble with the verbal GRE because after four years of college they still don't have a college-level vocabulary.
Yes, vocabulary is one of the big problems that I see with my students. They can't read their introductory computer science texts because they don't have the vocabulary. Words like "iteration" or "operand" are beyond them. They also don't know how to read carefully. In a recent lab, I asked the students to work on a program to read in a string and determine if its length is even or odd. About a third of the class just sat in frozen horror. As I worked my way around the room, I realized that those students had all missed the key concept : "length of a string". They were trying to figure out how a sentence itself could be even or odd and were writing things like
if (sentence == even)
I am really puzzled by all this SAT paranoia. It clearly hasn't affected MY students! I doubt any of them ever went near an SAT prep course. Is it just that you guys all want your kids to go to Harvard?
"...so it isn't like I learned this behavior from either of those two sources."
Does that mean something else is going on here because you didn't see it personally?
"And how much better would they be at sorting the kids the way that the schools want."
What is the way that schools want?
They could require achievement test scores or substitute AP test scores. Are we talking about math ability or something else?
"One could imagine that the questions individually look really bad, but collectively correlate quite well with whatever the schools want."
The schools want to sort kids using a test that covers a limited domain of material. They hope that it reflects aptitude, but it really reflects specific preparation. College demand is soaring and the competition is getting tougher. Go ahead and not prepare your kids. Maybe it won't help anyway. In my case, I see clearly what special knowledge and skills my son needs.
"Is it just that you guys all want your kids to go to Harvard?"
Yeah, that must be it.
"...so it isn't like I learned this behavior from either of those two sources."
Does that mean something else is going on here because you didn't see it personally?
It means that this approach is not *ONLY* taught and used for things like SAT tests and math competitions. I don't know what the breakdown is. I suspect that no one does.
-Mark Roulo
"I don't know what the breakdown is."
You raise two separate issues; whether these tests are appropriate for some unknown college agenda, and second, whether the needed knowledge and skills can be obtained from something other than SAT prep and math competitions.
I'm not sure what that adds to the conversation. Nobody is saying that there aren't some who don't have to prepare. Most kids, however, need specialized preparation, not just some sort of general math education. And second, SAT math is the game whether you like it or not.
I think the game is being pushed to an unnatural limit, but even ETS makes money off of helping students play the game. They expect you to prepare. Some claim that test prep really doesn't help that much. The advantage goes to the rule beaters.
@Mark, the reason I said it was a "toy problem metaphor" instead of an "example" was because I wanted every reader to immediately see both the general and quick, special case approaches. It's a metaphor for much less familiar problems, where people with strong real-world math skills, who know how to solve such problems correctly, can be at a *speed* disadvantage to those with contest training.
My point in this "Tastes great! No, less filling!" debate is that both sides are right. Your score depends on both math skill and test prep. It depends so much on real math skill that it is very sensitive over most of its range to math skill alone. So, over most of its range, the test does what everyone wants it to do. You can get a high score with great math skill and no test prep at all. But you can also get a high score with good math skill and lots of memorized test prep tips, such as Catherine's "function movement rules." And without good math skills, no amount of test prep will give you a high score.
But at the highest levels, those where elite admissions and elite waitlists are separated, you probably have to have both great math skill and test prep, and because of the time limit and shortcuts issue, it could very well be that the range between 720-800 is more sensitive to a marginal increase in test prep than to a marginal increase in the kinds of math skill more useful to scientists and engineers.
That's frustrating to an engineer like Steve, because (if I'm interpreting it correctly) he wants his son admitted to a very competitive university AND he wants to spend his son's precious remaining math study time on the kinds of math that are likely to be of more value at that university and, at least to some extent, it's hard to see how he can do both.
I think Steve's right. (And, yes, Steve, I left out the word "long", as you guessed). I also think that the claims that 1) the most most important factor in a high SAT math score is strong math skills and, 2) that you can get a high math score with no test prep are also right. So is the claim that you can raise your score by tens of points by test prep alone without significantly improving your knowledge of math. These are not contradictory claims.
@SteveH - Do you know about the USAMTS contest? It's different from the other math contests in that students have a month to solve the problems, there are two or three rounds. Your son might want to look into it, though the round 2 problems for this year are already available. I don't know if you can start in the middle, but it might be OK, and looking at the math would probably be worthwhile.
I'm assuming that he's going to do AMC10 or 12 already.
I realize that USAMTS isn't SAT prep...but the point to getting a high SAT score is to demonstrate skill and stay in the college applicant pool. A high SAT score is not the only way to demonstrate ability and interest colleges.
"USAMTS contest"
I only know a little bit about it. I also know that there are other ways to demonstrate ability other than with SAT, but it's never in place of SAT. At every level, you are competing against others who are willing to outwork you. If you are not trying to get into a top college, you are trying to trade for more money.
"You can get a high score with great math skill and no test prep at all. But you can also get a high score with good math skill and lots of memorized test prep tips,.."
There has to be some test prep at all levels. Otherwise, it's a big risk. What is "some"? Each student has to determine that for him/herself by doing practice tests. ETS expects you to do that. Maybe they think that preparation will even out the "tricks" factor. If so, then that would really put the non-preparers at a disadvantage.
I see too many special case things that my son needs to know and I've mentioned them over various posts. For him, that some is not trivial, and he is a very good math student. That would be my advice to others. Everyone has to practice specifically for the test.
I find the process very annoying, but that doesn't mean that I'm going to pretend that the game doesn't exist or that I will rely on his natural ability or some other source to demonstrate ability. The fact that colleges look at the total student is not the point. He will be competing against others who will be doing everything.
Is this paranoia? No. It's being realistic. Perhaps it would be if I was trying to get him to be something that he isn't. Everyone should try a real timed SAT test and see for themselves exactly what's going on.
"...he wants his son admitted to a very competitive university AND he wants to spend his son's precious remaining math study time on the kinds of math that are likely to be of more value at that university and, at least to some extent, it's hard to see how he can do both."
This shouldn't be difficult if (as a good math student) test prep doesn't take long. That's my point. This process is not necessarily easy even for good students. That can't be assumed unless a good math student is defined by the SAT test. I would disagree strongly with that.
I did another SAT-math test last night and it again struck me how important speed is. It also stuck me that this speed is not simply a matter of general math knowledge or some vague mathematical maturity. Maturity might make you more calm, but it doesn't make up for the speed required on the test.
I approach all problems methodically, even simple ones. SAT speed is never required in the real world. That's what I teach my son. He crossed a big hurdle in 7th grade when he finally slowed down and stopped making many simple mistakes. That's not enough for the SAT. You have to practice speed. Reading speed and comprehension is also critical. You have to practice reading and understanding the wordy problems.
And yes, there are annoyingly tricky and unusual problems. You have to see the shortcuts. This takes practice. Finishing a problem in 1 minute versus 3 is all about practicing these problems, not general math knowledge and maturity. There may be a correlation, but that's not the game going on here. ETS can't tell if correct answers are due to aptitude or preparation.
One might argue that the simple problems give you extra time for the more difficult ones, but after reading the question, being careful not to be tricked, and then then filling in the correct bubble, there is not much time gained. The issue with some simple problems is that you are left wondering why they are so easy. You go back over it because you can't believe it. I still do that. You have to see the trick in each problem to convince yourself that you got it right.
Almost all of the math is not not difficult, so they have to make the test difficult in other ways. Some may grok this easily, but most need to work at it. Practice with real sample tests is huge. You might quickly get to a point of diminishing returns, but you can also use it as the basis of further learning. I would never recommend a general approach unless your fundamentals are really bad.
@SteveH - If you want to know more about the USAMTS contest, the home website is www.usamts.org
Links to past problems and solutions can be found there too.
Now I'm guessing this comment will be eaten, so I'll make it extra short.
Thanks kcab. I'll have him look at it.
Post a Comment