kitchen table math, the sequel: Are Schools Preparing Students Well for the SAT?

Friday, July 22, 2011

Are Schools Preparing Students Well for the SAT?

The College Board's website says in many different places that the best way to prepare for the test is to do well in school:
Keep in mind that the foundation of a student's SAT and college preparation is a rigorous curriculum of English, mathematics, science, history, and other academic subjects. Students should read extensively and develop good writing skills.
I want that for my children.

My question is, are schools really teaching this rigorous academic curriculum that the college board says is the best prep for the test ?

I took my son's 10th grade PSAT the other day, and found myself aghast (again) at how darn hard this test is. I'm not opposed to rigor, by the way; just wondering if our schools are on the same page.

There were passages dealing with Descartes, dualism, genomes, and neuroscience. Students had to compare two passages that were extremely sophisticated, with both authors agreeing on the main point, but from different perspectives, and their distinctions were subtle.

Not easy......

Cross posted on Perfect Score Project

102 comments:

SteveH said...

"the foundation"

My son is getting a decent foudation that is preparing him to do well at almost any college. However, his SAT scores may not be competitive enough to get into some of them, even though his class grades are high.

I won't argue that the material on the SAT is wrong or useless, but it does require extra work that high schools don't provide. It would be nice if schools would take this material more into account, especially in the area of critical reading and writing. In some of these cases, you could argue that the "foundation" criteria is not met.

Bonnie said...

Has the PSAT gotten that much harder over time? Back when I took it, I thought it was really easy, and this was after 3 years in a really bad Kentucky junior high.

Catherine Johnson said...

The PSAT is just a shorter SAT; all of the questions are drawn from SAT tests (as I understand it).

I just took several sections of PSAT math, and I couldn't see any difference between those & the real SAT.

Catherine Johnson said...

I would **love** to get my hands on a copy of one of the old tests.

Actually...I bet I could do that just by buying one of the test prep books published prior to 2005.

duh

Catherine Johnson said...

Hey Steve -

I think we have to make a distinction between reading & math on the SAT. In my view, high schools should absolutely be preparing their students to do well on SAT reading because SAT reading **is** reading: it is exactly the kind of reading you have to do in college & graduate school.

I never know what to think about SAT math. The questions are 'tricky'; in school, you don't have lots of practice in "SAT math."

I find SAT math fun, and I think schools ought to give kids practice in SAT-type math simply because the test is important to their futures (if they're taking SAT instead of ACT), but I have no idea whether getting good at SAT math per se is the best use of time.

Sian Bielock's book CHOKE makes me think that SAT math may **not** the right test. I'll get that material posted.

That said, the time I've spent with C. prepping for SAT math & trying to raise his score (successfully raising his score a good 50 points, in fact) has been very well spent; he's getting lots and lots of practice AND insight into the math he's been trying to learn for lo these many years, and so am I.

LynnG said...

How do you get copies of the PSAT test?

Catherine Johnson said...

You get them from your friends whose kids have taken the PSAT.

Catherine Johnson said...

I think there are some selling on Amazon, too.

Catherine Johnson said...

Steve H - if you're around - I just sent you an email & I'm not sure I still have the right address. So maybe check your junk mail - ?

SteveH said...

I got it.

I agree with your comment about reading and math, but I will comment when I get a chance.

ChemProf said...

Well, we know that persistence in STEM majors rises with SAT math score. Particularly, there is a sharp rise in persistence for math scores over 650. So the scores are useful for colleges.

SteveH said...

"because SAT reading **is** reading.."

I agree, but since SAT I & II, and ACT define the game that students *have to* play, schools can't turn up their noses and argue over whether that content is the best use of educational time. As I told my son (to his surprise), there are things like the Academic Index that colleges use to do an initial sort of students; two-thirds of the score is based on SAT (math and verbal) and one-third on high school rank.

Whether SAT represents meaningful learning is really a separate issue. My son gets very good class grades, but he has to spend so much extra time preparing properly for the SAT, especially because of the competition. If there was less competition, I wouldn't be sitting here worrying (as much) about missed content. At some point, it's not a matter of deciding whether or not a student can handle a particular college or not, but a competition to decide the best of those who can. Another way to look at it is that the better you do on the SAT/ACT test, the more choices you will have. Schools have to do more than offer SAT prep couses and claim that their regular classes offer a proper foundation.


SAT math is not the right test because it's limited in the material it can cover. That forces the questions to become more tricky. The SAT II test is better, but I don't know how it's used by colleges in their sorting formulas. Does anyone have any examples of formulas? I heard that there is a form of the Academic Index that includes those scores, but I haven't seen the formula.

My son can't get much higher grades in math, but I expect he will need a lot of work to prepare for the Math SAT. When we get to that point, I will report on what his initial SAT math practice test score is, but I don't think I can get him to take the test without any preparation.

Allison said...

Steve, I don't know what you'd call meaningful learning, but the math portion of the general SAT (the SAT I) is a terrific test of mathematical maturity. That is, doing well on the SAT I math portion demonstrates the conceptual understanding a student has, and also is written so that understanding is demonstrated with problems that require procedural fluency.

There's precious little on the SAT I math that is really at the high school geometry or algebra 2 level--a few questions, but really, the bulk of the material is geared at pre algebra or lower. It's just that the more higher math you've succeeded at, the more you (should) have mastery of that less high material.

Typical SAT 1 math questions require you to immediately be fluent with half a dozen concepts or definitions (by fluent, I mean recalling them without even realizing you're recalling them/using them.)

There's no time pressure posed by the SAT I for a student truly fluent in elementary through hs math--there's more than enough time for them. But for a student who is not fluent, time is a big factor.

I gave a talk with some slides that took examples from prior SAT tests that was about what mastery looks like. At least one of those questions was from Catherine here, actually. Here is the link to the talk:
http://www.msmi-mn.org/home/msmi-programs/msmi-talks/MSMI-parent-SATmastery.pdf

SteveH said...

The question is whether schools prepare kids properly to take the SAT or provide a proper foundation. If one talks about what is a foundation versus what is test prep, I would say that schools (especially K-8) do not provide the proper foundation. Of course, it depends on how you define foundation.


For SAT math, perhaps the assumption is that if you can prove yourself at that limited level of content, you won't have any problem at a higher level. That's why I raised the question about how colleges factor in the SAT II score.

In our very competitive college application world, a few difficult (tricky?) questions become magnified in their importance, especially at that limited level of content. My view is that time is an issue unless you prepare properly. You are competing against students who have prepared properly. However, I don't necessarily equate being properly prepared for the few top-level questions of a specific test as mathematical maturity. Do those skills automatically transfer over when the content gets to something like divergence and curl? Preparation skills, yes; content skills, not so much.

The issue I have with SAT math is that the limited content domain is overemphasized. On average, this might work out, but for an individual who want to be viewed properly compared to others at his/her level, it requires a good amount of test prep beyond any foundation that the schools will provide.

What this means for my son who got good grades in algebra II as a freshman is that he has to spend a lot of time preparing for a test on lower level material. Does this mean that he missed something in terms of mathematical maturity? What score on SAT math demonstrates mathematical maturity? When my son takes the test, his goal is not to demonstrate mathematical maturity, but to compete with other kids for more college choices. At the top level, colleges are well past mathematical maturity. It's a game based on a winning a limited number of marginal points.

Catherine Johnson said...

Well, we know that persistence in STEM majors rises with SAT math score. Particularly, there is a sharp rise in persistence for math scores over 650. So the scores are useful for colleges.

Interesting.

I'll also be interested to hear your take on the Choke chapter on SAT scores, working memory, etc.

Catherine Johnson said...

In our very competitive college application world, a few difficult (tricky?) questions become magnified in their importance, especially at that limited level of content.

Right.

The questions are 'tricky.'

The research covered in Choke - which I'll get posted - tells us that the smarter the kid, and the more seriously he/she takes math, the MORE likely his or her performance is to suffer on SAT math precisely because the questions are tricky.

I now believe that my original hypothesis about SAT math test prep - that you're trying to automate 'tricky' question solutions to the point that the questions are no longer tricky but routine - is correct.

And I think the reason you need to automate SAT questions isn't that it's a timed test but that BECAUSE it's a timed test, and because it's a high-stakes test, your ability to solve tricky problems is lower when you're taking the test (possibly 10% lower, in fact).

So....does it make sense for high-achieving kids who want to pursue careers in STEM fields to spend a fair amount of time prepping in order to get around the limitations on working memory?

I am guessing that it doesn't make sense.

In our case, we're spending the summer trying to routinize SAT questions for C (and for me...) -- instead of having him re-take precalculus, which is what he needs to do. I couldn't enforce a summer spent prepping for SAT AND re-taking precalculus, so we chose SAT prep.

On the other hand, the SAT math prep we're doing is also a form of remedial review that I'm very glad C. is having a chance to do. His grounding in algebra & especially geometry is gap-filled, and some of those gaps are being plugged now.

But for Steve's son, I'm skeptical about what can be gained by prepping for SAT I.

Catherine Johnson said...

the limited content domain is overemphasized

right -- SAT I tests "problem solving" instead of content knowledge.

AND: problem solving is precisely what suffers in a high-stakes testing situation -- and suffers most in high-IQ students looking to pursue math/science as a career!

Given what I'm reading about math anxiety & IQ, I'm thinking the SAT I math section doesn't make sense.

I'm also thinking critical reading and the grammar section **do** make sense (although I need to think that through a bit more).

Catherine Johnson said...

I don't necessarily equate being properly prepared for the few top-level questions of a specific test as mathematical maturity.

I'm not sure what you mean by mathematical maturity, but I can tell you that I'm now pretty reliably scoring in the 700 to 800 range...I miss very few questions, and the number of questions I miss is going to drop a bit further.

I wouldn't characterize myself as having mathematical maturity --- ALTHOUGH if it makes sense to say that I have "mathematical maturity" in algebra 1, then maybe.

What do you mean by mathematical maturity?

Catherine Johnson said...

What this means for my son who got good grades in algebra II as a freshman is that he has to spend a lot of time preparing for a test on lower level material.

right, exactly

Allison said...

--The question is whether schools prepare kids properly to take the SAT or provide a proper foundation.

Answer: no, they don't do either. Next question?

There is no need for test prep for
those who have mastery. A proper foundation is a necessary but not sufficient condition for mastery.

For those who don't have a proper foundation in fractions, decimals, pre alg, and alg 1, test prep of the SAT I is of limited value--sure, strategies of test taking, practice under timed situations, etc. will help some, but it isn't going to raise a 550 to a 700.

STEM fields in college require more than mastery of fractions, decimals, and alg 1, though, so sure, the SAT is "limited." Again, it's a good test of one's maturity on the foundations: you know why the answer is true, could explain it mathematically, can justify your reasoning.

If you didn't get those foundations mastered, you can't possibly master the pre calc, calc, analysis, linear alg, etc. needed beyond. Of course that mastery transfers to the above--but again, it's necessary but insufficient for learning the content of div, grad, curl.

I don't know why you seem to insist someone who is brighter and knows more than other at his level needs a good amount of test prep at all to be "viewed properly".

If your concern is that it's an arms race, yeah, it is, but the top students aren't wasting their time on test prep, they are wasting their time on other college app-padding ways :), be it getting that medal in the science fair or doing summer research in biochemistry or composer and conductor summer camp.

The ones spending time on test prep are the ones who just weren't able to master the material in the first place, for whatever reason.

Allison said...

--The research covered in Choke - which I'll get posted - tells us that the smarter the kid, and the more seriously he/she takes math, the MORE likely his or her performance is to suffer on SAT math precisely because the questions are tricky...you're trying to automate 'tricky' question solutions to the point that the questions are no longer tricky but routine - is correct.

Yeah, i'd like to see such claims backed up by research because I don't buy it. There's nothing tricky on the SAT I math test. Yes, of course you're trying to "automate". But that's what mastery means: what novices think of as tricky, masters don't. What does smarter here mean? Smarter by what measure? some specific IQ test?

SATVerbalTutor. said...

"The ones spending time on test prep are the ones who just weren't able to master the material in the first place, for whatever reason."

Thanks for saying that Allison. What most people think of as serious test prep isn't really test-prep at all but rather remediation work. If someone's skills are truly that solid, the "tricky" element is irrelevant. They're just answering the questions.

Besides, for a lot of kids, tricky = I don't want to read the question that carefully and figure out *exactly* what I'm being asked to do. I just want to race through and get the right answer without having to try that hard. When first exposed to the level of precision they need to work with at in order to truly ace the SAT, most kids are absolutely flabbergasted.

Allison said...

--When first exposed to the level of precision they need to work with at in order to truly ace the SAT, most kids are absolutely flabbergasted.

Right right right!

This is absolutely true of the math on the SAT, but also of higher-than alg 1 math in general. (and I as the assume true of the verbal SAT, but I'm simply not familiar enough to know myself)

Success in hs algebra and beyond requires knowing and understanding definitions and being able to manipulate them to derive general results. Such definitions must be precise. Recognizing that precision in a problem and being able to express that precision themselves is critical to success in math.

This is why analogies in math and group projects in math grades 5-8 are so devastating--they rob students of the ability to gain experience with precision and definition at the point when it would help them the most, to prepare them for algebra.

The ones who have not internalized that level of precision are utterly outclassed no matter how high their grades are in their math classes.

SteveH said...

If schools do NOT prepare kids properly to take the SAT or provide a proper foundation (I agree), then how are these top students (who apparently don't waste their time preparing for the SAT) created? What other prep is done to make up this gap? Is this extra preparation done without looking at the SAT test? Is there a linear relationship between this extra (non SAT focused) work and an increase in SAT scores, especially above 700?


"I just want to race through and get the right answer without having to try that hard."

This is absolutely NOT what I am talking about.


This is not just about precision and mathematical maturity. It has to do with the arms race and the critical importance of relatively small changes in your SAT score. It doesn't matter what science medals you've won if you are starting out with a 675 on your math SAT. Who is getting a 750 in math without doing extra outside of class? What does this extra work look like?

When I look at some of the SAT questions, I don't see maturity or precision. I see speed and whether you have seen the problem before in some form. This isn't about (re)learning content. It's about seeing the right angles they try so hard to hide away in a geometry problem. It's knowing which relationships are more important that others. It's seeing the shortcut because you can't possibly solve a problem the long way. These are contrived problems that you rarely see in real life, and, you have to do them quickly. Before long, you might as well not mention the science medal because you won't make the cut.

Catherine Johnson said...

i'd like to see such claims backed up by research because I don't buy it

She's doing her research with University of Chicago students!

Catherine Johnson said...

When I look at some of the SAT questions, I don't see maturity or precision. I see speed and whether you have seen the problem before in some form. This isn't about (re)learning content. It's about seeing the right angles they try so hard to hide away in a geometry problem.

That's sure the way I experience the test.

As to whether top students do test prep .... I'm pretty sure they do.

I'll ask PWN.

My perception of the top kids and test prep is that the top kids **do** have test prep; they just need less of it.

As to finding the hidden right angles, that is a huge issue on the test -- and I'm pretty sure it's a problem type that handicaps students with high working memory.

You have to disaggregate the image, which is not what we're built to do.

I found the term for this the other day....& now can't remember it.

I'll find it.

I'm also midway through a post about why the geometry problems are the hardest (at least for a lot of people); I believe it has to do with visual perception.

Catherine Johnson said...

If someone's skills are truly that solid, the "tricky" element is irrelevant.

I'm pretty sure PWN disagrees with this --- he's told me there's always some question that will flummox you -- and he's talking about tutors who routinely score 800 on all 3 sections.

btw, I'm talking about the math section, not the verbal sections.

I don't see either of the verbal sections as having trick questions in the sense that the math section has trick questions.

Catherine Johnson said...

The verbal sections have questions that are "easy to miss" ----

(So ask me to define "easy to miss" versus "tricky"!)

Catherine Johnson said...

More evidence that the top kids are doing test prep: the testimonials on the back of Chung's book.

He's got kids enrolled in MIT thanking him for his help on SAT 1.

Bonnie said...

I have to ask this question again - when did the SAT (and PSAT) get so hard? I took the PSAT without even knowing what it was about, and ended up a National Merit Scholar based on that score. I took the SAT as a senior and had a 720 on the math (780 verbal) with no test prep. My husband made that magic 800 mark, again, no test prep. We both went to really bad high schools that taught significantly less that the schools teach today. This was in the late 70's. Did something happen to the SATs? I need to know because I had just assumed my kids would do fine on it, as I did.

Anonymous said...

I'll second Bonnie.

I took the SAT in 1984 (so PSAT in 1983 I think). No test prep for either (or none that I remember). I didn't get a book of sample problems. I didn't sign up for a test prep class. It didn't even occur to me that this was something to do. I don't think most of my friends did any test prep, either.

I scored a bit less than Bonnie did. My friends scored a bit higher than I did. Going from memory we went off to UCLA (3 of us), UCSB (1), MIT (1), Pomona (1) and Harvey Mudd (1).

I don't know entirely what changed in the last 20 years.

*PART* of what changed is that we have more kids competing for roughly the same number of slots in the top schools.

Another part I think is that the demand for the top-20-ish schools has gone up a lot (I have no data for this), but the slots in those schools haven't increased.

A third bit is that we have more foreign students competing for those slots ... but I think the foreign component is mostly showing up in grad school.

But I don't see how these things together can explain the apparent increase in competition and difficulty we see in the last 20 years. Maybe they do, but it doesn't seem to me that the three items above are enough to explain this.

Maybe the K-12 education has gotten worse? Except that I think that for the top 10% or so the K-12 education is probably better than 20-30 years ago.

I'll echo Bonnie's question: What has changed to explain this?

-Mark Roulo

Catherine Johnson said...

The test changed in 2006.

I think it's much harder than it used to be -- which makes the entire issue of how SAT scores relate to schools confusing because scores on the old test declined steadily between 1963 and 1979.

In 1995, the College Board "recentered" the scores, which means that my verbal score on the old test would be higher today. (My math score stayed the same ...)

But then in 2005 CollegeBoard changed the SAT and, in my opinion, made it significantly harder.

I don't think there's any way I could have handled the SAT writing section as a teen. I certainly couldn't have handled the math section...if I had to guess, my score on the current test would have been no higher than 500 max.

The reading test is very demanding. I can't tell whether it's harder; I just don't remember the old test well enough, although I do know I found the reading test easy when I was 17.

I don't find it easy today although I do get virtually ever question right and I finish with at least 5 minutes to spare.

See: Schoolbook Simplification and Its Relation to the Decline in SAT-Verbal Scores on the decline in SAT scores.

Catherine Johnson said...

I would LOVE to get my hands on a copy of an old test -- old meaning a test from, say, the early 1980s.

Bonnie said...

So I assume that means average scores have gone down?

lgm said...

What has changed is access to the content information necessary to score in the 700s.

I'm spending part of my summer supplying my son with the math content that the district didn't. The English isn't so bad, b/c the public library has literature and he can read. But the math - where does a kid go to make up what his school didn't bother with? And I'm not talking the 'hard' PSAT/SAT questions only...I'm talking entire units that were in the NY state syllabus for the Alg I course, but were either omitted in the actual class or taught as calculator keystrokes to pass the Regents.

Catherine Johnson said...

The test has obviously changed more than once over the years. Looking at "10 real sats," I see references to things that weren't on the test I took.

Anonymous said...

"So I assume that means average scores have gone down?"

I'm guessing that the scores did not go down after the 2006 change. The re-centering in 1995 (to get the mean back up to 1000) caused enough wailing about dumbing down the test that I expect the the College Board folks do not want to go through this again. Since they control the questions, I'd be stunned if they changed the test in such a way as to force the average down non-trivially.

And ... I'm mostly correct.

http://professionals.collegeboard.com/data-reports-research/sat/cb-seniors-2010/tables

has the data.

2005 (R,M): 508, 520
2006 (R,M): 503, 518
2007 (R,M): 502, 515
2008 (R.M): 502, 515

If the test is harder, it isn't showing up in the reading or math average scores.

-Mark Roulo

Allison said...

The recentering as Mark points out was to make 500 correspond to the average score on the test again. The original scoring was defined from a reference set of 10,000 students in 1941; the new numbers corresponded to a reference set in 1990.

Largely, the issue was the changes in the body of students taking the test. More and more students of lower and lower proficiency were taking the test, as more and more high schoolers were being encouraged to go to college or at least apply to college.

So the score recentering made it much easier to get a score about 500, and for a perfect 800, the chances went from being exceptionally small (like 1 in the whole test taking population in a given year) to being more like 3 sigma-- 1/2 of 1% of the test takers.

http://professionals.collegeboard.com/profdownload/pdf/200211_20702.pdf

has more details.

Stylistically, the test has changed recently. You should look at how there are "write your answer out" problems, and how you are supposed to write them out and bubble them in, as opposed to pure multiple choice. Some people might find this harder, others easier. For those of us who were used to the prior types of tests, and for whom all the other types of tests mimicked the SAT, maybe there was some advantage in the consistency. I don't know how this new method compares to say, state standardized tests across the board (MN has its own "write out your answer" state tests--but they are different than how the SAT does it.)

Catherine Johnson said...

Largely, the issue was the changes in the body of students taking the test. More and more students of lower and lower proficiency were taking the test

Not sure I understand you (I'm skimming comments, so may have missed something)...

If you're referring to the 15-year decline in SAT scores, it's not explained by a change in composition of the test-taking population.

Scores declined the most at the top.

From the Charles Murray study:

"The SAT decline was real and large for the students who take the SAT, meaning that the decline was real for America's most capable young people.

The culprit is not the democratization of the SAT population, but rather the 'mediocritization' of the college track in high school--our term for the downward trend of the educational skills of America's academically most promising youngsters toward those of the average student."

source:

What's really behind the SAT-score decline?
Charles Murray & R.J. Herrnstein
Public Interest, 106 (1992:Winter) p. 32

Catherine Johnson said...

It's seeing the shortcut because you can't possibly solve a problem the long way. These are contrived problems that you rarely see in real life, and, you have to do them quickly.

Steve is exactly right -- that's what the SAT is about.

Beilock's research on working memory and choking shows that high-working memory students are precisely the students whose performance deteriorates most under these circumstances.

High working memory students (and adults) don't take shortcuts; low working memory students do.

I've got a bunch of stuff to post, but for now I'll just say that I've reached two conclusions about SAT test prep for academically talented students.

I base this on my own experience of SAT math tests, on the Khan videos, and on Beilock's research:

1. your goal in preparing for SAT math is to turn the test from a test of problem solving to a test of procedural memory (procedural memory includes cognitive skills -- something I'll get to at some point). In other words, you shouldn't experience SAT problems as "problems." SAT problems should become so familiar to you that they are simple "exercises." You shouldn't have to think much at all while you're taking the test.

2. You should practice until you're 10% better than the test because performance under pressure suffers by 10%.

These two conclusions are really just an application of the notion of "overlearning" to SAT math.

Catherine Johnson said...

I've been thinking about the possibility that the SAT reading test favors high working memory kids while the SAT math test handicaps them...

Or, more likely, that the SAT reading test is 'kinder' to high-IQ students than SAT math.

Haven't thought it through.

Catherine Johnson said...

I assume that means average scores have gone down?

Average scores declined for 15 years, and then the College Board "recentered" the scores, turning a critical reading score of 500 into a 580.

SAT Score equivalents

Catherine Johnson said...

My 720 on reading would be a 790 today.

That's the difference.

The reason this is confusing to me is that I had long assumed that 'the schools got worse' (which I believe they did), BUT now that I'm immersed in SAT prep I see that the test also got harder.

My one question about the reading test is whether the vocabulary is less demanding today. If Erica is around, she can fill us in.

I think most of the passages on the SAT are drawn from contemporary sources, and I'm pretty sure contemporary writing uses simpler vocabulary (and simpler sentence structure & shorter paragraphs) than published writing in the first half of the 20th century.

Take all this with a grain of salt!

Catherine Johnson said...

But the math - where does a kid go to make up what his school didn't bother with?

When C was in 6th grade, his math teacher sent home a Xeroxed list of dozens of skills with instructions to "brush up" on the noted skills over the summer.

No advice on how to do that.

Apparently he was supposed to a) create practice materials for himself and b) complete the practice materials, then c) check his answers against d) an answer key that he had constructed himself.

That was the only logic I could discern.

Of course the real message was that if Mom and Dad wanted their child to learn the course content, they would have to reteach it over the summer.

No advice to us on how to do that: which skills come first, which come second, and what books & practice materials to use.

All the books had been collected and returned to the school.

Catherine Johnson said...

What has changed is access to the content information necessary to score in the 700s.


Absolutely.

lgm said...

>>High working memory students (and adults) don't take shortcuts; low working memory students do.


This isn't the explanation though.

What you are seeing as a shortcut is not a shortcut to a kid scoring over 700 math with no prep b/c that kid sees the fast way as the obvious way to solve the problem since he has mastery of the concept and can apply that understanding to novel situations.

Suppose the problem was finding the sum of the first 100 integers. People that have mastery of number theory will do this in a flash compared to those that like to plug and chug and to those that like to recall formulas. And they enjoy it!!

Remember the first chapter in AoPS Intro to Counting, where they summed series? A kid that scores 700+ thinks this stuff is awesome and will give up free time to discuss these techniques...and they won't memorize the formula, they'll grok the pattern and the concept. As a first grader, they were the kids that looked puzzled when the teacher wanted them to skip count by 3s only starting with 0..they were wondering why they couldn't start with 1 or 2.

I think the difference that Steve was inquiring about - who are the kids that score high without massive prep - is that these are the kids that delight in playing with numbers and number bonds and have done so for years. They can also think and reason well.

There are studies of who scores well; I can't access many but the conclusions are that suburban students do better than rural, usually attributed to access to advanced courses (ie honors, not gen ed). In my time, there was no honors math - one didn't take Alg I if one didn't do well in Pre-alg. Alg I was only taught at today's honors level. That makes a big difference in acheivement on the SAT.

SteveH said...

These are contrived problems that you rarely see in real life, and, you have to do them quickly

"...that's what the SAT is about."

It's what the upper end is all about. I see many SAT questions that are perfectly fine. However, I'm talking about the arms race and the difference between getting good grades on AP calc track courses and doing equivalently well on the SAT. Your peers are not just having fun in math with no test prep.


"What you are seeing as a shortcut is not a shortcut to a kid scoring over 700 math with no prep b/c that kid sees the fast way as the obvious way to solve the problem since he has mastery of the concept and can apply that understanding to novel situations."


"No prep" above what, exactly? For many problems, the fast way is NOT the obvious way. Should it be obvious that variables automatically drop out due to a hidden dependence of equations? (Only if you go in with a SAT mind set.) Does finding that shortcut show mathematical maturity? What about the slow process of defining a system of equations and reducing them to find the rank of the matrix?

To me, to do well requires a special kind of test prep that is focused specifically on real SAT questions. At the beginning of the summer, my son spent time deriving equations of series just for fun. This is great, but that's not the most efficient approach to maximizing performance on the SAT. There is no time to derive anything during the test. The solution has to be immediate and procedural.

If the SAT is supposed to be some sort of aptitude test, then I don't buy it. You can look smart with a lot of practice. You can train youself to see most of the shortcuts. But I don't buy the idea that it reflects on one's mathematical maturity. It's a high-stakes game that's focused on content that is too limited.

I'm not anti-SAT at all. I just find that (at the upper end) the test requires my son to hone specific skills that are not the best use of his time, except for competition reasons.

lgm said...

No prep above the classes the student is taking plus, if he is a suburban child, most likely his math club activities.

>>Does finding the shortcut show mathematical maturity?

It shows insight was generated and the student is on his way to mastery.

lgm said...

Duke TIP Article 30 year study

was interesting too. It compares the 7th grade high scorers over the last thirty years.

SteveH said...

"It shows insight was generated and the student is on his way to mastery"

But I'm not talking about averages. I'm talking about individual students and what the game is. It would be fine if the competition wasn't so great, and I don't buy the idea that just working on math in general will give you the best SAT score.

Allison said...

Catherine, you should read the article I pointed to from college board about the recentering. Here's a piece.

From 1941 until 1951-52, the SAT V mean dropped from 501 to 476, and the SAT M mean dropped from 502 to 494, such that the SAT V and SAT M means differed by 18 points in 1951-52. Ten years later, in 1961-62, the SAT V mean had dropped an additional two points to 474, while the SAT M mean increased by one point to 495.
By the late 1950s, concern with the 20th set of scales reached a state that required a series of special studies led by S.S. Wilks (1961). His report, Scaling and Equating College Board Tests, published in 1961, examined growing problems with the SAT scales that that been set in 1941-42. The report acknowledged that the educational arena had changed dramatically
between 1941 and 1961, and that this change led to a major shift in the SAT test-taking population. The testtaking population was no longer mostly restricted to a selective self-selected group of students applying to Ivy League colleges and other prestigious Eastern colleges.
World War II had changed the role of women. The GI Bill had expanded educational opportunity. College
Board member colleges had gone from 44 to 350 between 1941 and 1961, nearly a nine-fold increase. Many of these new colleges came from the South and the West. Scholarship programs had also expanded opportunity. These increases in educational opportunity resulted in changed populations and presented scaling problems for the 1941-42 scales.

Allison said...

another excerpt:

The real decline in SAT scores did not start until after the Wilks report was issued. Shortly after the Wilks report, from about 1963 until 1980, both SAT V and SAT M means dropped noticeably from about 475 for SAT V to around 425, and from about 500 to 470 for SAT M. Now the difference in SAT V and SAT M mean scores was close to 45 points. By 1990, the SAT M mean had increased to near 475, while the SAT V mean remained around 425, a 50-point difference. Except for the famous score decline of the mid-1960s to late-1970s, SAT mean scores have been remarkably stable.

palisadesk said...

So the score recentering made it much easier to get a score about 500, and for a perfect 800, the chances went from being exceptionally small (like 1 in the whole test taking population in a given year) to being more like 3 sigma-- 1/2 of 1% of the test takers.

This can't possibly be quite right....when I took the SAT (well before recentering), 3 or 4 people in my class got 800's -- my best friend got 800's on BOTH the Verbal and the Mathematical sections. Somebody else got a 799 on the Verbal (how do you lose 1 point?)

So 800's could not have been so rare as to be one per yearly-tested population. There were bright, studious kids in my class but not geniuses, and hardly the best in the USA.

When did they remove the "Analogies" section from the SAT -- I know it was recently? I'm told that the SAT now is much more an achievement test than (as before) highly correlated with IQ.

I do know none of my peers at the time did "SAT prep" even though all were gunning for the competetive colleges and knew that the scores were very important. It just wasn't part of the culture at the time.

palisadesk said...

So the score recentering made it much easier to get a score about 500, and for a perfect 800, the chances went from being exceptionally small (like 1 in the whole test taking population in a given year) to being more like 3 sigma-- 1/2 of 1% of the test takers.

This can't possibly be quite right....when I took the SAT (well before recentering), 3 or 4 people in my class got 800's -- my best friend got 800's on BOTH the Verbal and the Mathematical sections. Somebody else got a 799 on the Verbal (how do you lose 1 point?)

So 800's could not have been so rare as to be one per yearly-tested population. There were bright, studious kids in my class but not geniuses, and hardly the best in the USA.

When did they remove the "Analogies" section from the SAT -- I know it was recently? I'm told that the SAT now is much more an achievement test than (as before) highly correlated with IQ.

I do know none of my peers at the time did "SAT prep" even though all were gunning for the competetive colleges and knew that the scores were very important. It just wasn't part of the culture at the time.

lgm said...

>>But I'm not talking about averages. I'm talking about individual students and what the game is. It would be fine if the competition wasn't so great, and I don't buy the idea that just working on math in general will give you the best SAT score.

I'm not talking averages either. It is up to the individual to master each and every concept tested. He can do that when he takes the class or he can do that as test prep. Mastering those concepts will improve the student's score if the test has the 'harder' questions on that topic, but if the test only has the 'easy' ones that can be plugged in to the calculator, then the studying didn't pay off for this particular test. Whether that results in the 800 depends on the particular exam and the test taking skills of the individual.

Bonnie said...

First of all, I totally disagree that schools have declined since the 70's (which seems to be one possible explanation). I went to school in the 70's, and honestly think American schools were at their nadir then. My 5th grade son is writing multi-page research papers now, something we didn't touch until 11th grade in my day. I was never asked to cite anything until my senior year in high school. We didn't have all of the math opportunities. My high school was the only high school in my state to offer calculus, for example. My trig course was taught by the gymnastics coach, who was spectacularly ignorant of math. My husband went to high school in the northeast in the same era, and tells me he was never asked to write an essay until college. His school did not offer calculus.

It is likely that our schools have gotten worse since the 30's and 40's. But schools in the 70's were dreadful. The fact that we did well on the SATs despite our schools tells me that something about the SATs themselves have changed

Grace said...

When first exposed to the level of precision they need to work with at in order to truly ace the SAT, most kids are absolutely flabbergasted.

But it's not only to "ace", but to simply do well in the SAT. I see this in our schools. Students can earn high grades without necessarily mastering the material. Group projects, extra credit, easy grading, etc.

In the areas of reading comprehension and writing, for example, I've seen students turn in sloppy writing with ambiguous analysis get full credit. This is awfully poor preparation for SAT-type questions.

One junior year English honors class spent about the first six weeks of the school year reviewing basic grammar. I view that as an indication they were trying to catch up the students for the SAT, having downplayed grammar instruction in years K-10.

Grace said...

My 5th grade son is writing multi-page research papers now...

It's these types of assignments I've seen receive high marks, but basics like sentence structure and grammar went uncorrected. It sounds as if your son has a good teacher and/or has learned on his own, but I did not see that type of instruction as the norm at our school.

Catherine Johnson said...

I totally disagree that schools have declined since the 70's (which seems to be one possible explanation

Let's see....when did the textbook decline happen....

OK, here it is:

The 50+ point decline in mean SAT-verbal scores between 1963 and 1979 is widely attributed to changes in the composition of the test-takers. Several inconsistencies in
that explanation are identified. That explanation also ignores the pervasive decline In
the difficulty of schoolbooks found by analyzing the texts of 800 elementary middle,
and high school books published between 1919-1991.


Anecdotally, Ed's family says that the decline in their public schools happened around 1970 or thereabouts. It was quite abrupt in their experience. They sent their first two kids to great public schools; they sent their third child to the same public school, only now it was bad.

Catherine Johnson said...

Here's Marilyn Jager Adams on the contributing factors of changed population taking the test vs. decline at the top.

"A second prominent hypothesis was that the decline was due to changes in the demographics of the test takers. Analyses shows this hypothesis to be largely correct, but only for a brief while.

[snip]

Over the 1970s, however, though the test-taking population stabilized, the scores did not. Instead, the decline continued, even steeper than before, while the extent to which it could be ascribed to demographic shifts shrank to 30 percent at most. Furthermore, the scores that dropped most were those of the strongest students, the students in the top 10 percent of their class; the scores of students toward the bottom of the distribution held steady or even increased."

lgm said...

Bonnie, you teach now right? What percent of students are offered real, honors level 70's space age style pre-algebra and algebra in your district? How many get the gen ed watered down, regent's version?

Here nobody gets the honors level, everyone gets gen ed. That's a decline since I went to school in the 70s, since back then the top half were offered the real thing. Here the top 15% get offered gen ed algebra in 8th instead of 9th. Big whoop, it won't get them to a STEM career. They need the real course, not the how to use the calculator to beat the test course.

Catherine Johnson said...

What you are seeing as a shortcut is not a shortcut to a kid scoring over 700 math with no prep b/c that kid sees the fast way as the obvious way to solve the problem since he has mastery of the concept

Gotta get all this stuff posted.

Bielock (I think it was Bielock) finds a significant sex difference in use of shortcuts --- !

Girls don't use them nearly as much as boys (these are girls & boys matched for math achievement & possibly for IQ as well).

The explanation in one study was that boys were learning math shortcuts at home.

There are a bunch of studies showing that girls with same IQ & math achievement do worse on SAT math than boys.

Catherine Johnson said...

My 5th grade son is writing multi-page research papers now,

I think that's bad teaching (& bad curriculum).

Ten-year olds aren't ready to write multi-page research papers.

Most college students aren't ready to write multi-page research papers.

For me, a good school is one that gives kids work geared to their level.

I remember the 8th grade 10-page research paper all the kids had to do here. It was a nightmare involving HUGE quantities of parent work and 'help.'

Also a waste of time. Most of the kids writing the paper couldn't write a proper paragraph & no one that I knew really understood what a complete sentence was.

SteveH said...

"It is up to the individual to master each and every concept tested."

Obviously, and with the math SAT, that requires specific preparation, not general.

Anonymous said...

"There are a bunch of studies showing that girls with same IQ & math achievement do worse on SAT math than boys."

One possibility is that the boys are actually better (in the sense of getting the correct answer) at math and the girls are better at school (in the sense of pleasing the teacher). The SAT doesn't care about pleasing the teacher.

How is "math achievement" defined?

-Mark Roulo

Bonnie said...

"Here the top 15% get offered gen ed algebra in 8th instead of 9th. Big whoop, it won't get them to a STEM career."
In my junior high in the 70's, a far smaller proportion of students took 8th grade algebra. In fact, I doubt 15% even took 9th grade algebra! It wasn't a graduation requirement, and the big state school a mile away didn't care if students had taken it or not.

I can remember sitting in "rap classes" that counted as English in 10th grade (in a wealthy suburb of Seattle), a 12th grade physics class in which the teacher mainly showed us vacation slides and talked about aliens (this was in a middle class school in KY), and a first grade class in which we did dittos all day while the teacher read magazines up front (Texas). Things were really bad back then. There is no way I would see any of this crap in my kids district. Back then, the problem wasn't whether the schools were teaching "mastery" - it was whether they were teaching anything at all.

Bonnie said...

"Most college students aren't ready to write multi-page research papers."
I hope you don't really believe this because in college, students need to write multi-page research papers. That is what college is about. We expect that they would have written research papers BEFORE they get to college.

lgm said...

>>There is no way I would see any of this crap in my kids district. Back then, the problem wasn't whether the schools were teaching "mastery" - it was whether they were teaching anything at all.

Yet, the garbage is in mine and I'm only two counties away from you.

My experience in the 70s is what your kids have now; only I was in a rural country high school with well educated teachers. You are in one of the richest districts in the nation, right? I recognize a lot of what Frank McCourt and Jay Matthews describe in their books as happening in my district...unprepared teachers & disengaged students who are just passing the time.

Bonnie said...

The fact that my current district pays its teachers well may explain some of the difference - the old "you get what you pay for" phenomenon - but it doesn't explain the awfulness of even my Washington State high school. That was a district very comparable to the one I live in now. The high school in question is now considered to be one of the elite high schools, like Scarsdale.

SATVerbalTutor. said...

Catherine,

To respond to your question about simpler vocabulary and sentence structre in passage sources, I actually have to say that I don't really think the SAT has been dumbed down much if at all in that regard. Words like "recondite" and "sedulous" and "chary" are not words that most high school juniors have ever encountered. Since many of them haven't had a vocabulary test in years - if ever - vocab questions often pose a significant challenge for them. Granted I only took the SAT in the late '90s, but I don't recall the vocabulary being significantly different at that point. I have the impression that the overall level has stayed pretty stable for quite a while.

Re: the level of writing in the passages. I think it's necessary to define the kind of "hard" we're talking about here within the specific context of the SAT. Yes, much analytical/critical writing from the nineteenth and early twentieth centuries was considerably more florid than the kind of analytical writing that most people do today, but I don't think that's really the point of the SAT. It's a reasoning test, not a literature test -- the goal isn't really to see whether kids can cut through flowery language (although occasionally they are asked to) but rather to see whether they understand how an argument hangs together and what sorts of valid inferences can be made on it. That said, many of the texts are not easy. They may not be "old-fashioned hard" but they do often contain serious academic writing with the kinds of long, complex clauses and sophisticated vocabulary that a lot of high school students aren't at all accustomed to grappling with. Yes, the College Board can go a bit overboard with the whole multicultural thing (Debbie Stier and I were discussing how it almost feels like a parody of itself at times), but on the whole, for the *average* 16 year-old, the Verbal portion poses a significant challenge.

Catherine Johnson said...

I hope you don't really believe this because in college, students need to write multi-page research papers.

It's not a question of belief!

Ed, who teaches at NYU and has extremely capable & well-educated undergraduate students, says that very few entering freshmen can write a paper that makes an argument. In fact, many graduate students have some difficulty with this.

Making an argument is a slightly different assignment from writing a research paper....but still. A good research paper makes an argument, marshalls the evidence to support it, and deals with the evidence that does not support it. These are extremely advanced skills, and they cannot be managed by 10 year olds.

Catherine Johnson said...

the old "you get what you pay for" phenomenon

My own direct personal experience leads me to believe that the relation between more pay and quality is an inverted-U.

We are paying out of pocket to send our son to a school where teachers are paid much less, don't have pensions, and teach many more students.

The teaching is better.

At some point in public education more money starts buying you more problems.

Catherine Johnson said...

How is "math achievement" defined?

I'll get it all posted.

Achievement was defined by scores & grades.

There are all kinds of ways people can get psyched out.

I read a fabulous study last night showing that if you have the kids taking AP tests fill out all their personal data, which includes marking a box for gender, AFTER they finish the test, girls scores on the AP calculus exam go up significantly.

That's "stereotype threat," and I directly experienced it when I went on a game show years ago where I had to compete against two men, one white and one black. I had HORRIFIC stereotype threat, horrific. I had an Ivy League degree and a Ph.D. AND I had severe stage fright, and I found myself thinking, "I can't beat a man" --- meaning it would not be possible for me to do better than a man.

The only reason I won -- well, actually I would like to think there are two reasons I won.

The reason I like to believe is that it made me so furious to find my own mind thinking I couldn't do better than ANY man that I had to win just BECAUSE.

The other reason I won was that the black man, who was a better player than I, was suffering severe stereotype threat --- so severe that I believe the audience could see it.

I don't know what happened with the white man who I beat except that I'm sure he was having stage fright, too, so that probably evened out the deficits between the two of us because he wasn't a good player. (Wasn't a good player meaning didn't have as much knowledge of trivia as the black guy and I both did.)

Catherine Johnson said...

In this thread, we're talking about two different things (well, I am - and I'm not making the distinction clear): math knowledge/ability AND performance under pressure.

Anonymous said...

Fifth graders would be able to write "research" papers that are an exercise in gathering and organizing information. I did this back in the '50's in sixth grade, and it was not hard. The teacher showed us how to do an outline (not hard, because each kid was writing about a Latin American country, so the outlines all had the same features), showed us how to gather information at the public library, and set us loose. In my opinion, this was the perfect first step in writing papers, because it forced us to master the information gathering and organizing, long before we were required to form a thesis or frame (and sustain) an argument, which indeed are much harder. BUT, much of the writing that most people have to do later in life has far more to do with finding/presenting information than it does with defending an original thesis.

Bonnie said...

I agree with Anonymous's comment. The point of having 5th graders write research papers is not that they are going to master the construction of an argument. It is that they will *start* the process of learning to gather information and tie it together. We wrote those kinds of papers in the 5th grade when I lived in Germany.

Catherine Johnson said...

BUT, much of the writing that most people have to do later in life has far more to do with finding/presenting information than it does with defending an original thesis.

Yes -- and thanks for writing this!

I was struggling to put this idea into words.

That said, I'm not **absolutely** sure that there's quite as large a distinction between defending a thesis and writing informational memos in real life ----

I assume it's rarely the case that you need to come up with an original thesis, but there's often an implicit argument in informational writing.

Something needs to guide which research is included and which is not.

The big research paper kids in my district had to write in 8th grade had to include a thesis statement, and the assignment was beyond the capabilities of any student I knew, including our son, who is a good writer & thinker & was at that time, too.

palisadesk said...

The point of having 5th graders write research papers is not that they are going to master the construction of an argument. It is that they will *start* the process of learning to gather information and tie it together.

I think the "writing a research paper" skills need to be taught incrementally, starting early. I was very impressed with what a second grade teacher at my (low-SES) school did this year. As part of the science curriculum,which focused on animal life cycles, he included a short research report that each student had to complete. However, instead of telling the kids to "do" a project, he led the entire process, directly teaching the skills (which also included literacy skills for the grade), and walking them through the steps.

Each child selected an animal of interest, but the teacher explicitly taught the whole class the features of a non-fiction text, provided them with an outline of what would be included, a rubric for how it would be graded, went over the criteria (and showed examples) for meeting the expectations. The students were also given guidance and supervision over sources:they had to include books, internet material and (I think) written notes from video sources. The students were given papers the teacher made up that corresponded with the different sections required: Table of Contents, Introduction, sections on the animal -- habitat, diet, predators, etc. ---, glossary, list of sources. They were also to include several other features, such as pictures with captions, maps, etc. They were not allowed to print stuff from the internet, though they could use a pre-printed map and color it appropriately.

The teacher gave a lot of guidance on topics like re-phrasing (paraphrasing is a little beyond second graders without help), determining which facts were most important, how to organize them etc. All work was done in class, and of course the finished products varied considerably in proficiency and expertise. However I bet every student came away with some idea of how to research and organize a non-fiction report (they learned a good bit about their topic, too). If students did this, with graduated levels of difficulty in every grade, they WOULD know how to write a proper report by the time they graduate from high school.

Bonnie said...

To palisadesk - that is pretty much how our school approaches paper writing in the elementary school grades. Each time my kids have had to write anything beyond a paragraph, they are given a detailed format, with blank paper that is organized into sections corresponding to the format. Even the rough draft paper is organized this way. This isn't just one teacher, either - they have done this in every grade since 3rd. I wonder if this is common? The teacher has to do a lot of feedback with these assignments, so it might be hard to do with more than about 25 students.

FedUpMom said...

Bonnie said:

***
I agree with Anonymous's comment. The point of having 5th graders write research papers is not that they are going to master the construction of an argument. It is that they will *start* the process of learning to gather information and tie it together. We wrote those kinds of papers in the 5th grade when I lived in Germany.
***

Bonnie, by your own account, when you went to school in Germany, school ended at lunchtime, and then you went home where your handy stay-at-home Mom walked you through the homework in the afternoon. Right?

Contrast that to the situation faced by a lot of American families today, with two working parents. Often the child doesn't even get home from school until 6:00, and then the entire family is burdened with big, amorphous projects (like research papers) that the child really hasn't been prepared for by the school. It's evening, everybody's exhausted, and the 10-year-old suddenly remembers she's got a research paper due the next day. Not a great learning experience ...

Bonnie said...

So they can do more of the research paper work at school. Personally, I am in favor of making school days longer - kids should go until at least 5. That would give ample time for library sessions at school (that is, unless budget cutting schools axe their libraries because "everyone has Internet at home").

In fact, in our school, a lot of the process happens at school, because the teachers want the kids to follow the "plan". We parents mainly get asked to find resources (usually we are told to search the Internet, but I always obstinately send in library books instead). It is nothing like the infernal art projects, where we parents ARE asked to do a lot of the project. As a busy working parent myself, I can deal with the papers, but I wish they would dump the dioramas, requests for homemade costumes, and last minute requests for odd supplies.

lgm said...

>>I am in favor of making school days longer - kids should go until at least 5.

Not me. I want efficiency and less wasting of the student's time. Place by instructional need instead of putting such huge ranges of acheivement in one 30-35 kid classroom and expecting a single teacher plus multiple push-in sped teachers to 'differentiate' five grade levels worth of content and skills. We've done it long enough to show it doesn't work. Eject disruptors and get their needs taken care of in alternative school. School would then be a place of learning academics, not a place of waiting. After school can then still be used for exercise and making Vitamin D.

I do favor an 8 hour workday, year round for staff. This would allow staff to have enough preparation and business time, while still time to teach and have lunch.

Catherine Johnson said...

Ditto everything lgm just said EXCEPT that I'd add some kind of school-wide 'positive behavior support' to radically improve classroom (and hallway/recess/lunchroom) management -- THEN define disruptive kids as the kids who aren't manageable under a good system (which would include my two autistic sons, btw).

Also, I wouldn't define "alternative school" as "separate building" (and I realize lgm may not be doing so, either).

Strongly support an 8-hour day (or some such) so teachers have built-in time for 'professional learning communities' (hate that term).

An 8-hour school day would have been terrible for us. I had to reteach all math in 7th & 8th grades; I had to do all the math HW assignments in grade 8, correct my son's HW, and have him re-do problems he'd missed. I also had to teach spelling; my husband had to work on writing.

We needed the after-school time to teach.

Catherine Johnson said...

We parents mainly get asked to find resources

I don't think that's a good idea.

You open up a major gap between educational haves & have-nots, not to mention between two-working-parent families & families with a stay-at-home mom.

I also don't believe in defining research as Google. (I say that in spite of the fact that for me research is Google --- but at this point school's aren't teaching Google research as a nonfiction writer practices Google research.)

Bonnie said...

>>Not me. I want efficiency and less wasting of the student's time.

Then you will be back to the problem that parents will have to oversee lots of learning, which seems to be what people were complaining about.

Learning takes time. I strongly believe it should be an 8 hour a day activity for kids over, say, 2nd grade. Whether that time happens largely at home, as in Germany, or in school is an open question. In Germany, we had what you describe as "efficient" learning - lecture only, heavy tracking, no hands on work in school. We got out at noon, with hours of hands on work to be done at home, under the watchful eyes of mom. There is no substitute for time spent "doing", whether in school or at home.

akil said...

For Cathrine Johnson .. I hope the resolution is high enough but this is images of the SAT from 1990

https://www.facebook.com/media/set/?set=a.10150259752523624.343143.14393013623&type=1

akil said...

Darn I wish i had time to read all 85 comments posted today! Please forgive me but I'm cherry scanning comments so i'm clearly going to miss some of the really great points made.

Stereotype threat
http://www.reducingstereotypethreat.org/

Some questions from 1990 SAT (WAY more vocab)
https://www.facebook.com/media/set/?set=a.10150259752523624.343143.14393013623&type=1

I think that the SAT has certainly changed.. way back in the day it had latin! Is it easier is hard to say cause it reflects other changes going on in schools. I think that it requires less vocab than it once did and thats probably better. We also have to consider teh shift in the college going population.. in the past it was upper middle class kids of a fairly uniform background and schooling then as we enter the 60s, 70s, and 80s there is a huge number of minorities, immigrants and low income students taking the test bringing with them a different skill set, education background and exposure. So the percentiles and averages shift. I think this is also true of schools and the point of public school.

Back to the point with regards to the SAT.. I've tutored SAT for 20 years starting at princetion review (upper middle class mostly private school kids) in 1990 and today at Bell Curves (mostly public school kids and some private school kids). the greatest difference I've found is exposure and flexibility. These test take you out of your comfort zone and thats the intent of a reasoning test. Can you still understand how to apply skill X if its presented in context Y. Can you still understand a passage if its about genomes when you have never read about genomes? CAn you track the authors logic when asked to read about Chinese authors rather than american classics.

the other interesting point is all of the passages given you can probably google and find the source.. they are from sources that are some what reasonable for HS kids or entering freshmen to be able to grasp.. its not esoteric college professional journals but its novels and the like, its just older. One of my kids this past spring read a passage in one of the SATs and she knew what book it was from (it was a book i did not know) and had read it.

I think the SAT isn't inherently unfair or flawed on a theoretical level however its not reflective of how kids are taught or tested and if the students is flexible enough to adapt to the style of the test they are in trouble.

Catherine Johnson said...

In Germany, we had what you describe as "efficient" learning - lecture only, heavy tracking, no hands on work in school. We got out at noon, with hours of hands on work to be done at home, under the watchful eyes of mom. There is no substitute for time spent "doing", whether in school or at home.

Definitely true -- and this is one of the reasons I'm so committed to precision teaching. Precision teaching is all about efficiency: when you teach to a criterion of speed and accuracy, learning is faster.

Catherine Johnson said...

My working theory is that precision teaching could 'square the circle": precision teaching could allow Americans to catch Asians without adopting 3-hour a day practice regimens (at least in academic subjects - not sure about music or sports---)

Catherine Johnson said...

Akil - thank you for that link!

Catherine Johnson said...

Can you still understand how to apply skill X if its presented in context Y.

This is a major challenge for all humans, something I didn't realize when I was co-writing Animals in Translation.

I gather the 'transference of knowledge' issue sparked some of the earliest research in learning and education (but will have to look that up).

I'm going to be reading a review article on learning & the basal ganglia, which appears to argue that 'basal ganglia' learning, which is what I believe I'm using doing SAT practice, is not particularly flexible or generalizable while temporal lobe learning is more so....

palisadesk said...

Definitely true -- and this is one of the reasons I'm so committed to precision teaching.

Catherine, you had been planning to go to Morningside this summer -- did you have a chance to do that?

ChemProf said...

I'm coming back late to this thread (sorry, but the two year old and two month old are conspiring to keep me off the computer!). The 2006 changes to the math SAT make more sense if you remember that the College Board's clientele is really colleges rather than students. My understanding is that colleges wanted the following (this is largely scuttlebut, so take it with a grain of salt, but it fits):

1. top schools wanted a way to distinguish the stellar from the merely spectacular. With the 1995 re-centering, there were too many students with 750+ SATs, given the huge number of students applying.

2. second tier schools didn't want the average to move much, as they didn't want to see their 25%/75% numbers (which matter for US News rankings) move.

3. Neither set of colleges wanted the changes to disadvantage minority students. These students are rarely on the 8th grade algebra track, so would not be ready for algebra II questions on the PSAT. Remember, students take the PSAT in October of their junior year, so to be ready for algebra II questions, they'd have to be in algebra II in 10th grade.

So how do you do all of this? You add "tricky" questions that cover algebra and geometry. The average students won't be hurt much, as they have more of a chance of getting a tricky question right than a question on math they haven't learned. The top students are hurt more, which serves the top schools' goal of spreading those students out more.

Plus, as a bonus, much of the "trickiness" means you have to read the question very carefully. While I don't know of any studies, I bet this benefits girls (who are often better at details at this age), and leads to a few more high scoring girls at the expense of a few high scoring boys. This helps mitigate a complaint about the SAT, that it favors boys too much.

So the changes may mean that students have to study specifically for the SAT more than we did a generation ago, but the changes do serve the College Board's clients well!

Catherine Johnson said...

Catherine, you had been planning to go to Morningside this summer -- did you have a chance to do that?

Nope!

I ended up postponing because of SAT prep ---- they let me sign up for next summer ----

Catherine Johnson said...

two month old

Congratulations!

I hope it's OK with Allison if I tell everyone she has a new baby, too! (Just a couple of weeks old, I think!)

Catherine Johnson said...

With the 1995 re-centering, there were too many students with 750+ SATs, given the huge number of students applying.

That makes a lot of sense.

Catherine Johnson said...

So how do you do all of this? You add "tricky" questions that cover algebra and geometry. The average students won't be hurt much, as they have more of a chance of getting a tricky question right than a question on math they haven't learned. The top students are hurt more, which serves the top schools' goal of spreading those students out more.


wow!

That is very interesting.

I have no idea whether it's true, but my experience of the test -- and I have now taken a lot of SAT math sections -- corresponds exactly with this description. Up to around the 650 level (or maybe a little higher 0 ?), the test is One Thing; after that point it is A Different Thing. The test really does feel slightly....bifurcated.

Crimson Wife said...

An interesting data point is that my mom and I scored the exact same on the SAT-V (both prior to re-centering) but when we recently took a vocabulary test her result predicted a total vocabulary size of 37,900 to my 32,700. And while my estimated total was slightly over the average for those in my age bracket with my SAT score, hers was slightly under the average for those her age with the same SAT score.

Glen said...

Congratulations, Allison!

lgm said...

>>The fact that my current district pays its teachers well may explain some of the difference - the old "you get what you pay for" phenomenon

I doubt that. This district compensates its teachers and staff well. Many live in higher cost of living areas for the better schools which do have honors/AP classes and extracurriculars such as middle school sports for their own children.

It seems to me an attitude problem..I've heard the 'why should I cast my pearls before swine' comments enough to realize that culture and management of the culture is the issue.

Anonymous said...

It will be interesting to see how the vocabulary size curves look once they have enough "younger than 15" data.

I had my 10 year old do this (I checked the boxes and he told me what he thought the words meant), so they now have at least one data point :-)

It looks like I score below the average for my age and SAT score ... but I also note that I've read a lot more technical/niche/non-fiction and much less fiction over the last 10-15 years.

I wonder how this reflects in the scoring (e.g. I didn't see any astronomy terms or technical engineering terms). I wonder if techy people as opposed to humanities people score lower.

What I *REALLY* wonder is if there is any causal directionality (does a higher vocabulary boost your SAT verbal score, or do people who were going to get high SAT verbal scores read more and pick up more vocabulary or what...).

-Mark Roulo

palisadesk said...

Check out this fascinating article on the SAT by Malcolm Gladwell:
Examined Life:
What Stanley H. Kaplan taught us about the S.A.T


I was looking for more info on the various changes to the SAT over the years, and stumbled across that article en passant.

Wikipedia actually has a readable precis of the various incarnations of the SAT:Wikipedia entry