School's getting out, but the dreaded Scholastic Assessment Test, better known as the SAT, looms just a summer away for next year's high school seniors.
Given this, many might be inclined to agree with the president of the University of California, who announced several months ago that he would like to abolish the test as a requirement for admission to the school. (He would retain the SAT II, which measures achievement within particular disciplines.)
[snip]
How predictive of success in college are SAT scores? More precisely, what is the correlation between high school SAT scores and first-year college grade point average?
[snip]
Most studies find that the correlation between SAT scores and first-year college grades is not overwhelming, and that only 10 percent to 20 percent of the variation in first-year GPA is explained by SAT scores.
This association appears weaker than it is, however, for an interesting, but seldom noted statistical reason: Colleges usually accept students from a fairly narrow swath of the SAT spectrum.
The SAT scores of students at elite schools, say, are considerably higher, on average, than those of students at community colleges, yet both sets of students probably have similar college grade distributions at their respective institutions.
If both sets of students were admitted to elite schools or both sets attended community colleges, there would be a considerably stronger correlation between SATs and college grades at these schools.
Those schools that attract students with a wide range of SAT scores generally have higher correlations between the scores and first-year grades.
This is a general phenomenon; the degree of correlation between two variables depends on the range of the variables considered.
[snip]
Soccer Assessment Test
Assume there were an SAT (Soccer Assessment Test) that measured the speed, coordination, strength, and soccer experience of students in a certain city. Assume further that the students roughly divided themselves into five leagues depending on their scores on this SAT, players in the top leagues having higher SAT scores on average than those in the lower leagues.
One wouldn't expect that a measure of success in the sport, say number of goals scored, to vary much among the leagues. There would be good scorers and bad scorers in every league and, just as grade point distributions are similar in most colleges, the distribution of goals scored would probably be similar in the five leagues.
In each league the better scorers would probably have only slightly higher SATs on average. In other words, there wouldn't be a high correlation between SAT scores and success in soccer within any league.
There would, however, be a much higher correlation between SAT scores and soccer success were the students randomly assigned to the teams in the five leagues.
[snip]
The analogy between soccer and scholastics is not perfect, of course, but the point remains. Like the soccer "SAT," the scholastic SAT provides incomplete, but useful information to students and colleges. A rough measure of intellectual preparedness, the SAT shouldn't be made into a fetish, but neither should it be ignored.
Without it, colleges would undoubtedly place more emphasis on high school grades and extracurricular activities, measures that also have serious shortcomings grade inflation and meaningless resume-puffing being the main ones.
The SAT is a flawed predictor, but it is also relatively objective and, among other virtues, sometimes provides a way for the bright, yet socially inept student to be recognized.
I was happy to read this.
I always knew the "SAT scores don't predict college success" meme was wrong, but I couldn't work out why.
For what it's worth, Ed says GREs absolutely predict success, and do so in a fashion that's entirely commonsensical.
Applicants with verbal scores in the 700s are a lock. (for newbies: I'm talking about a graduate program in French studies) Professors know these candidates are intellectually equipped to succeed in a masters or doctoral program. The GRE doesn't tell an admissions committee anything about a candidate's emotional stability or potential for suffering Ph.D.-killing life crises, but admissions committees don't expect it to.
Applicants with scores in the 500s likely will not be able to handle the program.
Applicants in the 600s are a mystery. Some will wash out; some will be in the middle along with their scores; and some will be flat-out brilliant, best in show. These are the folks who come bearing surprises.
I like that.
another reason to teach to mastery
why SATs predict college success
more stats from 2007 SAT
Years ago I read a study I wish I could still locate. However I remember basics. This was a study of all the students taking freshman chemistry at some large school so there were lots of students and a reasonable range of SAT courses. (IIRC they were looking for the effects of study sessions, etc. on the final course grade.)
ReplyDeleteWhat they discovered was that while many students got poorer grades than would be predicted from their SAT scores, almost no students did better.
(I have a vague feeling I've mentioned this before. Sorry if this is a duplicate.)
Is this the new test? Because the SAT I took didn't have a verbal section -- only a Critical Reading, Math and Writing Section.
ReplyDeleteAlas, I got only 690 for writing, 10 points short of the 700 thresthold you speak of. Does a 740 in Critical Reading compensate?
I also want to smack the marker who gave me a 5 for my essay over the head, because the other one gave me a 6. At least if they gave you feedback or some sort of marking comment (they scan in your essay) it would be better.
Personally I think the AP Language and Composition exam is a far better measure of verbal ability, since you are graded on three different essays as well as answer questions that deal with rhetorical intent.
Also, I think the issue is viewing the SATs as a "if you got this score, you're absolutely capable / incapable of making it in a certain college. So if you didn't meet the threshold in the SATs in a certain area, but show yourself to be an outstanding writer by other qualifications for example (e.g. the International Competitions given by the University of New South Wales, which I've always admired), then those scores shouldn't be a damning factor.
ReplyDelete"Without it, colleges would undoubtedly place more emphasis on high school grades and extracurricular activities, measures that also have serious shortcomings grade inflation and meaningless resume-puffing being the main ones."
But wait, isn't the main emphasis on the grades you get in class anyway?
I think there's also an ironic style problem with that section of the article, but I'll overlook it as a typo. ;-)
Susan - no, I hadn't seen that comment
ReplyDeleteinteresting
I'd say that jibes with what Ed said, though I hadn't thought of it quite that way. He obviously sees a "verbal" (meaning the reading section) score in the 500s as very strongly predictive.
Hi lrg!
ReplyDelete740 puts you in the Sure Thing category.
Ed, looking at your scores, would simply assume, without further question, that you are capable of doing excellent work.
I'm willing to bet there's not an admissions committee in the country that doesn't.
With super-competitive schools no SAT score guarantees you a slot, obviously. But the committee knows you can do the work.
The writing test is fairly irrelevant (according to what I'm hearing and reading thus far, but take this with a grain of salt).
ReplyDeleteFirst of all, writing tests are fairly difficult to score for obvious reasons.
Second, it's conceivable that the writing test adds a bit of "extra" predictive power to the total score (the College Board has a study out saying so), but I would be stunned if a writing score can add much of anything to a 740 reading score in terms of predicting ability.
In fact, I'm going to say it can't add anything. Writing, in a fundamental way, is reading. That is, it's built on reading; it's an "offshoot" of reading....I can't put it better than that at the moment, unfortunately.
A person with a 740 on reading is a person capable of scoring a 740 on writing, regardless of the score he actually got.
Another strange thing about the writing test: girls score higher on writing tests in general.
No one knows why.
As soon as the writing section was introduced girls scores went up. (I think boys scores went down - don't recall...)
ReplyDeleteYour writing score is great.
ReplyDeleteEspecially considering the average is whatever it is...
Because I'm kind of a snotty person, I can't resist the opportunity to mention that education majors consistently do poorly on the GREs. Unfortunately, students majoring in special education seem to do particularly badly.
ReplyDeleteCatherine, it seems to me that you are right about reading and writing (I'm just kind of thinking out loud here) but not entirely.
ReplyDeleteI read voraciously from a young age. I credit that as the reason for my large vocabulary and ability to write as well as I do. I absolutely believe that nearly everything I know about writing, I learned by reading.
However, I do not believe I could write professionally without remedial education.
I had little formal writing instruction. I've never diagrammed a sentence. I don't know a lot of grammatical terms. The terms "gerund" and "participle" come to mind, and I really can't tell you what they mean. I can read something that I am _certain_ is grammatically incorrect, and I can provide a correction, but I often cannot explain the error, or explain why my correction is better.
I recently began trying to teach myself Latin from Wheelock's, and it's clear to me that there are a lot of gaps in my knowledge of writing.
When I took 2 semesters of Composition in college, I really struggled to put together coherent essays.
So, I guess what I'm saying is that I think reading will provide writing skills that are adequate for most people, but maybe not everyone.
http://www.time.com/time/nation/printout/0,8816,1521184,00.html
ReplyDeletehttp://www.time.com/time/magazine/article/0,9171,1005973,00.html
LRG - if you're still around, see if you can access these two links.
They have some interesting data on the writing test.
Susan-I find it interesting, too, that Ed apparently has never had a candidate with verbal (reading) scores in the 700s who was not very, very capable intellectually.
ReplyDeleteI believe it.
I took a practice SAT test last fall. It was very high level, IMO.
I've mentioned several times that Ed read one of the sections, an essay on the history of American Indians, and answered the questions.
He said the same thing. The SAT consists of high-level reading passages that require the reader to draw correct inferences in order to understand the passage.
I read voraciously from a young age. I credit that as the reason for my large vocabulary and ability to write as well as I do. I absolutely believe that nearly everything I know about writing, I learned by reading.
ReplyDeleteSame thing here.
I was a voracious reader.
That's a category unto itself.
btw, the math test is much more advanced than it was when I took it.
ReplyDeleteIn high school, I got a 620 on math with zero review, zero prep, zero anything. I got a 640 on the GRE math having taken no math at all for 6 years.
On the sample SAT math test I took last school year I scored in the range of 540-620 -- and this was after working through all of Saxon Algebra 1 (and I think a bit of Saxon Algebra 2, though unfortunately I didn't write down the date I took the test).
I'm going to retake as soon as I finish Algebra 2 (19 lessons left to go).
ReplyDeleteAt Susan S's house I looked at her sample ACT tests & discovered that I can now "guess" the answer to function problems, which I couldn't begin to do last year.
That was interesting.
Obviously there's some kind of intermediary stage in which you've acquired a novice level of knowledge about a concept....to the point where you start being able to eliminate obviously wrong answers, at least.
Actually -- that reminds me.
Doing one of Susan's problems, I had the same experience Chris did on the fraction problems.
I instantly saw, consciously, that two of the choices were wrong because the graph had to open upwards, not downwards.
The two choices left stumped me, although I could have worked them out via trial and error (i.e. by plugging in some values).
Since there was no reason to do that at the moment, I chose the answer that "seemed" right.
That was the answer.
Unfortunately, students majoring in special education seem to do particularly badly.
ReplyDeleteoh, no!
Is that true???
oh gosh
I hate to hear that
I had little formal writing instruction. I've never diagrammed a sentence. I don't know a lot of grammatical terms. The terms "gerund" and "participle" come to mind, and I really can't tell you what they mean. I can read something that I am _certain_ is grammatically incorrect, and I can provide a correction, but I often cannot explain the error, or explain why my correction is better.
ReplyDeleteI don't know any of this stuff, either, but it hasn't stopped me.
It does aggravate me to heck, though, now that I realize how useful a formal knowledge of grammar is to reading archaic texts.
Acquiring a solid knowledge of grammar is one of my goals.
If you wanted to be a professional writer, you could be. Easily, I would imagine (easily meaning it would be "easy" for you to learn how to write professional articles and/or books; breaking in & earning an income & so on are a different issue).
ReplyDeleteYou're right about the challenge, though: STRUCTURE, STRUCTURE, STRUCTURE.
That's the hard part.
Writing a coherent paragraph, theme, chapter, book --- AND FIGURING OUT A THESIS --- these are the killer "skills" of writing.
Naturally constructivist approaches to teaching writing give the kids "free choice" in "choosing" a topic.
Choosing a topic in the real world takes blood, sweat, and tears.
I think I said on another Comment thread that Temple and I spent, I think, two years figuring out the topic for our sequel -- AND WE WROTE AN ENTIRE BOOK PROPOSAL ON THE WRONG TOPIC.
We had to go back & start all over again.
Just this August I finally figured out the book's thesis -- and the book is due in November.
It's taken the better part of a year to go from topic to thesis.
I'm a professional writer; have one book that has been on the bestseller lists with good reviews in the TIMES & in SCIENCE (Science or Nature - have forgotten which) and it takes me 8 months to figure out a thesis.
It is INSANE for middle schools to be telling kids to "choose a topic" and "form a thesis."
K, my college freshman, scored an 800 on the SAT Writing; a score of 790 on the regular portion and a 10 on the Essay. Her ACT Writing score was a 35. What was her path to this?
ReplyDeleteWell, she is a voracious reader. She is also a writer and she has received excellent instruction at various points in time. For example, her 8th grade English teacher (since retired) was an old-fashioned "I'm going to teach grammar whether you or anybody else likes it or not" teacher.
K also loves grammar (does that sound odd?). She has also had five years of Spanish, which not only has given her a second language, but has also helped her learn English grammar at an even deeper level.
She started taking the ACT/SAT in alternate years when she was in 6th or 7th grade through our district's participation in a program sponsored by Northwestern. So, she was already familiar with the format and length of the test, and the endurance factors involved.
Also, she practiced! She used the practice exams to identify areas of grammar in which she was a bit weak, and then she practiced those items as well.
However, and I think this is very important, she had a very strong and very fundammental language foundation to begin with.
I've got to run some errands, but I will also weigh in later with some comments about her Writing journey, as it were.
So, I guess what I'm saying is that I think reading will provide writing skills that are adequate for most people, but maybe not everyone.
ReplyDeleteAbsolutely.
Reading is critical; it's the core.
But figuring out how to create a thesis statement and how to structure a coherent composition IS HELL.
That's the part you don't get from reading -- although I think the text reconstruction folks are correct that you can get it from reading if you take texts apart and put them back together.
Reverse engineering other people's writing is a very good idea, IMO.
I bet a lot of writers do this unconsciously....my hypothesis about expertise is that people who are good at a particular skill may do more incidental learning in that realm than other folks.
I think the same about math.
My guess is that the kids who've managed to learn math in the middle school accelerated math class can pick up a lot of math incidentally.
I'm sure C. probably picks up some math incidentally, but I'd bet the ranch his incidental learning of math doesn't even come close to that of the "math brains" in his class.
I'd say all the Commenters on ktm could be professional writers if they wanted to.
ReplyDeleteThe writing level in the Comments section is extremely high.
One last thing.
ReplyDeleteThe Cambridge Handbook of Expertise has a paper on professional writers.
ALL of them, universally, are obsessive readers & were from childhood on.
That's the hallmark of a writer.
So - Lesley - you're a candidate!
Catherine: Go for linguistics!
ReplyDeleteI find grammar studies restricted to English a tad on the dull side, but universal grammar (SVO order generally results in prepositional sentences, SOV order generally results in postpositional sentences), and morphology studies to be a bit interesting, and since you can analyse any language's grammar.
To me, the existence of an SAT preparation culture just turns me off. I don't know if this is my flaw, but I just don't like to study.
This reminds me of my theory about why a lot of good readers struggle with math.
ReplyDeleteIt seems to me that part of the difficulty is with the need to practice. Those of us who grew up reading a lot felt like language arts skills just came "naturally" while math skills took lots of practice. But of course, we were "practicing" language arts skills constantly, without realizing it.
So we think we are good at language arts because we don't need practice, and bad at math because we do need practice.
I'm sure if I had spent half as much time practicing math as I did reading, I wouldn't have struggled.
Catherine, if you google "average gre education majors" you'll find tons of stuff. It's not a new phenomenon either, Thomas Sowell was complaining about it 15 years ago.
ReplyDeleteMeaning no disrespect to any teacher reading this, but one of the sad things about our education system is that as a whole, too many of our teachers were the worst students. I mention this because I think it goes a long way toward explaining the avoidance of rigorous education methods in favor of endless poster board collages and journals about feelings about math.
Oops, above comment was me, Lesley.
ReplyDeleteLesley, you are completely right and I commend you for saying this.
ReplyDeleteOthers have pointed this out but back when K-12 teaching was one of the few careers open to women, the situation was a lot different. Back then, schools could pay below-average salaries and still get highly qualified people and I'm not sure they ever adjusted to the changes brought about by Women's Lib.
Also, back in my brief high school teaching career in the mid 1960's, there were quite a few competent men who'd gone into teaching because the job market had been poor when they first started.
To me the two things that would make the biggest improvement in our public schools are:
(1) Raising salaries and instituting competency tests such that teachers are above average relative to society at large
(2) Not allowing disruptive students to remain in the classroom
I think there's potentially a third problem. As I've seen discussed on KTM and elsewhere, some highly educated professionals, having already made their nut in the private sector, have attempted teaching as a second career, and been thoroughly disgusted by the ed school classes.
ReplyDeleteIn other words, I don't think raising salaries is enough, in the spirit of "You couldn't pay me enough to do that." I think there is an entire culture that has to change.
I completely agree than an entire culture has to change. (I kind of hinted at that with my mention of removing disruptive kids.)
ReplyDeleteEven though the government paid for me to get an M.A.T. in teaching high school chemistry, I decided in 1969 -- after only 3 years of teaching (plus 1/2 year of subbing after I had my first child) -- that you could not pay me enough to continue doing it and I went back to grad school and changed careers.
At one time I thought I might go back to teaching as a "third" career but it's gotten much worse in the ensuing 40 years. I had some wonderful experiences teaching and still miss it but those students and that first school environment are long gone.
Here's the minimum of what I'd need. (Salary doesn't matter so much given that I have a pension I can live on but I think it's important in general for recruiting competent personnel.)
(1) Being treated as a professional. This means proper support services and being respected by my managers. Also a private office. And making my own lesson plans.
(2) Not being expected to show that I "care" about students by having to do unpaid extra work such as being a club sponsor or cafeteria monitor.
(3) Not being expected to act as a therapist for troubled students. (In the private sector you are trained to identify employees or colleagues who need to be referred to HR for, say, possible alcoholism or personal problems affecting performance, but you aren't expected to help them yourself.)
(4) No cell phones, iPods, or similar devices allowed in the classroom
There's probably more.
I don't know if this is my flaw, but I just don't like to study.
ReplyDeleteLOL! Why does this strike me as sooo funny?
I don’t know, but I can just imagine the parade of students who may have uttered this line (or were thinking of it) when meeting with their guidance counselors.
On the other hand, my son would explain his poor grades by saying, in a very serious tone, “I don’t know if this is my flaw, but I just don’t like to turn in class assignments.”
Due to poor grades and low SAT scores, my high school best friend could not get in to college any other way except by majoring in education. After being denied admission by her first, second and (IIRC) third choices, the education school finally accepted her. I think she had first applied to major in social science, and then language.
ReplyDeleteMy apologies to all the competent teachers, but my subsequent experiences have mainly served to reinforce my poor opinion formed from that initial experience with education schools.
tex: Well I get decent grades. The question to me is whether doing SAT prepatory material would really help me.
ReplyDeleteTo me, I've never studied harder simply because an exam is coming up, save for some minor revision of key formulas. I never really did any *extra* PSLE preparation of my own volition for instance. In this case I must thank my P6 teachers extra assessment books, mock exams and making us stay back for remedials and lectures. I never went for private tuition and rarely did extra assessment books. And I did decently and got a 250 for 2002 (top 5%).
Perhaps I should say I don't like to study *hard*. I always like to learn things as my own leisure, rather than work. I'm wondering had I done dozens of assessment books like the others, whether I would have gotten a 280 instead of a 250 for the PSLE, a 2300 instead of a 2090 for the SATs, etc. For some time now I've assumed I'm a person who doesn't have to study very much (and I've often come to the conclusion that it's about using the right techniques, rather than how hard you study), but I could be dead wrong.
To me, SAT preparation is about drilling to do well on the test, i.e. rote, and that has always been my excuse from shying away from it. It could also be just that -- an excuse.
le radical, I hope I didn’t give the impression I thought you were making excuses for poor grades. From what you’ve written, it sounds like you’ve usually done very well in school without studying hard, thank you very much.
ReplyDeleteMy son is like that, and he will tell me that spending two hours studying in order to raise his grade from a 92% to a 95% is not worth the trouble. That attitude disappoints me, but I also apply that type of logic all the time in instances where the goals are just not sufficiently important to justify the extra effort.
For instance, I want my house clean. But, spending an extra six hours (or whatever) to achieve that extra super sparkle spotless appearance is definitely not worth the extra time.