kitchen table math, the sequel: rat psych - "careless errors" in reading the SAT

Sunday, October 23, 2011

rat psych - "careless errors" in reading the SAT

During my year of living dangerously, doing SAT math prep off and on with C., I was chronically amazed stunned by the number and type of "careless errors" he and I both made taking timed sections of the test. In particular, I made repeated errors of "simple" reading, particularly when I was tired or the room was hot. I made so many reading errors that when I finally took the real test, I had no way to predict my math score at all: no way to estimate how many reading errors I had -- or had not -- made.

I eventually came up with a theory of careless errors, the details of which I've forgotten at the moment. I do recall that it had to do with working memory. Arguably the SAT tests working memory above all: all 10 sections put you into working memory blowout. I experienced working memory blowout so often that I began to notice a connection. As far as I can tell, you make more careless errors when your working memory is overtaxed (and you hit the limits of working memory much more quickly when you're sleep-deprived or overheated).

I've just come across a new study that I think confirms my subjective experience:
This study resolves two long-standing debates in the field. Does our working memory function like slots, and after our four slots [emphasis added] are filled with objects we cannot take in any more; or does it function like a pool that can accept more than four objects, but as the pool fills the information about each object gets thinner? And is the capacity limit a failure of perception, or of memory? [emphasis added]

“Our study shows that both the slot and pool models are true,” says Miller. “The two hemispheres of the visual brain work like slots, but within each slot, it’s a pool. We also found that the bottleneck is not in the remembering, it is in the perceiving.” [emphasis added] That is, when the capacity for each slot is exceeded, the information does not get encoded very well. The neural recordings showed information about the objects being lost even as the monkeys were viewing them, not later as they were remembering what they had seen.
Picower: 1 Skull + 2 Brains = 4 Objects in Mind
Failures of working memory are failures of perception!

Subjectively, that's what I experienced taking practice sections; that's what it felt like. Once I hit a certain level of tiredness, or heat, or working memory blow-out, I stopped being able to read.

The same thing happens on the reading and writing sections, too. The reading and writing sections are so taxing that you reach points where you simply cannot take in what the sentence or paragraph before you says. * I'm not talking about losing the ability to answer questions about the sentence or paragraph.

I'm talking about losing the ability just to read the words on the page.

I'm a 10
rat psych: what to do about SAT math (part 1)
rat psych: what to do about SAT math (part 2)
rat psych: what to do about SAT math (part 3)
rat psych: careless reading errors on the SAT

* I say "you" because I know I am not alone in this.


Daniel Ethier said...

I've noticed something like this in my teaching of honors algebra students early in the year.

We start with order of operations. They know the rules. But in complex expressions, they make mistakes.

It just seems like their working memory gets taxed and they stop knowing order of operations very well.

Later in the year, this is much less of a problem, probably as they gain more automaticity, which reduces the load on working memory.

Debbie Stier said...

I'm pretty sure I make more errors on the last section (Writing) than on the prior sections -- because I'm fatigued.

There are times when I look at a problem -- say a Improving Sentences Question -- and if the whole sentence is underlined, and I'm on section 10 (4-5 hours into this test), it might as well be dog barks.

I just can't process it sometimes...when I'm fatigued.

Catherine Johnson said...

It just seems like their working memory gets taxed and they stop knowing order of operations very well.

There is absolutely something like this going on.

btw, my post sounds as if I'm talking about math -- but I'm talking about the whole test.

I have **frequently** had the experience of not being able to read a sentence in the writing and reading sections.

Simply cannot read it.

I can decode each sequentially, but when I get to the end of the sentence, I have no idea what I've read.

This has nothing to do with "reading carefully."

It has to do with working memory being maxed out. I actually can't get the content into my WM.

I think there've been times I've re-read a sentence as many as 5 times just trying to hang onto it.

If I take the test again (or if I just take a sample section again) I'm going to see if I can experiment with willfully blanking-out whatever's in WM at the moment.

Not sure that's going to work....but it's a possibility, especially with the long reading passages.

ChemProf said...

I'd also suggest to you that this increasing load on working memory is partly a function of the SAT allowing graphing calculators, at least on the math section. For example, when I took the SAT in the 80's, there were order of operation questions. Even with a scientific calculator, it was helpful to know order of operations cold, so you could efficiently enter the data.

Now, students just enter the whole expression and hit "enter." There is no point in having these questions on the SAT, but that isn't because students are proficient. In fact, I'm always amazed at students who waste time entering zeros (like 1.000) because they don't really get how the calculator works. But losing those kinds of simple questions means that the remaining ones are going to become trickier and trickier.

I will also say that I found the best way to study for the SAT was to take the SAT, but I was a weird CTY kid who took it a bunch, so I'm not sure that my experience helps anyone else!

ChemProf said...

Actually, the graphing calculator is another reason they avoid "solving for x" questions -- modern calculators will do that for you if you know how to use them! That's also why they want the answer in radical rather than decimal form (even if their mechanism is that too many students get the problem right).

SteveH said...

Is the mental overload due to the complexity or the time constraints? We've talked about being able to look at questions and knowing whether you can solve them. If it then takes too much time to solve the problem, is it because you don't know enough math, aren't prepared, or that you don't have some sort of aptitude.

I have a very negative reaction to being rushed. I want to slow down and ensure that every step is carefully defined. Accuracy, not speed, is critical. I remember working on a project with a team member who loved to throw code together. It was quite amazing to watch. The code was also full of bugs.

As for calculators, The SAT could ban all but simple ones if they wanted to. Calculators aren't driving these questions. The issue is that they are trying to identify some sort of aptitude, but speed and special cases are the only tools they have to do that. In the arms race against preparation, they are left with doing anything that will ensure that only a certain number of kids will get 800s.

ChemProf said...

"The SAT could ban all but simple ones if they wanted to."

They could, but that is a logistical pain in the butt, especially as many students only own the expensive graphing calculators. (Trust me on this -- I banned graphing calculators in my class, and it is amazing how many students have never used anything else.)

For that matter, the SAT could ban calculators all together (and used to), in which case you would see a return to simpler problems. I'm not saying that's the only problem, but I am saying that using their method of constructing the test (see what fraction of students can solve the experimental problems, and adjust from there) allowing graphing calculators is going to push the exam in certain directions.