kitchen table math, the sequel: Testing in Basic Math in Community College

Monday, September 13, 2010

Testing in Basic Math in Community College

I finally got a copy of Teach Like a Champion. Since I teach at a community college, not everything applies. But I do teach a basic math class. And the book has inspired me to start taking data on my tests. This led me to analyze my tests to see what and how much I test on each topic.

At the end of the summer semester, I used the department written final to see if I could determine how much mastery my students had on each topic that I taught. Unfortunately, I don't have data from the beginning of the semester. But what I found wasn't very good: I measured the mastery on 15 topics by calculating a percent correct for each topic. For example, there were 5 problems on whole number operations and 11 students. This gives a total of 55 problems. Only 76% were answered correctly. If I had to take a guess, I would say that the class probably had over 70% mastery on whole number operations before they came into class. My highest % mastery was decimal operations, fraction operations and dimensional analysis with 93%, 85% and 93% mastery respectively. My lowest topics were divisibility tests and lowest common multiple and prime factorization. I then analyzed each incorrect answer to see what the most common errors were. 66% of the errors were concept errors: either the student didn't answer the question or did not use the correct mathematical technique. 19% of the errors were process error. In this category are errors like the following: a student knows that when multiplying mixed numbers, he must turn the mixed number into an improper fraction, but he does this incorrectly. 7% of the errors were mathematical. Another 8 % were divided evenly between not reducing fractions and sign errors.

So I'm teaching basic math again this semester, and I am trying to determine what I should be doing to ensure mastery. I am going to try curriculum based measurement. This is a technique I read about that is used in some elementary schools. You give students very short exams on a particular topic. Instead of counting the number of problems that are correct, you count the number of correct digits. For example, if a student has to add 99 and 49, there is a possibility of 3 correct digits, not one correct problem. You then do some number crunching and adjust your teaching accordingly. (I realize I'm gliding over this.)

It doesn't seem like any of the other instructors at my community college are collecting and analyzing data like this. If anyone has any links or resources on this topic, I would love to hear about it.


Catherine Johnson said...

I haven't read yet but wanted to say that I've been having students do one-minute 'sprints' in my developmental composition class.

After I found the "Rule of 20" article, I figured: OK, I don't know how to put together a "standard celeration chart" and I don't have a precision teaching curriculum in any event, but what the he**?

When I tested myself on the first worksheet, on the its/it's distinction, lo and behold I finished 20 questions correctly in 1 minute.

So my plan now is to use the exercises from the Purdue OWL site, figure out how long it takes me to do them, then set that time as the standard for fluency.

The fun part for the class so far is seeing how quickly students get faster. No one hit 20/minute the first day; on the second day at least 4 students were already there -- and I'm positive they didn't time themselves in between classes.

As soon as a student hits 20, I'll move him or her on to the next sheet AND to an 'interleaved' worksheet (aka mixed review).

I'm going to be watching to see what happens in their papers & paragraphs....but I'm not sure how to measure improvement or lack of improvement.

Catherine Johnson said...

An interesting aspect of the class so far: although the class is 'developmental,' students' spelling, punctuation, and grammar isn't all that bad --- certainly not as flawed as I'd expected.

At least so far, the two big weaknesses I see are in 'sentence combining' and paragraph development & unity.

The sentence combining issue is interesting, because I can see that a number of the students are reaching towards sophisticated sentence construction -- and getting tangled up in it.

They're 'past' simple, declarative sentences, but they can't fluently produce complex sentence structures.

At least, that's the way it looks to me.

Catherine Johnson said...

As for resources, I find the precision teaching work fantastically helpful.

Also, definitely take a look at the TIMES article a couple of people posted -- especially the article on "interleaved" practice.

I'll send you my article on "cumulative practice," which I think may be the most important research on teaching math that I've seen.

I'm trying to use the concept of "cumulative practice" in my composition class.

The Effects of Cumulative Practice on Mathematics Problem Solving by Kristin H. Mayfield & Philip N. Chase

Catherine Johnson said...

The core insight of precision teaching is that 'mastery' defined in terms of percent correct isn't enough.

To **really** remember what you've learned in class, you have to achieve fluency, too -- you have to 'get up to speed.'

That's what I'm working on in my class with punctuation, its/it's, sentence combining, etc.

The precision teaching folks also seem to find that once you are fast as well as accurate, you start to be able to apply what you've learned to new problems & situations - but I'm less clear about what exactly they find in this realm.

Nevertheless, simply REMEMBERING what you've been taught would be a very good thing.

Anonymous said...

Change the capital HTML to lowercase.

-Mark R.

Anonymous said...

Sigh ... Comment got eaten

Lisa said...

I find all this interesting. I'm trying to see how to use it in teaching my own kids at home. Going to go read the linked article now.

Anne Dwyer said...

I've started to give timed mastery tests at the beginning and end of class to assess whether students have mastered the skills. My conclusions are really too long to give in the comments section. But it's clear that students believe that all they need to do is the homework. They really have not concept of making sure that they have mastered the material. It is fascinating to look at the errors they make under those circumstances.

Allison said...


--But it's clear that students believe that all they need to do is the homework. They really have not concept of making sure that they have mastered the material.

Two thoughts: first, are you sure the second means the first is true?

Is it possible they are simply working that hard to tread water, and have no time/effort/clue how to do more than the homework? Is successfully doing the homework problem set so big that they don't know when (let alone how) they'd do more?

Second, why would they know they have to do more? When have they ever had to? How would they know how to build more toward mastery, if not for doing their homework? As in, what's your concept of what homework is for? Should you be able to get a fairly easy A on the homework if you've already done practice to mastery? Is the homework the culmination of what's learned? Or is there more past that that they need?

For example, is there a textbook that has additional problems for them to work out, with solutions in the back? Do you assign "exercises", or some other set of problems for them to work before they tackle the homework? Do they have any way to build up to the homework? If the homework isn't far enough out, is there a way to build past it?