I finally got a copy of Teach Like a Champion. Since I teach at a community college, not everything applies. But I do teach a basic math class. And the book has inspired me to start taking data on my tests. This led me to analyze my tests to see what and how much I test on each topic.
At the end of the summer semester, I used the department written final to see if I could determine how much mastery my students had on each topic that I taught. Unfortunately, I don't have data from the beginning of the semester. But what I found wasn't very good: I measured the mastery on 15 topics by calculating a percent correct for each topic. For example, there were 5 problems on whole number operations and 11 students. This gives a total of 55 problems. Only 76% were answered correctly. If I had to take a guess, I would say that the class probably had over 70% mastery on whole number operations before they came into class. My highest % mastery was decimal operations, fraction operations and dimensional analysis with 93%, 85% and 93% mastery respectively. My lowest topics were divisibility tests and lowest common multiple and prime factorization. I then analyzed each incorrect answer to see what the most common errors were. 66% of the errors were concept errors: either the student didn't answer the question or did not use the correct mathematical technique. 19% of the errors were process error. In this category are errors like the following: a student knows that when multiplying mixed numbers, he must turn the mixed number into an improper fraction, but he does this incorrectly. 7% of the errors were mathematical. Another 8 % were divided evenly between not reducing fractions and sign errors.
So I'm teaching basic math again this semester, and I am trying to determine what I should be doing to ensure mastery. I am going to try curriculum based measurement. This is a technique I read about that is used in some elementary schools. You give students very short exams on a particular topic. Instead of counting the number of problems that are correct, you count the number of correct digits. For example, if a student has to add 99 and 49, there is a possibility of 3 correct digits, not one correct problem. You then do some number crunching and adjust your teaching accordingly. (I realize I'm gliding over this.)
It doesn't seem like any of the other instructors at my community college are collecting and analyzing data like this. If anyone has any links or resources on this topic, I would love to hear about it.