Posts
I never really paid much attention to the What Works Clearinghouse (WWC) because I thought it just evaluated research and did not pass judgment on curricula. I really should pay attention!
Ken's blog refers to:
"... an alarming pattern in which the WWC allows the evaluation of the research to include a testing instrument developed by the authors of the education program as a valid measure of success."
and:
"This seems like extremely flimsy evidence to me and it seems to send a message to publishers on how to cook the books to get the thumbs up from the WWC."
I think it's worse than that.
In discussions yesterday with the curriculum person at my son's school (a very nice person), she talked about WWC and their recommendations. Recommendations? I thought they just evaluated research. How could they make recommendations if there really wasn't enough research? Well, I started my education of WWC. Perhaps Ken (and others) can get me up to speed. It appears that there is too little data and too much judgment.
As an example, look what I quickly found at their site.
Elementary School Math
"Everyday Mathematics was found to have potentially positive effects on students’ mathematics achievement."
[I really need to see what this is based on.]
"The Saxon Elementary School Math was found to have no discernible effects on mathematics achievement."
"Progress in Mathematics © 2006 was found to have no discernible effects on math achievement."
"No discernible effects"
They don't say that there is not enough data. They say that they don't see any effect. Wow! They don't say that for study 'A' or study 'B', they see no effect. They are making a judgment about the curriculum. How can they do this even for a relative judgment?
Middle School Math
UCSMP Algebra was found to have potentially positive effects on math achievement.
Saxon Middle School Math was found to have positive effects on math achievement.
The I CAN Learn® Pre-Algebra and Algebra curricula were found to have positive effects on math achievement. [The I CAN Learn® Pre-Algebra and Algebra computerized curricula are designed to cover mathematics and problem-solving skills for ethnically diverse, inner-city students in grades 6–12.]
I CAN Learn has positive effects, but compared to what? Relativity is nice, sometimes, but not in math.
Note that EM & UCSMP are positioned quite well - no negative marks. Apparently, it's enough for our curriculum advisor to keep them on the list and provide ammo for the pro-EM crowd.
It seems to me that WWC is being used for much more than it should be, and WWC is not going to stop them. It's nice to be popular, even if you don't have enough data to support it. The problem is that our curriculum advisor is using relative WWC judgments to make decisions, not absolute criteria, like the California Green Dot standards.
Subscribe to:
Post Comments (Atom)
3 comments:
The WWC summarizes its findings in very short .pdf documents. To the casual reader, one might worry that a curriculum director, or other school employee, may well look only at the one statement and conclude EM is a positive program, anointed by the WWC.
This is a real risk. We each know how thorough our decision-makers are when evaluating a school curricular program (not).
It takes work to dig into the WWC findings. The first thing that strikes me about EM is that WWC threw out 57 of the 61 "research" studies as a preliminary matter.
Only 4 studies passed the baseline credibility threshhold. This point is often lost. Yes, EM has 61 studies, but 57 weren't even worth reading and the whole lot of them got about one sentence in the WWC report. This fact is easily overlooked by decision-makers.
Only one of the surviving 4 studies used a norm-referenced standardized math test (The Iowa Test of Basic Skills - ITBS). That study by Woodward and Baxter found that EM actually performed worse than non-EM.
I'm concerned about What Works, too.
Why aren't the two positive Saxon studies from CA there?
They have both been published in refereed journals.
Even if WW were to decide that the studies don't meet their standards, the studies should be cited and evaluated.
Steve has made the point on numerous occasions that research per se solves nothing. Research doesn't identify values and goals; nor can it.
That's slightly OT, but what it all comes down to for me is that I never tie everything to "research says."
I always come back to politics, power, and choice.
When I say "come back to," I mean that I come back to it in conversations with other parents and with administrators.
Post a Comment