kitchen table math, the sequel: "These 19 curricular projects have essentially been experiments."

Friday, February 16, 2007

"These 19 curricular projects have essentially been experiments."

conclusions of the National Research Council:


Executive Summary

Under the auspices of the National Research Council, this committee’s charge was to evaluate the quality of the evaluations of the 13 mathematics curriculum materials supported by the National Science Foundation (NSF) (an estimated $93 million) and 6 of the commercially generated mathematics curriculum materials (listing in Chapter 2).

The committee was charged to determine whether the currently available data are sufficient for evaluating the effectiveness of these materials and, if these data are not sufficiently robust, the committee was asked to develop recommendations about the design of a subsequent project that could result in the generation of more reliable and valid data for evaluating these materials.
page 1

The following 13 mathematics curricula programs1 (The K-12 Mathematics Curriculum Center, 2002) were supported by the NSF, and the evaluations of these materials were reviewed by our committee:


Elementary School:

  • Everyday Mathematics (EM), Grades K-6 (SRA/McGraw-Hill)

  • Investigations in Number, Data and Space, Grades K-6 (Scott Foresman) [ed.: also called "TERC"]

  • Math Trailblazers, Grades K-6 (Kendall/Hunt Publishing Company)



Middle School:
  • Connected Mathematics Project (CMP), Grades 6-8 (Prentice Hall)
  • Mathematics in Context (MiC), Grades 5-8 (Holt, Rinehart and Winston)

  • MathScape: Seeing and Thinking Mathematically, Grades 6-8 (Glencoe/McGraw-Hill)

  • MathThematics (STEM), Grades 6-8 (McDougal Littell)

  • Middle School Mathematics Through Applications Project (MMAP) Pathways to Algebra and Geometry, Grades 6-8 (currently unpublished)


High School:

  • Contemporary Mathematics in Context (Core-Plus), Grades 9-12 (Glencoe/McGraw-Hill)

  • Interactive Mathematics Program (IMP), Grades 9-12 (Key Curriculum Press)

  • MATH Connections: A Secondary Mathematics Core Curriculum, Grades 9-12 (IT’S ABOUT TIME, Inc.)

  • Mathematics: Modeling Our World (MMOW/ARISE), Grades 9-12 (W.H. Freeman and Company)

  • Systemic Initiative for Montana Mathematics and Science (SIMMS) Integrated Mathematics, Grades 9-12 (Kendall/Hunt Publishing Company)

page 19


These 19 curricular projects essentially have been experiments. We owe them a careful reading on their effectiveness. Demands for evaluation may be cast as a sign of failure, but we would rather stress that this examination is a sign of the success of these programs to engage a country in a scholarly debate on the question of curricular effectiveness and the essential underlying question, What is most important for our youth to learn in their studies in mathematics? To summarily blame national decline on a set of curricula whose use has a limited market share lacks credibility. At the same time, to find out if a major investment in an approach is successful and worthwhile is a prime example of responsible policy. In experimentation, success and worthiness are two different measures of experimental value. An experiment can fail and yet be worthy. The experiments that probably should not be run are those in which it is either impossible to determine if the experiment has failed or it is ensured at the start, by design, that the experiment will succeed. The contribution of the committee is intended to help us ascertain these distinctive outcomes.

The charge to the committee was “to assess the quality of studies about the effectiveness of 13 sets of mathematics curriculum materials developed through NSF support and six sets of commercially generated curriculum materials.”

[snip]

In response to our charge, the committee finds that:

The corpus of evaluation studies as a whole across the 19 programs studied does not permit one to determine the effectiveness of individual programs with high degree of certainty, due to the restricted number of studies for any particular curriculum, limitations in the array of methods used, and the uneven quality of the studies.
page 188


source:
On Evaluating Curricular Effectiveness: Judging the Quality of K-12 Mathematics Evaluations (2004)
The National Academies Press


The National Academies Press (NAP) was created by the National Academies to publish the reports issued by the National Academy of Sciences, the National Academy of Engineering, the Institute of Medicine, and the National Research Council, all operating under a charter granted by the Congress of the United States. The NAP publishes more than 200 books a year on a wide range of topics in science, engineering, and health, capturing the most authoritative views on important issues in science and health policy. The institutions represented by the NAP are unique in that they attract the nation's leading experts in every field to serve on their award-winning panels and committees. This is the right place for definitive information on everything from space science to animal nutrition.

10 comments:

Instructivist said...

The corpus of evaluation studies as a whole across the 19 programs studied does not permit one to determine the effectiveness of individual programs with high degree of certainty, due to the restricted number of studies for any particular curriculum, limitations in the array of methods used, and the uneven quality of the studies.

So what does that mean? That all the "research shows" cited to show that voodoo math is wonderful and the way to go is a hoax?

Catherine Johnson said...

yup

that's what it means

this book was published in 2004

Catherine Johnson said...

Where math instruction is concerned research shows damn little.

Catherine Johnson said...

We have research showing that Direct Instruction works generally, across many subjects.

We have research in cognitive science establishing the principles and mechanisms of learning and cognition.

We do not have research showing us specifically how math should be taught to students.

Tex said...
This comment has been removed by the author.
Tex said...

This is definitely going in my arsenal for when I need to refute the “research based” argument my school puts out!


“Here’s a photo of some of my students/guinea pigs”:

http://www.meerschweinchen.de/cavia/hollanders_red.jpg

“Aren’t they cute!”

Catherine Johnson said...

This is definitely going in my arsenal for when I need to refute the “research based” argument my school puts out!

I always have it handy; yesterday I realized I hadn't posted it to ktm 2.

The great thing about Blogger is that you can find things incredibly quickly using the tags in the sidebar.

The search engine is good, too.

Any time you want to find this quote you can click on "Catherine," "constructivist math," or "education research."

I think I'll also put it under the tags for TERC, EM, and Trailblazers.

Markov Chaney said...

Hmm. Interesting the spin people are putting on this. In plain English (including words in the actual quotations from the report): "we would rather stress that this examination is a sign of the success of these programs to engage a country in a scholarly debate on the question of curricular effectiveness and the essential underlying question, What is most important for our youth to learn in their studies in mathematics? To summarily blame national decline on a set of curricula whose use has a limited market share lacks credibility." And yet, that's exactly what you folks and others are doing. Is that supported by the report? Not in the least.

lisa said...
This comment has been removed by the author.
Anonymous said...

" Instructivist said (...)
So what does that mean? That all the "research shows" cited to show that voodoo math is wonderful and the way to go is a hoax?"

February 16, 2007 2:48 PM
Catherine Johnson said...

"yup

that's what it means

this book was published in 2004"

February 16, 2007 4:17 PM

No, it doesn't mean it's a "hoax". You let your bias lead you too often. Your divisive mindset will not lead to productive discussion and progress. Very sad.