"These 19 curricular projects have essentially been experiments."

conclusions of the National Research Council:


Executive Summary

Under the auspices of the National Research Council, this committee’s charge was to evaluate the quality of the evaluations of the 13 mathematics curriculum materials supported by the National Science Foundation (NSF) (an estimated $93 million) and 6 of the commercially generated mathematics curriculum materials (listing in Chapter 2).

The committee was charged to determine whether the currently available data are sufficient for evaluating the effectiveness of these materials and, if these data are not sufficiently robust, the committee was asked to develop recommendations about the design of a subsequent project that could result in the generation of more reliable and valid data for evaluating these materials.
page 1

The following 13 mathematics curricula programs¹ (The K-12 Mathematics Curriculum Center, 2002) were supported by the NSF, and the evaluations of these materials were reviewed by our committee:

Elementary School:

• Everyday Mathematics (EM), Grades K-6 (SRA/McGraw-Hill)

• Investigations in Number, Data and Space, Grades K-6 (Scott Foresman) [ed.: also called "TERC"]
  

• Math Trailblazers, Grades K-6 (Kendall/Hunt Publishing Company)

Middle School:

• Connected Mathematics Project (CMP), Grades 6-8 (Prentice Hall)

• Mathematics in Context (MiC), Grades 5-8 (Holt, Rinehart and Winston)

• MathScape: Seeing and Thinking Mathematically, Grades 6-8 (Glencoe/McGraw-Hill)

• MathThematics (STEM), Grades 6-8 (McDougal Littell)

• Middle School Mathematics Through Applications Project (MMAP) Pathways to Algebra and Geometry, Grades 6-8 (currently unpublished)

High School:

• Contemporary Mathematics in Context (Core-Plus), Grades 9-12 (Glencoe/McGraw-Hill)

• Interactive Mathematics Program (IMP), Grades 9-12 (Key Curriculum Press)

• MATH Connections: A Secondary Mathematics Core Curriculum, Grades 9-12 (IT’S ABOUT TIME, Inc.)

• Mathematics: Modeling Our World (MMOW/ARISE), Grades 9-12 (W.H. Freeman and Company)

• Systemic Initiative for Montana Mathematics and Science (SIMMS) Integrated Mathematics, Grades 9-12 (Kendall/Hunt Publishing Company)

page 19


These 19 curricular projects essentially have been experiments. We owe them a careful reading on their effectiveness. Demands for evaluation may be cast as a sign of failure, but we would rather stress that this examination is a sign of the success of these programs to engage a country in a scholarly debate on the question of curricular effectiveness and the essential underlying question, What is most important for our youth to learn in their studies in mathematics? To summarily blame national decline on a set of curricula whose use has a limited market share lacks credibility. At the same time, to find out if a major investment in an approach is successful and worthwhile is a prime example of responsible policy. In experimentation, success and worthiness are two different measures of experimental value. An experiment can fail and yet be worthy. The experiments that probably should not be run are those in which it is either impossible to determine if the experiment has failed or it is ensured at the start, by design, that the experiment will succeed. The contribution of the committee is intended to help us ascertain these distinctive outcomes.

The charge to the committee was “to assess the quality of studies about the effectiveness of 13 sets of mathematics curriculum materials developed through NSF support and six sets of commercially generated curriculum materials.”

[snip]

In response to our charge, the committee finds that:

The corpus of evaluation studies as a whole across the 19 programs studied does not permit one to determine the effectiveness of individual programs with high degree of certainty, due to the restricted number of studies for any particular curriculum, limitations in the array of methods used, and the uneven quality of the studies.
page 188

source:
On Evaluating Curricular Effectiveness: Judging the Quality of K-12 Mathematics Evaluations (2004)
The National Academies Press


The National Academies Press (NAP) was created by the National Academies to publish the reports issued by the National Academy of Sciences, the National Academy of Engineering, the Institute of Medicine, and the National Research Council, all operating under a charter granted by the Congress of the United States. The NAP publishes more than 200 books a year on a wide range of topics in science, engineering, and health, capturing the most authoritative views on important issues in science and health policy. The institutions represented by the NAP are unique in that they attract the nation's leading experts in every field to serve on their award-winning panels and committees. This is the right place for definitive information on everything from space science to animal nutrition.