kitchen table math, the sequel: the deathless meme of the high-performing school

Tuesday, April 29, 2008

the deathless meme of the high-performing school

The school system, the line goes, is in crisis, with its students performing particularly badly in science and math, year after year, in international rankings. But the statistics here, although not wrong, reveal something slightly different. The real problem is one not of excellence but of access. The Trends in International Mathematics and Science Study (TIMSS), the standard for comparing educational programs across nations, puts the United States squarely in the middle of the pack. The media reported the news with a predictable penchant for direness: "Economic Time Bomb: U.S. Teens Are Among Worst at Math," declared The Wall Street Journal.

But the aggregate scores hide deep regional, racial, and socioeconomic variation. Poor and minority students score well below the U.S. average, while, as one study noted, "students in affluent suburban U.S. school districts score nearly as well as students in Singapore, the runaway leader on TIMSS math scores." (pdf file) The difference between the average science scores in poor and wealthy school districts within the United States, for instance, is four to five times as high as the difference between the U.S. and the Singaporean national average. In other words, the problem with U.S. education is a problem of inequality. This will, over time, translate into a competitiveness problem, because if the United States cannot educate and train a third of the working population to compete in a knowledge economy, this will drag down the country. But it does know what works.

The U.S. system may be too lax when it comes to rigor and memorization, but it is very good at developing the critical faculties of the mind.

The Post-American World
Fareed Zakaria
excerpt posted at:
Foreign Affairs
realclearpolitics

This is the logic of NCLB: white schools good; black schools bad. Affluent white parents exercise choice by moving to affluent white suburbs where they can enroll their children in affluent white schools.

True.

That is exactly what affluent white parents do, affluent white parents and their camp followers, a hardy band of non-affluent white, black, and Hispanic parents who move heaven and earth to get their kids into these schools, too.

But the story doesn't end there. If it did, kitchen table math wouldn't exist.


number one

Number one, let's review the meaning of the word study.

The words "students in affluent suburban U.S. school districts score nearly as well as students in Singapore, the runaway leader on TIMSS math scores" do not appear in a study.* They appear in a trade publication published by Tau Beta Pi, The Engineering Honor Society.

Tau Beta Pi, interestingly enough, sponsors a K-12 initiative to improve the public schools: (pdf file)

...significantly large percentage of our high-school graduates lags behind their peers in many developed countries, with respect to performance in mathematics and science. In many cases, these countries are our competitors in the global marketplace, and hence this situation has a direct impact on our economy in the long term.

Now that is a world many parents whose children are currently attending affluent suburban schools will recognize.

The article Mr. Zakaria quotes, which appeared in the Winter 2007 issue of The Bent of Tau Beta Pi, cites the same studies everyone cites when speaking of U.S. kids' math and science knowledge in an international context: TIMSS & PISA. Neither study found that advanced students in the U.S. are on par with advanced students elsewhere. They found the opposite. Advanced students here are not advanced elsewhere. The only U.S. students who are competitive with their peers in Europe are those taking Advanced Placement calculus, which is 5% of the total.

And, of course, we have no idea how AP calculus students would fare in a head to head competition with students in Singapore because Singapore wasn't included in the comparison of advanced students. Given the fact that Singapore students performed significantly better than all 16 of the countries that were included, I think it's safe to assume that America's AP students would take a shellacking if they were to go up against Singapore students studying calculus at the same age.

Here is what the Department of Education has to say about advanced students in the U.S. and how they compare to advanced students in Europe:

Overview and Key Findings Across Grade Levels

At the fourth grade, U.S. students were above the international average in both science and mathematics. In the eighth grade, U.S. students scored above the international average in science and below the international average in mathematics. At the end of secondary schooling (twelfth grade in the U.S.), U.S. performance was among the lowest in both science and mathematics, including among our most advanced students.

[snip]

Achievement of Advanced Students (16 Countries) **

The advanced mathematics and physics assessments were administered to a sample of the top 10-20 percent of students in their final year of secondary school in each nation. In the advanced mathematics assessment, this included the 14 percent of U.S. students who had taken or were taking pre-calculus, calculus, or AP calculus compared to advanced mathematics students in other countries. In the physics assessment, this included the 14 percent of U.S. students who had taken or were taking physics or AP physics compared to advanced science students in other countries.

  • The performance of U.S. physics and advanced mathematics students was below the international average and among the lowest of the 16 countries that administered the physics and advanced mathematics assessments. The U.S. outperformed no other country on either assessment.
  • When you compare U.S. twelfth graders with Advanced Placement calculus instruction (about 5 percent of the U.S. cohort) to all advanced mathematics students in other nations, their performance was at the international average and significantly higher than 5 other countries.
  • When you compare U.S. twelfth graders with Advanced Placement physics instruction (about 1 percent of the U.S. cohort) to all advanced science students in other nations, their performance was below the international average and significantly higher than only 1 other country.
  • More countries outperformed U.S. students in physics than in advanced mathematics. This differs from results for mathematics and science general knowledge, where more countries outperformed the U.S. in mathematics than in science.

update: I'd forgotten that the U.S. has decided to "sit out" the TIMSS test of advanced students in 2008:

This article reports on reactions to the U.S. Department of Education's first time decision to sit out an international study designed to show how advanced high school students around the world measure up in math and science. Mark S. Schneider, the commissioner of the department's National Center for Education Statistics, which normally takes the lead in managing the U.S. portion of international studies of student performance in those subjects, said budget and staffing constraints prevent his agency from taking part in the upcoming study, which is known as the "Trends in Mathematics and Science Study-Advanced 2008." The study, in which nine countries have so far agreed to participate, will test students taking physics and upper-level math classes, such as calculus, at the end of their secondary school years. It comes as national leaders in the United States are promoting improved math and science education as critical to protecting the nation's economic edge. The statistics agency is still overseeing the regular administration of Trends in Mathematics and Science Study (TIMSS), which got under way in the United States this year. The larger of the two studies, the regular TIMSS assesses 4th and 8th math and science achievement in 62 nations.

Singapore Math exit test 6B

* At least, they do not appear in a study that is findable by Google.
** Australia, Austria, Canada, Cyprus, Czech Republic, Denmark, France, Latvia, Lithuania, Norway, Germany, Russian Federation, Slovenia, Sweden, Switzerland, United States NOTE: none of the Asian countries are included in this list

12 comments:

SteveH said...

"Here is what the Department of Education has to say about advanced students in the U.S. and how they compare to advanced students in Europe"

How do they select advanced students in 4th, 8th, and 12th grades? Are these the same students, or do the advanced students change? What percentage of the population are they talking about?

Catherine Johnson said...

I think they only compared advanced students in the 12th grade, which is (I assume) why the Asian countries dropped out. Singapore students, for instance, don't take calculus in high school -- and I believe they graduate somewhere around....age 16? iirc, they begin school at a later age than our kids do and they exit school slightly earlier.

I think they simply took the top 10 or perhaps 20% of each country's 12th grade students and compared their scores.

The AP kids did better in a more recent comparison (this one's from the 90s). In that one the AP kids were below just one country: France. However, once again none of the Asian countries were included.

Catherine Johnson said...

Here's the Schmidt statement:

What about the US's better students? When asked, Schmidt replied, "For some time now, Americans have comforted themselves when confronted with bad news about their educational system by believing that our better students can compare with similar students in any country in the world. We have preferred not to believe that we were doing a consistently bad job. Instead, many have believed that the problem was all those 'other' students who do poorly in school and who we, unlike other countries, include in international tests. That simply isn't true. TIMSS has burst another myth - our best students in mathematics and science are simply not 'world class'. Even the very small percentage of students taking Advanced Placement courses are not among the world’s best."

http://ustimss.msu.edu/12gradepr.htm

Anonymous said...

Am I reading this quote correctly?

"When you compare U.S. twelfth graders with Advanced Placement calculus instruction (about 5 percent of the U.S. cohort) to all advanced mathematics students in other nations, their performance was at the international average..."

The USA top-5% is equivalent to the rest of the world 50th percentile (ie, average)?

-Mark Roulo

Anonymous said...

One of our graduating seniors is from Singapore, and yes, they leave high school at 16 (graduating in December, which caused her some problems since she didn't realize that meant entering a US college mid-year). Typically, they then spend two years in college (like a community college here) before going to university for 3-5 years, depending on the program. So, there isn't an equivalent to our 12th graders.

Catherine Johnson said...

The USA top-5% is equivalent to the rest of the world 50th percentile (ie, average)?

In that study (which I believe was
the 1997 or 1998 study) our top 5% were at the median for the top 10 to 20% of European students.

That makes sense to me, given the 10 year rule for memory consolidation and for development of expertise. Virtually all of our top public school kids are moving much, much more slowly through the curriculum than their counterparts in Europe (as I understand it...)

Here's what worries me terribly about all of this.

The "naturals" will be fine, I think. Kids with natural math talent can come from behind. I could be too sanguine about this, of course, since they're trying to overtake equally gifted kids who've had a head start...but for some reason this seems do-able. (But maybe not for advanced work in mathematics and physics??? At NYU, all of the grad students in physics are foreign, I believe. Will check.)

The "smart-but-not-gifted" kids, like C., could be shut out. C. could easily be drawn to fields that require excellent mathematical skills & comprehension: economics, for instance.

Now, the field of economics has become over-mathified, I believe. (See, e.g.: An Important Emerging Economic Paradigm by Arnold Kling, which is no longer available online it appears.)

hmm...

I can email a copy of his article to anyone who would like to read. cijohn @ verizon.net

What I've seen in academia is that the various disciplines seem to "complexifiy" themselves, either by becoming "theoretical," in the case of language-based disciplines, or by becoming "mathified," in the case of the social sciences. Political science, for instance, has virtually nothing to do with politics at this point. It's all game theory.

I majored in psychology at Wellesley & Dartmouth, and I was a serious student. I did very well.

Today I can't understand peer-reviewed studies. I can barely understand some textbooks. Too much math.

C. is going to be drawn to the social sciences, and, at this point, the only remaining social science that does not depend entirely upon math (and has not been turned into "theory") is history.

That's fine; he may wish to major in history.

But I don't want him to have to major in history.

A couple of years ago we had several intense conversations with friends of ours whose son never learned long division. This kid is extremely smart; his SAT-V was 800. His first math score was in the 500s, iirc. Possibly the low 500s.

Even with massive, endless tutoring he never got beyond a score in the very low 600s (iirc).

His mom agreed with me that you can't "tutor" math one or even two years before the SAT. And, of course, now that I've discovered the 10-year rule for memory consolidation, I know why. Kids need a superb mathematics education from the get-go, and PART of what "superb" means is that math should be taught to mastery from the get-go and the mathematics curriculum should move along at a brisk clip.

C's verbal scores, assuming his ISEE scores are predictive, are going to be high; presumably high enough to get him into a reasonably selective liberal arts college or state university without high math scores.

But what happens once he's there?

He'll be fine in language-based classes, but in social science classes he'll be competing with kids who came in with 800 boards in math, a category that will include kids who went to private schools. (NYU now takes 35% of its freshman from private schools.)

I should turn this into a post because almost no one knows what goes on at major colleges and universities in terms of the evolution of disciplines.

Anyway, back to the boy I was talking about.

He was accepted by one of the "non-Ivy Ivies." Very good school; one of the "hot" schools today.

Like a lot of boys, he hadn't been very serious. Then, sometime during freshman or sophomore year, he got focused. He became a serious person.

At that point he became interested in majoring in psychology.

His parents told us they didn't see how he could possibly major in psychology "because of the math."

THINK ABOUT THAT.

This is a kid with 800 SAT-V who can't major in psychology at a good liberal arts college because he can't do math.

That's pretty much my nightmare.

(I haven't talked to his mom in a while -- I'll find out what he ended up majoring in.)

Anonymous said...

"What I've seen in academia is that the various disciplines seem to 'complexifiy' themselves, either by becoming 'theoretical,' in the case of language-based disciplines, or by becoming 'mathified,' in the case of the social sciences."

The mathefication is not just an academic thing. The real world is becoming more quantitative. Some examples:

(1) Major League Baseball has (slowly ... with a lot of resistance) started paying attention to statistics *much* more when evaluating players. The key here isn't just that people are paying attention to things like batting average. They have *always* done that. The key is that people are starting to try to correlate different things with winning. So, the modern baseball stats-head mostly doesn't care about batting average.

The stats are easily available to someone with a *solid* high school math education. The point, however, is that even baseball is getting much more quantitative than it historically has.

(2) Drug Design. More and more companies are trying to design drugs using computer models (not just physics/chemistry simulators). This requires *lots* of math beyond what a normal PhD chemist would know.

(3) Quantitative supply chain management. Think about what Wal-Mart does. Very mathy. Wasn't done by anyone 30 years ago.

(4) Wall Street Trading. I'd say that today if you don't have a very heavy math background, you basically *can't* trade stocks/bonds/etc on Wall Street today. There may be some exceptions, but the field is much more quantitative than 30 years ago. Read up on LTCM (which went down in flames!) for more details.

(5) SQM (Statistical Quality Control), used by many manufacturing companies, is a mathematical approach to quality control.

So it isn't just academia. A lot more jobs have gotten quantitative in the last 30 years.

Which sorta makes things like TERC even more scary.

-Mark Roulo

Catherine Johnson said...

oh my gosh --- must get this up front

you're way ahead of me

You're right.

I've "known" this is some kind of semi-conscious way, and it's been driving all my work with C. (and with me).

At this point I feel very handicapped by my lack of knowledge of statistics.

Catherine Johnson said...

Another data point: Nicholas Lemman, who is dean of the Columbia School of Journalism, taught himself how to do linear regression because reporters need, in his view, to move beyond classic "one side/the other side" reporting.

He's planning to introduce statistics courses to the journalism program, iirc.

Journalism has been the last bastion of math-free study, or one of them.

Catherine Johnson said...

I've come to believe in something like "mathematical literacy," almost speaking literally.

I want C. to be able to read a published, peer-reviewed article in the social sciences.

When I say "read," I mean that I want him to be able to read the statistics section.

I want the same thing for me.

I had a great moment the other day, vis a vis all this.

I was reading the procrastination guy: the one who came up with a formula to express procrastination.

A couple of years ago that formula would have stopped me cold. It's not that I couldn't have understood it at some level, once I thought about it.... it's that it would have 'left me cold.'

This time I read the formula, understood it pretty quickly (my only problem being trying to figure out how he'd defined a particular variable), AND "got something out of it."

The fact that his ideas had been expressed in formula form made them more accessible to me, not less.

That's progress.

Catherine Johnson said...

Meanwhile my school appears to have essentially no interest whatsoever in math or in mathematical literacy.

A few years ago the school board sent out an email saying that we were adopting Trailblazers because math has become language-based.

At the transition to high school meeting no one even mentioned math.

One parent asked whether kids are being well-prepared for the writing they'll have to do in college (answer: oh, yes, very well-prepared); the principal himself said kids aren't well prepared for college-level reading (gosh - we're just finding this out).

No mention of math.

From anyone.

Anonymous said...

--The "naturals" will be fine, I think. Kids with natural math talent can come from behind. I could be too sanguine about this, of course...


You are. The kids with natural math talent who are not utter prodigies DO NOT come from behind at a school like Harvard, MIT, Caltech in the math or sciences. They are completely outclassed by the Russians, Czechs, Estonians, Koreans, Japanese, Singaporeans, etc. In physics at MIT, the Russian kids were an order of magnitude ahead of the brightest American math kid in physics. In math, it was the same.

What American kids have going for them is an escape hatch: the "naturals" in math can more easily go into Investment Banking and other places that math skills are wanted than the foreigners, such as the Russians can, it seems (because of a lack of H1 B visas, maybe?)

But the only thing keeping more of the best math and science foreign kids out of MIT and the Ivies are restrictions on percentages of foreign students.

In grad school, it's almost a lost cause. There are virtually no Americans in the top programs, and white American men are almost unheard of. They have to have been the real prodigy (skipped high school, or college at 15, etc. and never fell off the train of perfection) to get there.