kitchen table math, the sequel: New winner in worst math program ever

Friday, May 25, 2012

New winner in worst math program ever

I try not to post judgments that sound hyperbolic. It makes me sound less measured, less reasonable, less reliable. But in this case, I'm making an exception.

ST Math is a math software/curriculum company that has a demo showing their unique "spatial temporal" approach to teaching algebra readiness. It looks like it is trying to build off the old PLATO programs from the 60s and the 70s--a tool like LOGO to help students play games that teach them concepts, they say.

It is the worst program I have ever seen.

Their opening slideshow explains that they will

"introduce math concepts without the use of language, numbers or symbols"

Here's the demo. see for yourself.

The examples are utterly counter intuitive and horribly ambiguous. They've failed to understand how their pictures could be interpreted in ways they did not foresee. Their examples teach students that algebra is a rigged game, where teachers tell you the right answers based on arbitrary hidden rules.

I only mention this because I already know a school that has adopted this program for next year.


Amy P said...

I watched the whole thing. It eventually made some sort of sense, but I'm a grownup who has made it successfully through traditional algebra. It wasn't that easy to figure out what they were getting at. How many ordinary children did they test this on before leaving beta, I wonder?

AmyP said...

Also, did they only use a little penguin? Because at least some of these kids are going to come away thinking that algebra only works for tiny animated penguins.

Lsquared said...

Hmm...when I watch the demo I'm seeing a rather good educational computer game. Thinking about how my children and children I know play computer games, I think they will be able to figure out what to do and may learn some math ideas along the way. From a math game design standpoint, my main concern is that when they get to using numbers, students have a multiple choice input rather than a numerical pad input, which makes it rather easy to game the system just by guessing.

Anyway, a computer game, good or bad, does not a curriculum make (is it a full curriculum?). I'd have to look at more than that to have a real opinion on it--the thing that makes a curriculum really bad is rarely visible from this sort of demo. A bad curriculum, from my perspective, can really only be identified by looking closely enough to tell what they don't do.

Cassandra Turner said...

I'm not seeing enough tie- in to the equation that the students are forming. It flicks by so fast.

Try this one: Shuttle Mission Math I was working through the levels on my ipad (there's an app, too) on a flight last month & the guy in the seat next to me was having a blast trying to figure the "puzzles" out.

I should warn you, it isn't as slick as the penguins, and there's a pretty annoying beeping noise, but overall, a very solid activity.

Crimson Wife said...

If schools want to teach solving easy linear equations in a more visual way, I really like Dr. Henry Borenson's program Hands-On Equations. It's a great intro to algebra that is accessible to elementary-aged students. Having worked through HOE has made the later Singapore Primary Math books a lot easier.

Amy P said...

My impression of a lot of educational software is that it's tough to find something with both 1) good graphics and game design and 2) sound educational content. It's too easy to think to yourself, "I've made some software with an adorable little penguin! My work here is done."

These kids are going to be wondering stuff like:

1) What is X, anyway?

2) Why do I need to help this stupid flightless bird?

I also agree that very few children are going to connect the problem with the equation.

Dennis Ashendorf said...

ST Math, affectionately called Jiji Math (the penguin's name), has been developed by the Mind Research Institute over more than a decade. It's whole point is to develop deep number sense as a supplementary activity for K-5. It isn't a curriculum.

Schools, who have adopted it OVER TIME, have seen incredible gains in test scores. It may soon make the "What Works" list as the sole program that actually delivers. I've used it slightly and the students have thrived on it, and I've used over 20 software programs over the past seven years.

It is expensive. Have a few bake sales, then buy it and use it.

Anonymous said...

Our school district is considering adopting ST Math as a supplemental program. I just went to a sales pitch given by a representative of the company. I am very seriously underwhelmed.

They are equating a rise in standardized test scores with a rise in understanding. But what is interesting is that every game starts with a pretest and ends with a post test that are in the format of a standardized test. Practice effect, anyone?

Also, they have not done studies that show how kids who have used the program do in higher math. To me, that is where the proof of efficacy lies.

They say that they have reduced every mathematical concept/operation to being able to move the penguin across the screen. Um, no (and thank goodness it is not so). For the higher level material, getting the problem right simply moves the the images from the problem out of the way so that JiJi can get across.

They also say that by the time children are finished with the program they will know what "invert and multiply" actually means, why it works for division of fractions. Except that as far as I can tell, they don't even teach division of fractions! Someone correct me if I am wrong.

My guess is that the reason that the thing "works" is that it is engaging to children and they actually practice doing math more than they would normally. No magic bullet here.

Dennis Ashendorf said...

Hmmm. Sadly, the sales rep did a poor job.

First, jiji really appeals. No language is used. English learners proceed well.

Second, it's pictorial (not concrete, like Singapore Math), but not overly abstract. However, the need to manipulate shapes to solve "puzzles" is very deep. In short, the initial main benefits include advanced spatial reasoning and a really good feel for fractions. To put it differently, with jiji, getting through the 4th-5th grade math transition occurs. This is shown in test scores from Santa Ana Unified (where jiji usage has slowly grown from a couple of schools to all 14-17 of them). For example, Madison has slowly gone from <4% adv in math to >40% adv in math on the state tests during the last 8 years. This is very big. This result alone is probably the greatest result in math curriculum in the United States.

Third, jiji has generally gone through 5th grade (although the 1st grade problems challenge my tenth graders - really, they are surprisingly hard.) The program is being extended through Algebra. For example, students manipulate trinomial quadratics, where they can actually "see" the effective of the first power "slope" term. Wild stuff. Real math.

Fourth, the Mind Research Institute did develop an Algebra Readiness program using a text. Frankly, it is too good. What I mean is that it required two textbooks to do the job correctly. Very few schools have the talent and the time to implement it. For example, the text spends an extraordinary time working the number line. Since I read the text years ago, I've used the number line almost everyday in reteaching basic math 1:1 to 16 year olds. It's effects are palpable.

Five, however as educators, belief matters. In my district, five schools have been using jiji for two years. Two of them claim almost miraculous benefits, another two like it, and a third claims no effect. In other words, it takes time to generate the data that later classes are succeeding far more and many admin/teachers look for immediate test results for the grades taking jiji. Like Singapore Math, the great benefits don't show in 1st or 2nd grade test results, but in the later grades, and this takes time to see and accept.

Anonymous said...

I'm sorry Mr. Ashendorf, but correlation is NOT causation, and any good skeptic would not buy your claims. (As any good scientist will tell you.) What else happened at Madison school in that 8 year period? Does your statistic measure the same cohort of students progressing through the grades, or just an average of all tested students?

Second, why does the number of textbooks equate a successful pre-algebra program? I know plenty of them that use only one and work just fine. You have great success using this while tutoring one-on-one. I'm sure you do. What else do you use during tutoring? What kinds of explanations?

Finally your argument in the last graph shifts. You state that teacher belief in success matters (confirmation bias), but then expound on that by claiming it will take time to see results. Huh?

I suggest that Mind Research Institute conducts a truly scientific rigorous study of their program, then we'll have something to talk about. Until then, it's all anecdote, speculation, and confirmation bias.

Dennis Ashendorf said...

Dear Anonymous,

Obviously I'm aware of your counters. The readers of this blog are not ordinary. I'm merely responding to posts.

If you want hard, publishable results, I'm told you'll soon have them at a level that will blow you away. What I've overheard in conversation and seen in presentations:

1. No math study has passed the "What Works" clearinghouse because of the demand for over 250 students and over a 0.2 standard deviation improvement ("real" significance) in externally judged results (eg state tests).

2. A study of the efficacy of ST Math is almost completed that shatters the "What Works" barriers. This research is not being done by ST Math.

Lastly, what belief means is "lets make this work." If you believe that jiji is junk, please don't be involved in implementing it! You'd just be a brick in the wall. Teaching is an active activity. Leave passive observation to observers. The school where jiji is failing is simply one that believes drill and more drill will generate better test results. At that school, the teachers belittle the program. What, you believe that jiji should still succeed; even if the implementers put it down and implement it haphazardly? Really?

AmyP said...

"The school where jiji is failing is simply one that believes drill and more drill will generate better test results."

Sorry, but it looks rote, repetitive and rather boring to me.

Amy P said...

I think that the lack of language is actually a problem. I initially didn't at all get what I was supposed to be doing in the demo. There are some confusing spots. Isn't there some likelihood that what is happening is that adults who are supervising are filling in the verbal blanks for kids, so that this program will yield very different results if the supervising adults are not as well-informed or active as those in the pilot programs?

I understand that this is your baby and you love it, and I think it does have potential, but based on the demo, it's not ready for prime time.

Dennis Ashendorf said...

Amy, your comments/questions are dead on. I'm in high school and I used ST Math demos to fill time when the Internet was down or substitutes were needed! My students always DEMANDED it afterword.

When a teacher or student first walks into jiji, it is "twisted." It appears not to be math, then you, or at least most of the students get into it, and it takes off. The lack of language is its strength - which is definitely counterintuitive! Actually, another version of ST Math includes music, but I haven't used it.

The irony is that jiji is not my baby, but the company, which I've never visited, is only one mile away. Also, the product was released 15 years ago and has undergone constant development since. It's a non-profit I believe and its annual gala is a high-tech heaven.

I have no friends at the Institute (which is ironical because I know may many math software companies), but following its growth as been fascinating. Both ST Math and ALEKS are UC Irvine spinoffs and both are totally different. Santa Ana which is the district next to mine receives a great deal of Title 1 funding and some schools started using it a decade ago. However, most teachers/admins are like you: 'this doesn't look real, etc..' Nobody blames you for being skeptical and it is costly. Santa Ana took over ten years to completely adopt it and it had/has 'easy' money relatively.

SteveH said...

Wow! Tetris Math!

What if the bridge is 3.732 units long?

Actually, it reminds me of bar charts in Singapore Math. I could come up with a better computer game based on that, where numbers in a word problem become bar segments that you try to stack to get bars to match in length. Students could give names to the bars and then define equations using the names. But then there are those pesky decimals.

Visualization and making practice more fun (thereby providing more parctice) are important. Whatever works. But, like bar charts, I don't see the understanding connection with more abstract math. What is the transition between problems that are easily visual to the ones that are completely abstract?

Some of the power of math comes from the fact that you don't have to visualize the problem. If you understand the definitions, identities, axioms, and proofs, you can just plug and chug. I let the math give me the understanding of the problem. I don't need some sort of visualization first.

As a supplement at the lower level, it might help, but the real problem is the shift from pie chart understandings of fractions to abstract rational expression understandings. There is little that is visual about that.

Why not use Singapore Math and include a computer program that does bar charts. It would be an integrated solution, not just a supplemental add-on. The only problem is that you might see the bar charts (or Jiji visualization) as what math is all about, rather than just a temporary bridge.

SteveH said...

"ST Math+Music:"

"Music has a mathematical architecture, and as young students learn music they are also learning to recognize patterns and symmetries, and experiencing and hearing the concepts of counting and fractions."

Gag me with a spoon. Let's dance all around a subject and not dive in. Heaven forbid that we expect kids to concentrate, think, and work hard. Thank goodness for real music lessons and competitive sports that teach kids the value of hard work and the idea that not everything has to be fun or natural.

Educators might find that these techniques produce better results that what they are currently doing (which isn't much), but, please, don't try to convince me that they are pedagogically "best practices" or what math or music is all about.

Expect mastery of a musical instrument. Expect mastery of math. Dive right in and get to work. Set high expectations. Ensure mastery.

Lsquared said...

I think that perhaps some of the more verbal of you are missing the math in the Jiji game. I think of myself as a more visual thinker, and the math just jumps out at me (so, like many things, it probably works better for some people than for others).

I'm most encouraged that it doesn't consider itself a curriculum. It looks very promising as a supplement.

Here are some of the things I like in the demo:
In the example of what happens if you pick a number that is too large, it shows the remaining amount as a block on the path. Now, what I like is that it shows the comparison as a size--a length on a number line. I'm not as attached to number lines as Wu is, but size is a really important number concept. You don't realize this until you have worked with a child who doesn't have this concept (a child for whom the guess 12-4=14 seems reasonable). Some children don't develop the relationship of size to value as early as others, and this can be a big problem.

I also notice in the problem that is equivalent to 2x=4, that the multiple choice answers are 1,2,4 and 8: 4 and 8 are good distractors. Now that's still not as challenging as having to type in your own number, but it's better than having random numbers out there to choose from.

In the later problems, like the one equivalent to 2x+2=8, you have multiple steps to calculate to figure out the solution. Since this is 5th grade, I'm expecting that children would not be doing this using formal algebra, but would be working backwards or guessing and checking their way to a solution. I don't think this is going to teach children algebra, but I do think it's going to teach mental math skills, and increase their comfort with multi-step calculations (something we often don't do in regular texts).

So, this doesn't look like a good replacement for a regular math program, but I would be quite happy with it as an addition to one.

Dennis Ashendorf said...


As a math person, I totally agree with your comments. However, as a teacher working with students where 100% of them need to complete math, most INVOLUNTARILY, the pure approach doesn't succeed.

See these programs for what they are: multi-representations to help a higher percentage of students master a certain level. The counter argument is really what do we do to help a FEW students achieve greatness, and what is the higher priority if we cannot practically do both.

In a way, this is what this blog struggles to find.

SteveH said...

"...the pure approach doesn't succeed."

I'm not talking about a "pure approach". I'm talking about the knowledge and skills one needs to keep all doors open. The transition from pie chart understandings of fractions to more abstract ones does not have to be mathematically pure or perfectly rigorous, but this transition should start before you get to pre-algebra. I see too large of a non-linear transition between 6th grade and 7th grade for many students.

ST Math might help in what I would call the pre-transition area to get kids closer to where they need to be, but I don't like the assumption that a visual or spatial approach to math is what math is all about.

I also think that because of so many bad K-6 math curricula, it's easy to be satisfied too quickly. I think kids can handle so much more on an absolute scale.

Math software has the potential for making huge differences, but I claim this is because schools will never do the job correctly themselves. They will never separate, allow acceleration, and ensure mastery of basic skills. However, schools won't mind if 21st century computer programs do the job. Teachers really don't mind direct instruction as long as someone else is doing the job. That's why many love the idea of flipping.

Dennis Ashendorf said...


You write too well. Since I agree with you, the answer may simply be self-paced mastery learning. Our age-based classrooms are a problem.

When I study blended classrooms, what I see are blended schools. Some courses are age-based, some aren't. MATH shouldn't be. Computer-assisted math allows this occur. We just need to use it.

Amy P said...

"I think that perhaps some of the more verbal of you are missing the math in the Jiji game."

Yeah, I am a very verbal, not very spatially oriented person, so it's not a good fit.