One year I had to sit through endless hours of meetings, analyzing test results (Massachusetts MCAS). The principal, VP, and 10 or so teachers would sit around a table and look for anomolous results (pretty easy to find) where our school really sucked vs. the state average.
Then we would basically be asked to guess what went wrong. We had hundreds of square feet of 'what went wrong' guesses posted all over the conference room. One particular question (with a horrible failure rate) has always stuck with me. It was something like "How many quarter pound hamburgers can you make from 3 pounds of hamburg?"
Our guess? Kids are having problems dividing whole numbers by fractions. I wasn't comfortable with the guess since it was my class that bombed this question and I was very confident that my kids knew how to do such a simple problem. As luck would have it, one of my really bright students was passing the room and I called her in to ask her about the question.
She floored us all. "You can't make hamburg from hamburg." Huh? It seems that in Puerto Rico (school is > 85% Hispanic population) the source material for hamburg is called ground meat. In our New England cacoon we make hamburger patties from hamburg. The real story is that kids were totally baffled by the seemingly bizarre question.
Please don't think I've told this little story to push on test bias. That could be a whole 'nother blog. No, I bring it up as an example of the futility of analyzing test results, especially when there's no kids in the room. You really have no idea what went wrong when kids bomb a question.
Regards from the ice cream shop.
Meanwhile, I'm sitting here thinking, "Hamburg"?
What is "hamburg"?
In the Midwest, we make hamburger out of hamburger. Somehow I managed to live in Massachusetts for 3 years of my life and never pick up on the fact that, apparently, folks in Massachusetts make hamburger out of hamburg.
From time to time I try to think exactly what kind of book could or should be spun off from Kitchen Table Math.
I'm pretty sure it's a book of stories from inside the black box.
Or the ice cream shop, as the case may be.
I live on the west coast and I've always though hamburgers were made from ground beef.
ReplyDeleteI assumed "hamburg" was a typo. Other than the city in Germany, I have never heard of such a thing.
ReplyDeleteHamburgers are made from ground beef. when I make them, I add eggs and breadcrumbs and minced onions to the beef, too.
But I assumed the question was:
"how many 1/4 lb hamburgers can you make from 3 pounds of hamburger" because asking "how mnay 1/4 lb hamburgers can you make from 3 pounds of ground beef" is an even DUMBER question--since it requires knowing the rest of the ingredients in the recipe.
So teachers really have no idea what their students are thinking do they?
I'm a vegetarian, so I've never actually made a hamburger but here in the south, hamburger patties are made from ground beef, not hamburg or hamburger. I would have a hard time answering the question because in my mind, a hamburger is a finished product (as opposed to the uncooked hamburger patty). You can start with a 1/4 lb. patty, but your finished product weighs less - how much less is dependent on how fatty the meat is. I couldn't answer the question.
ReplyDeleteThis post and the comments illustrate one of the ways word problems trip up my fifth grader. In the hamburger example, she might not be able to infer that you must ignore the fat and the ingredients that might be added to the ground beef before cooking the patties. On the other hand, my older kid would know what they’re getting at, and answer correctly.
ReplyDeleteThe school tells me that my fifth grader has reading comprehension issues, and I would agree. However, she also falls victim to imprecisely written problems, which I see more often that I would like to. And sometimes the content is just not appropriately matched with the age of the student. Maybe they’re pushing too hard to make math “real world”.
She recently had a word problem that asked about how many record albums a man had in his collection. That was funny.
When you program, you have to learn how to really understand errors. You have to examine the error and carefully work backwards. You have to study the code line-by-line. I had too many students who threw programs together and then tried to use guess and check (edit - build - run ad nauseum) to fix it. They never looked at their code carefully again. It's difficult to work backwards from the hard facts. Guessing is like trying to win the lottery.
ReplyDeleteThe comments so far, make my case. I may have bungled the hamburg/hamburger/hamburglar nuance but that wasn't really the point was it?
ReplyDeleteThe very fact that so many misinterpretations are possible from a very simple question (by adults no less) demonstrates the futility of analyzing test results without the kids in the room (which was the point of the post).
Now imagine the insanity of driving the 'analysis' into some kind of action plan to improve your school's performance.
IFR with no instruments, in a white out, while low on fuel.
"How many quarter pound hamburgers can you make from 3 pounds of hamburg?"
ReplyDeleteHow many quarter pound queebles can you make from 3 pounds of queeb?
I guess after years of solving word problems with rambutens, fishballs, and satay, mystery units, and bizarre questions have not been an issue for my kids.
The solution may be not purge the curriculum of all ambiguous and vaguely written word problems but to specifically teach the kids how to handle it when it comes up.
Exactly! But in bizarro land you increase the pressure to work on fraction computations.
ReplyDeleteELL kids are very literal with language. If you are learning a language your freedom for nuance analysis is very restricted. So a far more effective 'feedback' for my kids would be word problem decoding strategies. Remember though, in the original post the (expert) teachers weren't even close to this kind of tweak to their learning.
When analyzing test results, how granular is the data that the teachers see?
ReplyDeleteDo you get data by question? Do you know which wrong answers were picked? Or do you just know what that the answer was wrong?
From the released items I've seen, I'm struck by how imprecise and confusing the questions are for younger kids. I read one question over and over unable to figure out what it was trying to test. For that sort of question, will the data tell you anything?
In Massachusetts there are three types of questions; multiple choice, short answer (the written answer is the only thing graded), and open response (every scribble gets analyzed).
ReplyDeleteThe results I've has access to as a teacher are;multiple choice (we get a list of percentage answered for each question and each possible choice, short answer (we get percent write/wrong), open response (we get percentages who got 0-4 score)
We don't get to see it by student. Everything is aggregated. I'm not sure that you can't get finer (down to the student) just haven't seen it myself. The test is in mid May and the results come out the following (late)October.
You get all this really neat data that is old and irrelevant to your current class. In my school there is such high turnover it's probably not even relevant for the teacher who has those kids when the results come out since they will likely have a very different cohort of kids.
I grew up in Massachusetts, and we don't speak of "hamburg". We don't even speak of "hamburger" when we're talking of ground beef. We say, logically enough, "ground beef."
ReplyDeleteSo, I don't know who "Anonymous Engineer" is, but his account doesn't fit Massachusetts.