In control theory, there’s this wonderful treatment of a not so obvious reality. Without belaboring the math, it basically says that if you have a function that transforms something, you need to sample the output of the function and feedback an adjustment to the input in order to keep the whole system stable. Unstable systems fail (sometimes catastrophically).
So applying this theory to a classroom with a really good teacher you can picture something like; little empty minds in (the function input), lessons applied (the transformation), formative assessments (sampling), and individualized adjustments (the feedback loop) by the teacher.
My problem with NCLB is not with the testing but how the results are applied. Sticking with my analogy, the testing is nothing more than standardized (to a degree) sampling. The problem is that the measurement, the sample, is not applied to a feedback loop, its applied to the function. Results are not used to say, “Oh look, this child doesn’t understand common denominators yet. What adjustments can we make for this child?”
The results are used to take a big meat axe to the school district when a hundred tiny little scalpels would suffice. I work in a ‘failing’ district and I can tell you first hand that we are running an open loop control system that is wildly out of control. We have consultants pouring over us like hot fudge at an ice cream shop.
Their proscriptions always miss the mark and do not tweak the feedback loops. They are fussing about the scratches in the paint while the buildings burn. One of my favorite ‘solutions’ was the year of the bulletin board. We were, I kid you not, beat up for a year on the quality of our bulletin boards. We had bulletin board walkthroughs. We had bulletin boards so covered with crap that kids stopped looking at them. We had whole classroom blocks devoted to putting up a quality bulletin board.
Meanwhile, inside the classrooms, sixth graders that can’t add single digit numbers are being force fed a curriculum (the transform function) totally unsuited to where they are at by teachers who are not provided the sample data until 5 months after the testing for a class they no longer have.
It’s like trying to make course corrections for a rocket after it has crashed. It reminds of when I was a little kid playing at driving a car while sitting in a cardboard box holding an old bicycle wheel. The damn wheel never turned the box but it sure was fun!
I love this line:
We have consultants pouring over us like hot fudge at an ice cream shop.
Been there, done that!
This one's a keeper, too:
One of my favorite ‘solutions’ was the year of the bulletin board. We were, I kid you not, beat up for a year on the quality of our bulletin boards. We had bulletin board walkthroughs. We had bulletin boards so covered with crap that kids stopped looking at them. We had whole classroom blocks devoted to putting up a quality bulletin board.
I'm going to guess this person works in New York City.
Scratch that.
I'm going to hope this person works in New York City. I'd hate to think teachers elsewhere are having bulletin board walkthroughs.
update: Steve H on guess and check
Just saw Steve's riposte!
So, I guess a Bode Diagram is out of the question.
There is little feedback control in our schools. Testing results go straight onto report cards. It doesn't get fed back anywhere. That's because they think the input control is optimal already. Actually, they think the control starts in the home and report cards are the feedback to the parents. What the school does is part of the unchanging black box.
With NCLB, however, they are forced to apply a little bit of feedback to the system, but they can only use their favorite problem solving technique - guess and check.
Testing results go straight onto report cards....the control starts in the home and report cards are the feedback to the parents.
yup
see also:
The Logic of Failure by Dietrich Dorner
Brilliant!
ReplyDeleteWell stated. I believe every state has their own testing for NCLB (I'm sure someone will correct me if this is wrong). In Colorado we have CSAP and I have been "pro-CSAP" and I suppose pro NCLB. Also we have gotten some very interesting detailed results from our child's teacher that are not presented in the reports sent home. That said, I agree that the feedback mechanism is way too long and I can't argue with people who say that Colorado students are being "overtested" because that sure seems to be the case.
ReplyDeleteBut I want to get back to feedback for a second. Why does a teacher need a NCLB test to tell them their students can't add single digit numbers? I can tell you why:
They aren't grading the homework, or they're grading for effort not results.
Both of my daughters get insufficient feedback, if any at all from their math assignments.
It seems like the teachers have a schedule to keep, and they're not expecting enough from their students. If you are a teacher, ask yourself why you are asigning problems:
1. Is it busywork?
2. Is it practice makes perfect
3. Is it to provide you feedback?
4. Is it to provide the student feedback?
I’m the teacher (ex engineer) from the ice cream shop! My post was overly long and perhaps not succinct enough about the feedback loops.
ReplyDeleteOf course the teacher already knows about the individual need and is in control of each feedback equation. The problem is that they are not in control of the feedback system. So for example, if I know that Josue can’t add there is little that can be done by way of mitigating that deficiency.
Once you are covered with hot fudge sauce it’s really hard to move around. If you spend time on Josue with addition the hot fudge knows you haven’t produced his treatise on the quadratic equation called for in the increasingly sticky curriculum design.
The more you ‘fail’ the less you can do in the feedback loop. Remember the corrections are constantly being applied to the transform function, like a giant vacuum cleaner sucking the oxygen out of your room. While you work on the bulletin board, the kids are rotting in the fridge.
BTW: not from NYC but sprinkled with NYC consultants (Jimmys in New England :>}
Why does a teacher need a NCLB test to tell them their students can't add single digit numbers? I can tell you why:
ReplyDeleteThey aren't grading the homework, or they're grading for effort not results.
EXACTLY.
There is no reason on earth why a teacher should use the state tests for FORMATIVE assessment. The state tests are SUMMATIVE assessment, pure and simple.
Of course I'm sure that formative assessment, when it finally gets implemented in my district, won't be recognizable as formative assessment.
ReplyDeleteIt'll probably be authentic assessment & involve rubrics.
Also, it will cost an arm and a leg.
ReplyDeleteanonymous - your post was FANTASTIC!
ReplyDeleteI put it under Greatest Hits.
Gosh, I'm sorry to hear that bulletin board walkthroughs are happening in New England.
Once you are covered with hot fudge sauce it’s really hard to move around.
ReplyDeleteHave you read Dorner?
In his simulations people stopped acting -- stopped making decisions -- once things started going drastically wrong in the complex systems they were managing.
Is that what you see?
Do people stop acting (because it's hard to act)?
I would say people (teachers) stop reacting. Or more precisely they stop acting like teachers (tweaking the feedback loops) and fall into a sort of automatonic mode. It's kind of like shell shock. You yell, "Incoming!" (another insane correction to what isn't broken is being lobbed in by the district). Then you dive in your foxhole and do what you are told.
ReplyDeleteShaky districts with shaky budgets = shaky teachers, protecting their collective butts.
That's fascinating. I have to get some passages typed up and posted.
ReplyDeleteOf course, you're encouraging me in my view that we should eliminate the middleman (administrators, consultants, VENDORS) and move to a system in which teachers become full-fledged professionals hired directly by parents...
I think there are a lot of different ways that could work.
They aren't grading the homework, or they're grading for effort not results.
ReplyDeleteThis always seems so obvious to me. If the homework is graded, then what is the need for other formative assessment tools. It's all right there.
Just from looking at this over the years, it appears that it is going to get much worse before it gets any better.
As far as NCLB testing, I believe each state can either develop its own test or use their state one. I don't know whether they can choose another nationally recognized test or not.
They don't want to put in the expense or time of developing their own, so I think most just go with the state test as their NCLB test. This is the way I heard it, but I don't know for sure.
And the state tests, as we have discussed here before, are often another major part of the problem.
SusanS--charter member of the Save Your Own club.
What is an "an open loop control system."
ReplyDeleteI've just learned that there is such a thing as a "positive feedback" system - good Lord.
Now I'm wondering how many other hideous phenomenon I've never heard of are also running my life....
If the homework is graded, then what is the need for other formative assessment tools. It's all right there.
ReplyDeleteRIGHT.
That's what the super said when we told her math teachers never collect-and-correct homework. She said you can do either: collect-and-correct or perform formative assessments. (Naturally we've hired a consultant to write reports, conduct professional workshops, etc. so nothing has happened and teachers still aren't collecting-and-correcting.)
Of course, the fact that district math and science teachers can earn very high fees tutoring students on the side complicates matters.
As I understand the law, states can use whatever tests they choose.
ReplyDeleteKen's got the relevant language from the bill posted at his site.
When NCLB was passed, New York was testing only in the 4th and 8th grades, as national law required. (I forget which law put that requirement in place; I think it was done in response to Nation at Risk.)
The shift to yearly testing has been traumatic for schools, teachers, administrators, and for many parents & kids. (As usual, I should add that I'm pro-NCLB. But it hasn't been fun, and, as Anonymous Engineer points out, it's a meat cleaver approach.)
"What is an 'an open loop control system.'"
ReplyDeleteAn open loop control system is one in which you don't bother checking to see how you are doing. An example would be a robot arm, where to move to a certain position, one just moves a motor a certain number of revolutions, but doesn't check to see if maybe the gears are slipping.
Open loop systems tend to be simpler and cheaper than closed loop systems, but less reliable.
A closed loop system would be the robot arm with a sensor that fed back the arm position. In this case, even with slightly slipping gears you can get the arm to the correct location because the robot might issue one extra revolution command to make up for the slipping gears.
Poor schools and classrooms tend to run open-loop ... no one knows if the kids are learning what they are supposed to learn. Well run schools and classrooms run closed-loop by checking the kids for progress *AND THEN TAKING ACTION* (e.g. reteaching). Even better would be a school or classroom that was closed loop on the teaching itself. For example, if kids kept not learning, trying to figure out what to change so that the kids learned the first time around.
I don't get the impression that the second version is done formally in very many schools (or businesses for that matter).
Positive feedback means you do *more* as you get closer to what you want. I don't think any engineers design these things on purpose, but sometimes you get one anyway (I think ram-jets have a built in positive feedback ... without external controls, the faster they go the faster they want to go until they burn themselves up).
-Mark Roulo
"I don't get the impression that the second version is done formally in very many schools.."
ReplyDeleteBecause of NCLB, our state mandates that this process takes place. They send out (almost worthless) questionnaires to parents and students and they hold parent-teacher meetings to review the standardized test results.
For example, the committee might see that the standardized test scores for problem solving in math have gone down a few points. They will sit around and discuss the problem. It's down only a slight bit, so there is not much of a problem, right? Perhaps it's due to the small sample size (they get to sound rigorous). Well, in any case, the administration will tell the teachers in their next meeting to emphasize problem solving more. OK, next problem.
The flaw, as you can see, is that this is all relative. You can provide all of the feedback you want, but if the basic structure of the system is wrong, it will never be fixed. Standardized tests will never tell you that many more kids should be getting to algebra in 8th grade. In fact, it tells the school and many parents the opposite. (They conveniently ignore international tests.)
This reminds me of my son's Science Olympiad project. He can tweak things all he wants, but his basic vehicle design will never reach the required maximum distance.