in the Times
today:Disasters teach more than successes.
While that idea may sound paradoxical, it is widely accepted among engineers. They say grim lessons arise because the reasons for triumph in matters of technology are often arbitrary and invisible, whereas the cause of a particular failure can frequently be uncovered, documented and reworked to make improvements.
Disaster, in short, can become a spur to innovation.
There is no question that the trial-and-error process of building machines and industries has, over the centuries, resulted in the loss of much blood and many thousands of lives. It is not that failure is desirable, or that anyone hopes for or aims for a disaster. But failures, sometimes appalling, are inevitable, and given this fact, engineers say it pays to make good use of them to prevent future mistakes.
The result is that the technological feats that define the modern world are sometimes the result of events that some might wish to forget.
“It’s a great source of knowledge — and humbling, too — sometimes that’s necessary,” said Henry Petroski, a historian of engineering at Duke University and author of “Success Through Failure,” a 2006 book.
Taking Lessons From What Went Wrong
By WILLIAM J. BROAD
Published: July 19, 2010
Obviously, engineers learn from engineering failures.
ReplyDeleteMy question is whether the strong form of this statement is true -- ?
It's not a complete statement.
ReplyDeleteDisasters teach more *to whom*? Everyone? Lay people? Experts?
Politicians?
A better phrasing is:
"a willing participant learns more from his own failure than his own success." Not exactly the same.
Even personal disaster that's directly attributable to that person's choices doesn't teach most such persons anything. To learn requires a willingness to accept responsibility.
oh sorry - should have been more clear
ReplyDeletethe claim is that engineers learn more from failures than from successes, that failure moves the field forward more powerfully than other factors --
Eric H. Brown, a British engineer who developed aircraft during World War II and afterward taught at Imperial College London, candidly described the predicament. In a 1967 book, he called structural engineering “the art of molding materials we do not really understand into shapes we cannot really analyze, so as to withstand forces we cannot really assess, in such a way that the public does not really suspect.”
Among other things, Dr. Brown taught failure analysis.
Dr. Petroski, at Duke, writing in “Success Through Failure,” noted the innovative corollary. Failures, he said, “always teach us more than the successes about the design of things. And thus the failures often lead to redesigns — to new, improved things.”
I may be overstating his thesis... but I'm curious.
I think of scientific progress -- and other kinds of progress -- as coming from 'discoveries,' 'creativity,' 'insight'....
I don't think of progress as coming significantly from failure & certainly not from catastrophe.
Dr. Brown is speaking as an expert. Experts see the world differently than novices.
ReplyDeleteExperts can learn from failure because they know enough to discern which lessons are important. Also, experts can design interesting experiments, whereas novices can't.
People don't usually analyze the reasons for success, but they do analyze the reasons for failure. The company I work for has a thing we do called "root cause analysis" and "the five whys" where we dig and dig and dig to try to find the real reason for failure. We don't usually do this for successes. Maybe we should.
ReplyDelete- The customer reported a primary key violation on table FOO.
- Why is the customer getting a PK violation on FOO? Because the software is trying to insert two records for the same thing.
- Why is the software trying to insert duplicate records? Because two threads are trying to do the same work.
- Why are two threads trying to do the same work? Because there's a bug that wasn't caught.
- Why wasn't the bug caught before release? Because we didn't test the multi-thread scenario.
- Why didn't we test the multi-thread scenario? We were going to test it, but we ran out of time. (Or, you could branch off in a different direction here and go with "Because we didn't know the software needed to support multiple threads of execution.")
- Why did we run out of time? Because "just a small feature request" was added to the schedule after we did the planning and estimation.
- And so on...
Structural engineering is also particularly impacted by failures. When the Cypress structure in the San Francisco Bay Area collapsed in 1989, they learned that there was a kind of earthquake motion that they hadn't known about before, and that they hadn't designed freeways to withstand (the freeway was a two tier stack and basically bounced off of its pins and pancaked). When a building or bridge or whatever fails, you can learn things about design that you didn't know before.
ReplyDeleteMy grandmother always said that failure was valuable in scientific experiment because it was important to know what wouldn't work. Insight often comes from thinking through causes for failure.
However, as RMD says, this is for an expert who has the knowledge base to think these things through!
Usually, it's a failure to ask the Whys about the successes that leads to having to ask whys about the failures. You assumed things were the same, or that they worked for reasons you understood, and that is why you didn't probe deep enough to find a problem.
ReplyDeleteIn engineering, the failures are usually only visible after the fact. This is why you need root cause analysis, because you've gone several steps/days/years away from that root cause by the time the problem appeared. If a bridge were falling down immediately because you used the wrong width guesset plate, you wouldn't need root cause analysis.
Novices learn from successes. Most textbooks teach novices the right way to engineer something.That's because you need a base of knowledge in how something should be before you can possibly learn how to investigate how something went wrong. It would be unwise to introduce novices to the wrong methods, because how would they even keep track at that stage what were right ways, and what were wrong ways?
When debugging a program, a correct result is nice, but it's not as useful as a well-defined (and reproducible) error. Errors are like gold. You can track them down and fix the problems. My son's piano teacher told him once that mistakes are opportunities.
ReplyDeleteA few thoughts . . .
ReplyDelete1. Unfortunately, supporters of constructivism use this idea to teach by, well, not teaching . . . but instead let students "experiment" with concepts. Some knowledge can be imparted without additional experimentation because the experiments were done long ago and the knowledge that survived works and simply needs to be learned.
2. The article proposes the ideas that disasters can teach. This might be true. However, it's much simpler and less costly to use existing knowledge, with experimentation and other analysis techniques to determine whether an idea will work.
3. Why wait for a disaster to determine if something doesn't work? Use small experiments before disaster strikes to determine if your ideas are correct. Small piano mistakes, coding errors, etc. should be used to guide efforts. I wouldn't suggest to beginners that they wait until they are in front of an audience to determine if their piano technique is correct.
"I wouldn't suggest to beginners that they wait until they are in front of an audience to determine if their piano technique is correct."
ReplyDeleteThat spin is far from my point. I also understand that guess and check is not a proper technique for learning. There will be plenty of mistakes even with a lot of direct teaching.
I'm not suggesting that students never learn from errors--I'm suggesting that a student can't *on their own* recognize an error until they've got a grasp of what a non error is.
ReplyDelete--When debugging a program, a correct result is nice, but it's not as useful as a well-defined (and reproducible) error.
This is an odd way to phrase this. If your program works, why are you debugging? If a novice doesn't first learn what a correct program is, they won't know what the error is.
A reproducible error is mostly useless to a novice. That's why they do guess and check--they have no intuition or prior experience to guide them to their error.
-- Errors are like gold. You can track them down and fix the problems. My son's piano teacher told him once that mistakes are opportunities.
YES, they are, because he already is beyond being a novice. The movement to mastery requires learning from mistakes.
I'm not trying to suggest this is a linear progression. I'm just trying to clarify that assuming a novice to could from a mistake without a teacher to guide them is constructivist hogwash.
"That spin is far from my point."
ReplyDeleteSorry about that .. . I thought your point was spot on . . . small mistakes do have a role in learning, particularly in your example.
The original article, however, went too far in asserting that we wait until disasters to start our learning.
"I'm suggesting that a student can't *on their own* recognize an error until they've got a grasp of what a non error is."
ReplyDeleteI wasn't making a reference to any of your comments about novice versus expert. My comment just happened to appear after yours.
Sorry, sorry.
ReplyDeleteI think RMD's didn't appear til after I'd posted either. Ah well, blogger has its quirks.
My son's piano teacher told him once that mistakes are opportunities.
ReplyDeleteI haven't read the whole thread yet - but, gosh, that strikes me as soooo wrong!
Unless your son is much more advanced than I'm thinking.
The distinction various folks are making between novices learning from success versus experts learning from mistakes strikes me as extremely valuable and probably true.
fyi: (and I should get this posted up front) I am coming at this question from a couple of personal experiences idiosyncratic to me.
One is the fact that I have autistic children &, a few years back, discovered the concept of "errorless learning." I didn't learn as much about errorless learning as I would have liked, but given what I do know about it, errorless learning is used for the 'ultimate novices': people who have lost abilities due to stroke.
One study showed that people with stroke learned how to use a calculator (I think it was a calculator - or maybe a PDA) only when they were not allowed to make any mistakes at all. (I may be overstating the finding - but that was the jist.)
The second personal experience shaping my thinking is the two years I've spent learning to play tennis. I've made a lot of progress, but I'm nowhere with my serve -- and just a couple of weeks ago I basically refused to carry on trying to learn it in the way my teacher was teaching it because I'm certain I'm building up a big, strong habit memory of serve-as-dinking-the-ball-over-the-net.
(I recently read Daniel Coyle's The Talent Code, which explains myelin and learning....skill means lots of myeline wrapped around the 'skill circuits' in your brain.)
So my teacher came up with a different way of having me practice, and I seem to be suddenly making some progress.
That, too, is an example of a novice not being able to learn from failure, of course.
"My son's piano teacher told him once that mistakes are opportunities.
ReplyDeleteI haven't read the whole thread yet - but, gosh, that strikes me as soooo wrong!"
I think there are many ways to look at this. The first is to use mistakes as a basis for a constructivist-type approach to learning. That is beneath consideration.
"Disasters teach more than successes"
Assuming that a success is the result of knowledge, skills, and planning, then that is based on previous teaching, so it won't teach you anything new. A mistake, however, can be the basis for new learning or advancement. This doesn't mean that a mistake-driven process is a good teaching process. You could say that it indicates a failure of proper teaching.
However, there is a limit to the benefit of teaching. Students have to practice and there will be mistakes. A good teacher will adapt to the needs of the student, but students also need to learn to teach themselves. A process where mistakes are analyzed can be very useful. In programming, that involves analyzing the exact error and carefully working backwards. It involves the ability to concentrate and understand exactly what the program is doing, line by line. It doesn't mean guess, rebuild and check.
For my son, it means that he can't just keep going over the same notes. He has to analyze the fingering and the timing, and use a variety of tools (taught by his teacher) to fix the problem. These are skills like slow practice, rhythm practice, and separate hand practice. These are not tools just for the expert. A novice can properly identify mistakes and try to solve them.
There is the problem in music where a teacher can be too much of a pedagogue. That can deter finding new approaches to problems, like using the thumb on a black key. There are also more advanced problems, like phrasing, breathing, and control which require more expertise and experience before pianists can begin fix the mistakes themselves. Some learning might benefit from a zero-error approach, but that is unrealistic for most types of learning.
Behind all of this, however, should be the idea that you really would like to avoid errors in the first place. One of the things you can learn from a disaster is that it should have been avoided in the first place.
--He has to analyze the fingering and the timing, and use a variety of tools (taught by his teacher) to fix the problem. These are skills like slow practice, rhythm practice, and separate hand practice. These are not tools just for the expert. A novice can properly identify mistakes and try to solve them.
ReplyDeleteThese are not really skills for the novice, either. They are for someone in between. A novice isn't analyzing his fingering, because he wouldn't know what to analyze it *for*. The teacher is specifically teaching techniques to help break down that analysis into smaller steps, yes: if you're making a mistake, play that measure slower. But if we've already moved to where a student is supposed to know that their problem is fingering, then that's above novice.
--Mistakes are opportunities
I think the piano teacher example is actually focusing on a different issue, even though the words are the same. Good teachers who work with children try to get away from talking about mistakes negatively because of the confidence issue. They want to teach children *not to be afraid* of trying something you might make a mistake at.
This goes to the issue of smarts/talent vs. hard work. You want to help the student to believe that hard work is more important than talent. That means teaching a student to stretch themselves, to take a risk, not just to only do what they'll do perfectly. That means you have to teach them not to be afraid of making a mistake. So mistakes must become opportunities. It must be safe for them to make mistakes or they will not improve.
SteveH is also talking about making the transition from novice to expert. That's a hard thing, but is what (ideally) science students start to do with undergraduate research or upper division coursework, or what musicians do as they progress. Yeah, eventually we want students to be able to learn on their own and identify their own mistakes, but it is a process. The problem I see with some threads of educational theory is really that they don't distinguish between novices and experts, but we also need to recognize intermediate zones (journeymen, maybe?) where you can learn to analyze your own practice.
ReplyDelete"It must be safe for them to make mistakes or they will not improve."
ReplyDeleteSkills for dealing with mistakes can be learned at all levels. It's not just about not being afraid of making mistakes.