Posts
Saturday, April 19, 2014
Retrieval practice, the bad news
I was thinking the incredible results people find with retrieval practice sounded too good to be true, and now I'm worried they are:
Failure of further learning: the limits of repeated study and retrieval practice
Wish to heck I could pull the study. All I've got for the moment is the abstract:
I see that "interactive exploration" didn't help students remember anything beyond what they recalled during their first retrieval practice. (Another nail in the coffin of hands-on, guide-on-the-side activities as the cure for limited remembering and understanding.)
I see that difficult reading passages without titles (!) helped. That finding is especially striking in light of the SQ3R approach to reading comprehension, which involves paying attention to titles. (SQ3R isn't necessarily incompatible with this finding, but still...)
It sounds like the last sentence refers to a complete, or near-complete, re-creation of the original learning-and-quizzing episode.
Did students re-read and quiz themselves, both in the same "session"?
I should go read the summary at Jung's Biology Blog.
I'm probably going to enjoy it:
The last sentence has to do with word-for-word memorization, which does improve with repeated retrieval practice.
As for the rest of it, in fact, recall does improve over time, but not by nearly as much as one would hope.
"Elaborative" study techniques -- underlining, annotating, diagramming (Make It Stick highly recommends elaboration) -- had no effect at all.
Difficult texts without titles offer no advantage I can see from the summary: students recalled less during their first retrieval practice and then, over time, continued to recall more until they reached the level of students who had read a clearly-written text with a title. Lots more pain, no gain.
At this point, I don't think this study tells me too much. The authors' theory - which I do find quite interesting - is that a student's mental model blocks further learning. Specifically, the student's mental model blocks learning of content that didn't make it into the model.
That strikes me as highly likely; I've experience a "gist" effect myself, and I think "gist effects" are a major problem in any kind of reform effort.
However, fields of study are different from a book you are trying to remember, which is what the students in this study were trying to do.
When you study a field, one of the things you are learning is the field's organization and categories.
New content gets slotted into pre-existing categories. That's why the more you know about a subject, the easier it is to learn new aspects of that subject. You have a mental 'gist' of the subject that is 'hungry' for new content, or at least wide-open to it.
Failure of further learning: the limits of repeated study and retrieval practice
Wish to heck I could pull the study. All I've got for the moment is the abstract:
AbstractUnfortunately, I don't understand the final sentence in the abstract.
Previous research has shown that little benefit is achieved through spaced study and recall of text passages after the first recall attempt, an effect that we term the failure‐of‐further‐learning. We hypothesized that the effect occurs because a situation model of the text's gist is formed when the text is first comprehended and is consolidated when recalled; it dominates later recall after verbatim memories of more recent study episodes have been lost. Experiments 1 and 2 attempted to circumvent the effect by varying the activities of participants and requiring interactive exploration. In both experiments, recall after four, weekly sessions showed little benefit beyond performance on the first recall. Experiment 3 interfered with the formation of an immediate situation model by introducing passages that were hard to comprehend without a title. Performance improved substantially across four sessions when titles were not supplied, but the standard effect was replicated when titles were given. Experiment 4 made verbatim memories available by incorporating all re‐presentations and tests into one session; as predicted, recall improved over successive tests.
Failure of further learning: Activities, structure, and meaning
Catherine O. Fritz,
Peter E. Morris,
Barbara Reid,
Roya Aghdassi,
Claire E. Naven
I see that "interactive exploration" didn't help students remember anything beyond what they recalled during their first retrieval practice. (Another nail in the coffin of hands-on, guide-on-the-side activities as the cure for limited remembering and understanding.)
I see that difficult reading passages without titles (!) helped. That finding is especially striking in light of the SQ3R approach to reading comprehension, which involves paying attention to titles. (SQ3R isn't necessarily incompatible with this finding, but still...)
It sounds like the last sentence refers to a complete, or near-complete, re-creation of the original learning-and-quizzing episode.
Did students re-read and quiz themselves, both in the same "session"?
I should go read the summary at Jung's Biology Blog.
I'm probably going to enjoy it:
In other experiments, Fritz et al. show that FOFL occurs even when ideas are presented as itemized lists on Powerpoint slides (why am I not surprised).Update: OK, I should have read first, posted later.
The last sentence has to do with word-for-word memorization, which does improve with repeated retrieval practice.
As for the rest of it, in fact, recall does improve over time, but not by nearly as much as one would hope.
"Elaborative" study techniques -- underlining, annotating, diagramming (Make It Stick highly recommends elaboration) -- had no effect at all.
Difficult texts without titles offer no advantage I can see from the summary: students recalled less during their first retrieval practice and then, over time, continued to recall more until they reached the level of students who had read a clearly-written text with a title. Lots more pain, no gain.
At this point, I don't think this study tells me too much. The authors' theory - which I do find quite interesting - is that a student's mental model blocks further learning. Specifically, the student's mental model blocks learning of content that didn't make it into the model.
That strikes me as highly likely; I've experience a "gist" effect myself, and I think "gist effects" are a major problem in any kind of reform effort.
However, fields of study are different from a book you are trying to remember, which is what the students in this study were trying to do.
When you study a field, one of the things you are learning is the field's organization and categories.
New content gets slotted into pre-existing categories. That's why the more you know about a subject, the easier it is to learn new aspects of that subject. You have a mental 'gist' of the subject that is 'hungry' for new content, or at least wide-open to it.
Make It Stick - is 3 the magic number?
I'm reading Brown, Roediger, and McDaniel's Make It Stick in a mixed state of trepidation and fascination.
Trepidation because the wording is so incautious and, often, so imprecise that I think it's entirely possible we'll see Make It Stick touted as justification for bad teaching of every stripe.
e.g.: the preface spends a great deal of time denigrating "drill and kill," which the writers take to mean massed practice. Also, "easy" lessons are slammed on grounds that learning should be hard. (So drill and kill is too easy?) And at one point the text baldy avers, sans footnote, that "people do have multiple intelligences" while in the same breath asserting that there's no evidence learning styles have anything to do with anything. So instead of worrying about our learning styles we should all use all of our multiple intelligences all the time because you learn better when you "go wide," drawing on all of your aptitudes and resourcefulness, than when you limit instruction or experience to the style you find most amenable.
What can any of this possibly mean?
Hard is good but hard drill-and-kill is bad .... learning styles are meaningless but we should deploy all of our "intelligences" all of the time (dancing in math? math-ing in dance?) ...
And does the exhortation to "go wide" mean multisensory programs are always to be preferred? I'm completely open to that possibility myself, but there's no listing for "multisensory" in the index, so who knows?
But that's the preface.
The first chapter, on retrieval practice, is riveting.
Retrieval practice, or the testing effect, refers to the finding that taking a test -- any kind of test, in-class or a quiz you give yourself -- increases your memory of the material you are trying to learn. In other words, taking a test or a quiz is a form of practice in which you practice remembering.
What's more, simple retrieval practice is probably superior to the kind of "active," "higher-order" learning students are purported to do via concept mapping. It also appears likely that retrieval practice produces knowledge that is readily transferred to new contexts and to problem solving.
In short, memorization makes you smart. (That's not how Brown, Roediger, & McDaniel put it.)
What is shocking to me is the tiny amount of retrieval practice the middle-school students in Roediger's studies needed in order to recall the material they had covered in class: just 3 "low-stakes" (ungraded) clicker quizzes in all. One quiz at the beginning of the class (on material they were supposed to have read the night before), one quiz at the end of the class (after the teacher's lecture on the same material) and one before the unit test a few weeks later. Students scored a full letter-grade higher on quizzed than on un-quizzed material.
When Roediger expanded the study to 8th-grade science, students scored an average of 92% on quizzed material, 79% on un-quizzed material. They still remembered the quizzed material eight months later, for the final.
After 3 ungraded quizzes no one had to study for.
Is 3 the magic number?
Here are Rawson & Dunlosky on "How Much Is Enough?"
Clicker quizzes work.
I need a clicker.
Trepidation because the wording is so incautious and, often, so imprecise that I think it's entirely possible we'll see Make It Stick touted as justification for bad teaching of every stripe.
e.g.: the preface spends a great deal of time denigrating "drill and kill," which the writers take to mean massed practice. Also, "easy" lessons are slammed on grounds that learning should be hard. (So drill and kill is too easy?) And at one point the text baldy avers, sans footnote, that "people do have multiple intelligences" while in the same breath asserting that there's no evidence learning styles have anything to do with anything. So instead of worrying about our learning styles we should all use all of our multiple intelligences all the time because you learn better when you "go wide," drawing on all of your aptitudes and resourcefulness, than when you limit instruction or experience to the style you find most amenable.
What can any of this possibly mean?
Hard is good but hard drill-and-kill is bad .... learning styles are meaningless but we should deploy all of our "intelligences" all of the time (dancing in math? math-ing in dance?) ...
And does the exhortation to "go wide" mean multisensory programs are always to be preferred? I'm completely open to that possibility myself, but there's no listing for "multisensory" in the index, so who knows?
But that's the preface.
The first chapter, on retrieval practice, is riveting.
Retrieval practice, or the testing effect, refers to the finding that taking a test -- any kind of test, in-class or a quiz you give yourself -- increases your memory of the material you are trying to learn. In other words, taking a test or a quiz is a form of practice in which you practice remembering.
What's more, simple retrieval practice is probably superior to the kind of "active," "higher-order" learning students are purported to do via concept mapping. It also appears likely that retrieval practice produces knowledge that is readily transferred to new contexts and to problem solving.
In short, memorization makes you smart. (That's not how Brown, Roediger, & McDaniel put it.)
What is shocking to me is the tiny amount of retrieval practice the middle-school students in Roediger's studies needed in order to recall the material they had covered in class: just 3 "low-stakes" (ungraded) clicker quizzes in all. One quiz at the beginning of the class (on material they were supposed to have read the night before), one quiz at the end of the class (after the teacher's lecture on the same material) and one before the unit test a few weeks later. Students scored a full letter-grade higher on quizzed than on un-quizzed material.
When Roediger expanded the study to 8th-grade science, students scored an average of 92% on quizzed material, 79% on un-quizzed material. They still remembered the quizzed material eight months later, for the final.
After 3 ungraded quizzes no one had to study for.
Is 3 the magic number?
Here are Rawson & Dunlosky on "How Much Is Enough?"
The literature on testing effects is vast but supports surprisingly few prescriptive conclusions for how to schedule practice to achieve both durable and efficient learning. Key limitations are that few studies have examined the effects of initial learning criterion or the effects of relearning, and no prior research has examined the combined effects of these 2 factors. Across 3 experiments, 533 students learned conceptual material via retrieval practice with restudy. Items were practiced until they were correctly recalled from 1 to 4 times during an initial learning session and were then practiced again to 1 correct recall in 1–5 subsequent relearning sessions (across experiments, more than 100,000 short-answer recall responses were collected and hand-scored). Durability was measured by cued recall and rate of relearning 1–4 months after practice, and efficiency was measured by total practice trials across sessions. A consistent qualitative pattern emerged: The effects of initial learning criterion and relearning were subadditive, such that the effects of initial learning criterion were strong prior to relearning but then diminished as relearning increased. Relearning had pronounced effects on long-term retention with a relatively minimal cost in terms of additional practice trials. On the basis of the overall patterns of durability and efficiency, our prescriptive conclusion for students is to practice recalling concepts to an initial criterion of 3 correct recalls and then to relearn them 3 times at widely spaced intervals.The chapter also says that short answer and essay tests are probably superior to flash cards and multiple choice, but flash cards and multiple choice produce superior retention, too.
Clicker quizzes work.
I need a clicker.
Just imagine
Utopia can't wait
(As the parent of two autistic kids I've spent years of my life listening to Barney sing "Just Imagine...")
(As the parent of two autistic kids I've spent years of my life listening to Barney sing "Just Imagine...")
College Board takes leave of its senses
In a new type of advanced government class at Seattle’s Garfield High, the students rarely sit quietly taking notes while their teacher stands and lectures.Sit and get.
Instead, they debate each other. They write legislation. They run for president in mock elections and pretend they’re lawyers arguing cases before the U.S. Supreme Court.
They sometimes even stand up and holler, as Sanai Anang did recently, playing a member of a Virginia-based group that lobbies for strict immigration controls.
In a simulated public hearing, Anang, who loves to ham it up, jumped to his feet without being recognized and declared, in a mangled Southern accent, “Ee-lee-gals come over and take our jobs. They don’t bee-long here.”
His classmates and teacher Jerry Neufeld-Kaiser cracked up.
They are all part of a teaching experiment that began six years ago in the Bellevue School District when a handful of frustrated government teachers teamed up with University of Washington researchers and turned the usual Advanced Placement curriculum inside out.
[snip]
The [College Board] is watching the teaching experiment carefully, interested in its promising results. In 2012 the board invited project leaders to its A.P. conference to present their ideas to A.P. teachers from across the nation.
It’s important that students gain an in-depth understanding of a subject, said Auditi Chakravarty, an A.P. program vice president. “And that requires more than the passive sit-and-get kind of learning.”
That's a new one on me.
And when did high-school kids making fun of southern accents (and southern people) become a hands-on learning activity?
We are a long way from debate club.
Cliff Mass has a good comment in the comments thread.
Princeton Concludes that America is basically an oligarchy.
Princeton Concludes What Kind of Government America Really Has, and It's Not a Democracy
April 16, 2014
A new scientific study from Princeton researchers Martin Gilens and Benjamin I. Page has finally put some science behind the recently popular argument that the United States isn't a democracy any more. And they've found that in fact, America is basically an oligarchy.
This is a familiar topic to those of us who oppose the Common Core Standards Initiative. I've been scratching my head, literally for years, trying to understand how Common Core took hold, quietly and quickly, in America's representative republic. In my state of Missouri, it became clear to me that the "initiative" capitalized fully on the top-down governance structure in our education policies and their implementation. I believe that Missouri state statutes were violated in the adoption process, but unfortunately, there was no oversight mechanism in place to stop Common Core from the onset.
After researching the opposition to the Common Core Standards Initiative throughout the country for a number of years, there is no doubt in my mind that our American oligarchs were well aware that state legislatures were not equipped to investigate and address adoption of Common Core Standards in 2009-2010.
(posted by concerned)
April 16, 2014
A new scientific study from Princeton researchers Martin Gilens and Benjamin I. Page has finally put some science behind the recently popular argument that the United States isn't a democracy any more. And they've found that in fact, America is basically an oligarchy.
This is a familiar topic to those of us who oppose the Common Core Standards Initiative. I've been scratching my head, literally for years, trying to understand how Common Core took hold, quietly and quickly, in America's representative republic. In my state of Missouri, it became clear to me that the "initiative" capitalized fully on the top-down governance structure in our education policies and their implementation. I believe that Missouri state statutes were violated in the adoption process, but unfortunately, there was no oversight mechanism in place to stop Common Core from the onset.
After researching the opposition to the Common Core Standards Initiative throughout the country for a number of years, there is no doubt in my mind that our American oligarchs were well aware that state legislatures were not equipped to investigate and address adoption of Common Core Standards in 2009-2010.
(posted by concerned)
Subscribe to:
Posts (Atom)