kitchen table math, the sequel: Retrieval practice, the bad news

Saturday, April 19, 2014

Retrieval practice, the bad news

I was thinking the incredible results people find with retrieval practice sounded too good to be true, and now I'm worried they are:

Failure of further learning: the limits of repeated study and retrieval practice

Wish to heck I could pull the study. All I've got for the moment is the abstract:

Previous research has shown that little benefit is achieved through spaced study and recall of text passages after the first recall attempt, an effect that we term the failure‐of‐further‐learning. We hypothesized that the effect occurs because a situation model of the text's gist is formed when the text is first comprehended and is consolidated when recalled; it dominates later recall after verbatim memories of more recent study episodes have been lost. Experiments 1 and 2 attempted to circumvent the effect by varying the activities of participants and requiring interactive exploration. In both experiments, recall after four, weekly sessions showed little benefit beyond performance on the first recall. Experiment 3 interfered with the formation of an immediate situation model by introducing passages that were hard to comprehend without a title. Performance improved substantially across four sessions when titles were not supplied, but the standard effect was replicated when titles were given. Experiment 4 made verbatim memories available by incorporating all re‐presentations and tests into one session; as predicted, recall improved over successive tests.

Failure of further learning: Activities, structure, and meaning
Catherine O. Fritz,
Peter E. Morris,
Barbara Reid,
Roya Aghdassi,
Claire E. Naven
Unfortunately, I don't understand the final sentence in the abstract.

I see that "interactive exploration" didn't help students remember anything beyond what they recalled during their first retrieval practice. (Another nail in the coffin of hands-on, guide-on-the-side activities as the cure for limited remembering and understanding.)

I see that difficult reading passages without titles (!) helped. That finding is especially striking in light of the SQ3R approach to reading comprehension, which involves paying attention to titles. (SQ3R isn't necessarily incompatible with this finding, but still...)

It sounds like the last sentence refers to a complete, or near-complete, re-creation of the original learning-and-quizzing episode.

Did students re-read and quiz themselves, both in the same "session"?

I should go read the summary at Jung's Biology Blog.

I'm probably going to enjoy it:
In other experiments, Fritz et al. show that FOFL occurs even when ideas are presented as itemized lists on Powerpoint slides (why am I not surprised).
Update: OK, I should have read first, posted later.

The last sentence has to do with word-for-word memorization, which does improve with repeated retrieval practice.

As for the rest of it, in fact, recall does improve over time, but not by nearly as much as one would hope.

"Elaborative" study techniques -- underlining, annotating, diagramming (Make It Stick highly recommends elaboration) -- had no effect at all.

Difficult texts without titles offer no advantage I can see from the summary: students recalled less during their first retrieval practice and then, over time, continued to recall more until they reached the level of students who had read a clearly-written text with a title. Lots more pain, no gain.

At this point, I don't think this study tells me too much. The authors' theory - which I do find quite interesting - is that a student's mental model blocks further learning. Specifically, the student's mental model blocks learning of content that didn't make it into the model.

That strikes me as highly likely; I've experience a "gist" effect myself, and I think "gist effects" are a major problem in any kind of reform effort.

However, fields of study are different from a book you are trying to remember, which is what the students in this study were trying to do.

When you study a field, one of the things you are learning is the field's organization and categories.

New content gets slotted into pre-existing categories. That's why the more you know about a subject, the easier it is to learn new aspects of that subject. You have a mental 'gist' of the subject that is 'hungry' for new content, or at least wide-open to it.

1 comment:

Glen said...

Since some people do become experts eventually, it appears that at some point--more likely, at multiple points--they become aware of the shortcomings of their models and upgrade them.

This suggests that learners need a good way to find all model errors as quickly as possible, one that forces them to acknowledge each one. As soon as they acknowledge a specific flaw in their model, the correct model should be provided.

I like those big, fat books full of thousands of answered questions about a topic. Textbooks often provide a lot of information followed by a few questions. Presumably, this is just a statistical sampling technique: If you get most of the questions right, you probably understood most of the material.

But these big, fat question books aren't samplings; they are exhaustive. They try to make you use every aspect of your mental model again and again and show you how your model ought to be working. Exhaustive questioning quickly forces model errors into the open, and the explained answers provide you with the correction at the moment you conclude you need it.

Elaboration methods are good for tying the bundle together, but you still need a good way to pull the weeds out of the bundle.