kitchen table math, the sequel: clickers for college

Thursday, July 24, 2008

clickers for college

Ohio State University students who used the devices to answer multiple-choice questions during physics lectures earned final examination scores that were around 10 percent higher – the equivalent of a full-letter grade -- than students who didn't.

The clickers also appear to level the playing field between male and female students. In clicker classes, male and female students performed equally well. In the traditional, non-clicker classes, male students outperformed female students.

[snip]

Around the country, clickers are regularly used to maintain student attention in large lecture halls. At large universities such as Ohio State, even relatively advanced science classes may contain hundreds of students. Reay said that clickers are a good way to help students pay attention and learn in today's classroom.

[snip]

In clicker classes, multiple choice questions appear on a large computer screen at the front of the lecture hall. Students hold the wireless devices, which resemble small calculators. They cast their votes for the correct answer based on their understanding of the part of the lecture that was just given. A bar graph shows the percentage of students voting for each answer.

Physics educators have expanded the use of clickers at Ohio State by developing sequences of questions to determine if students really understand the underlying concepts of a lecture. The technique involves offering a series of questions -- typically three -- each with different wording and structure, but all designed to test the same concept.

One three-question sequence that Reay and his colleagues developed is used to test students' understanding of Faraday's Law. The class is shown a diagram of two wire loops of different sizes moving into a magnetic field, and asked which loop will experience larger induced voltage at the moment it enters the field.

For an introductory electromagnetics class of engineering majors, this is an easy question and Reay would expect more than 80 percent to answer it correctly.

"But not all students choose the correct answer for the right reason," he said. "A common misconception is that larger loops always have a larger induced voltage, which is not the case. That's why we then ask two more questions that involve loops of different sizes and shapes. The question-sequence method eliminates common student misconceptions, and helps students grasp the underlying concept in a short time."

The idea with clickers, Reay explained, is that both lecturers and students can gauge whether students understand the material in real time. If students don't understand something, the lecturer might try to get them to think about the topic in a different way, perhaps by discussing it amongst themselves to encourage understanding before moving on to a new topic.

Professors typically devote approximately 20 percent of a class to clicker questions, and the rest to traditional lecture and discussion.

[snip]

Students aren't required to use the clickers, but Reay said that participation over the years has held relatively constant at 90 percent. Small incentives, such as grading clicker questions or offering extra credit, can raise participation to 98 percent.

Students seem to like the clickers.

“When we conduct our quarterly surveys, we find that the percentage of students enthusiastically favoring use of clickers is higher than 90 percent,” Reay said.

"In addition to this popularity, our research indicates that our sequence method of using clickers offers students a significant advantage on learning, and we are working to sharpen our methodology for measuring these learning gains. We're finding that the timing of pretests and post tests during the quarter matters, along with different applications of incentives,” he added.

"In addition to this popularity, our research indicates that our sequence method of using clickers offers students a significant advantage on learning, and we are working to sharpen our methodology for measuring these learning gains. We're finding that the timing of pretests and post tests during the quarter matters, along with different applications of incentives,” he added.

[snip]

Both sexes had nearly equal gains in the clicker classes: females had a score gain of 6.2, and males a score gain of 6.7. In the non-clicker classes, the female score gain was 4.3, and the male score gain was 6.6.

As to why clickers seem to help female students more than male students, the anonymity of the devices may offer an advantage. In surveys, nearly all the students -- male and female -- said they liked having the ability to keep their answers private. Students can easily hold the clicker in one hand and block their neighbor’s view. “We suspect that anonymity is a benefit, but right now that is really just speculation," Reay said.

Students Who Use 'Clickers' Score Better On Physics Tests

Am I reading this right?

There was a tremendous gain for female students and essentially no gain for male students? (Of course, I don't know what 1/10th of a point amounts to.)

This seems strange.

Is this a common outcome in college physics courses?:

During the 2006-2007 academic year, the clicker classes outperformed the non-clicker classes by an average of 10 percent -- a full letter grade -- on the multiple choice part of their final exams. The non-clicker class averaged a score of 61 percent on this section; the clicker class averaged 72 percent.

12 comments:

Anonymous said...

It sounds like there was a larger gain by female students than by male students when using the clickers, but both gained. (Not "essentially no gain for male students").

In this case, it sounds like they are using the clickers for immediate feedback, much like DI (capitalized) includes lots of interaction to reinforce the specific ideas they are trying to get across - great to see what is essentially independent verification of Zig Engelmann's ideas (at least as I absorbed them from his DI Rubric, linked to from old KTM).

I suspect that the larger gains for female students with clickers won't be seen in all subjects, just the ones that are (traditionally) male-dominated - physics, engineering, computer science - since the anonymity helps prevent even unconcious preferential treatment from the professors.

Now all we need to do is get the same interest and methodology (try new things and MEASURE the results) shown by these college professors into the mainstream education industry.

"...our research indicates that our sequence method of using clickers offers students a significant advantage on learning, and we are working to sharpen our methodology for measuring these learning gains. We're finding that the timing of pretests and post tests during the quarter matters, along with different applications of incentives”

Anonymous said...

Like anonymous, I think this reflects on the instructors not the students.

The instruction changes with the feedback to address misunderstandings that might be gender based. The students do not become more engaged with clickers in their hands and suddenly improve their learning ability.

Sean

Rudbeckia Hirta said...

I think that my students become more engaged with clickers. In my class the clicker-questions count in their grades. I put the PPT slides (including the clicker questions) on the web 2-5 days before class, and the motivated students go through the slides ahead of time and work through the questions so they'll be sure they get them right. The "average" students make a point of coming to class so that they can get the points from attempting the questions. (They get 0.8 points for trying the question and getting it wrong and 1 point for trying it and getting it right.)

Catherine Johnson said...

boy, I can easily see myself getting more engaged --- this is the first piece of classroom technology that has instantly appealed to me.

rudbeckia --- wow

That is really interesting.

Clickers would stir up my competitive juices (both with other students & with myself) and they would help me keep my attention focused on the class.

Catherine Johnson said...

What are your students doing -- they're working out the solutions to a problem you've put on the board?

How does it work, exactly?

Rudbeckia Hirta said...

I ask a variety of questions with the clickers. Most are multiple choice, but some ask for a numeric answer.

Sometimes I'll ask a question, like "Did the homework prepare you for the test?" and have m.c. answers like "Yes, it did." or "No, I did the homework but felt unprepared" or "I wouldn't know because I didn't do the homework" or "I've been busy and haven't been studying" or "No comment" or "none of the above".

Near the beginning of class I'll sometimes ask a review question, based on something that we did in an earlier class. Sometimes from the recent past, sometimes from the distant past.

Sometimes the "warm up" question will foreshadow the skill for the day.

During lecture, I'll have a few questions. After I show an example, sometitmes I'll have the students do a question using the same technique. Sometimes I'll ask a non-calculational question ("Which of the following quantities would you rather have negative? A. The derivative of the profit function, B. The derivative of the cost function. C. The revenue function, D. The average profit.")

I use PPT with a tablet. All the PPT slides are on the web before class, and the students print them out. During class I use the tablet to write on the PPT slides, and show how the problems are done. All the clicker questions are on the PPT slides, so the students know all the question in advance. The best students work ahead to make sure they can meet the objectives.

Tex said...

My school is introducing clickers (Senteo devices for SmartBoards) in more classrooms this year.

It does sound appealing in that it enables teachers to gather student data quickly in order to efficiently assess understanding. Sad to say, the cynic in me believes it will not be used in this way by most.

The anonymous competition aspect sounds very appealing. It reminds me of a trivia game I came across during a Delta flight last year where I was competing against other passengers. It engaged my son and me to keep playing to earn points, and try to beat our fellow passengers. It was not clickers, but we played by touching the video screen in each seat. This game made the cross-country flight go much quickly than any movie or TV show could have.

Catherine Johnson said...

Rudbeckia --- wow -- that sounds wonderful.

I hate to sound like a person attending a Technology Camp, but this is the one edu-technology I've instantly sparked to, speaking as a student and as a parent of students.

I was thinking about all this....and I just don't see "technology" disrupting education in the way people keep hoping it will --- which is, basically, to get rid of teachers and books.

As far as I can tell, that's been a theme going back to Edison. "Technology" was going to replace teachers or books or both. (Someone - possibly Edison - said movies were going to replace books in school. Of course, that prediction has finally come true in my middle school so maybe I don't have a point here...)

If you look at all the miracle breakthroughs: movies, filmstrips, CDs (remember when everyone was going to learn everything on CD?), distance learning (a HUGE flop -- a costly flop)---the betting is always on the "content delivery" side.

The problem is, books and teachers have been around for a very, very long time, which tells me books and teachers are probably doing something right.

Or, if not right, something necessary or basic --- something we respond to.

I think Paul said, in a comment, that technology isn't going to replace teachers --- what technology can do is ramp up assessment into something that works.

That's where I would put clickers, in part.

I do think clickers have a lot of potential to "engage" students, as Tex describes (I know they would me).

Not sure what category that goes into.

But they have a huge potential, I think, to make informal assessments extremely rapid, efficient, precise, powerful, etc.

Did someone put up a post about that assessment company schools can use to have their kids taking regular assessments online that show them instantly where the kids are in the curriculum??

heck

I'm not sure what that study was.

I suspect technology is going to gives us -- or, at least, could give us -- terrific assessment tools.

Catherine Johnson said...

hmmm....

I'm thinking....SuperMemo is another animal....

No---actually, SuperMemo more or less belongs in the "assessment" category.

The point of SuperMemo isn't to teach you; the point of SuperMemo is to figure out when, exactly, you need to review what you've learned.

Catherine Johnson said...

Of course, I could be completely wrong given that the number of online high school courses is apparently growing like crazy.

Catherine Johnson said...

Even so, I don't see online courses replacing teachers. Not for the important courses - which means the "hard courses" and/or the courses you care about (i.e. the courses you aren't taking only because the state tells you to).

I can easily imagine online "Health" courses replacing Health teachers.

I can't easily imagine online history or literature courses replacing history or literature teachers.

(With math, who knows, given the situation with EM, etc.)

MNGator said...

Using the Renaissance 2Know! response system and it is fantastic. I can put the questions and answer choices right on the handheld unit, so no need for me to make powerpoints. Also very easy to use for formative assessments. I haven't tracked student improvement, but maybe will this year.