Posts
Students have long believed (on good evidence) that if the same paper is submitted to two teachers in two different sections of the same course, the paper is likely to receive two very different grades. In 1961, Paul Diederich and his colleagues proved that this student belief is no myth. When 30 student papers were graded by fifty-three graders (a total of 15,900 readings), more than one third of the papers received every possible grade. That is, 101 of the 300 papers received all nine grades: A, A-, B+, B, B-, C+, C, C-, and D. Diederich also reported that
94 percent [of the papers] received either seven, eight or nine different grades; and no essay received less than five different grades from fifty-three readers. Even when the raters were experienced teachers, the grades given to the papers by the different raters never attained a correlation greater than .40. Diederich, P.B., French, J.W., and Carlton, S.T. "Factors in judgments of writing ability." Research Bulletin RB-61-15. Princeton, N.J.: Educational Testing Service, 60 pp.
The Schools We Need and Why We Don’t Have Them
E. D. Hirsch
185-188
Sunday, December 4, 2011
from the archives: grading student writing
from 2009:
Subscribe to:
Post Comments (Atom)
6 comments:
This is so comforting, given that I feel like the ONE part of the December SAT I may have aced.....was the Essay!
We switched, in Gen Chem and Organic, to having one grader for each set of lab reports instead of having lab instructors grade their own reports, just because it was too hard to keep grading consistent across graders. And those are relatively simple to grade compared with papers in the humanities.
So I can't say I am surprised.
I suspect that for subjective grading courses the bell curve is narrower than for courses like math. And, the number of top grades are fewer. Most students know about courses where, if the teacher continued with the same points-off scheme, one could end up with a negative grade. That doesn't happen because the teacher starts finding less things wrong as the grade gets worse.
My son knows that courses like English and history are the fuzzy grading courses where it's almost impossible to get a high grade. Nobody at his high school gets a 4.0 GPA. They say so on their web site. The top GPA indicates that one grade was a 93 or less. It was probably in English.
My son's Am. Lit. teacher loves to make comments about how the different teachers grade. One was on how the lowest grade is probably the correct one. It just so happens that his grades are most often the lowest ones. He's a real peach of a guy. I emailed the head of curriculum about their posted grading policy on how teachers are supposed to calibrate their grading, but I heard nothing back. She also never responded to my query on whether group tests (summative) are allowed, especially when the one in question was a surprise decision on the day of the test and there was little time for collaboration.
There are many bigger problems in high school paradise.
so scary but not surprising at all.
Obviously, having the same essay in different sections of the same course receive an A or a C grade is wrong. But as a writing teacher, I think a smaller variation probably is not. Just as you'd be taught a slightly different version of a time by different history teachers, based on their specific areas of knowledge and interest, English teachers can legitimately see different qualities in a student's writing. Some are more focused on grammar and structure, for example, while some may be more likely to reward original thinking and vivid detail. If both are talented and reasonable teachers and are specific in their essay comments, the student will become a more well-rounded writer after having learned from each. However, even some strong teachers are so overburdened by the amount of student writing they must respond to that they can't clearly assess it all, which is another matter.
That would be fine if grades were destroyed at the end of each class and kids were rewarded with scholarships, college admissions, etc., solely on the basis of what they actually knew and could do at time of application. Then, their past teachers' personal preferences about how strictly to grade or their little "I'll show you who's boss" games, or their "I expect more from you than the other kids; this 'C' is for your own good; you'll thank me later" type of "differentiated instruction" and so on wouldn't make any difference when they applied to Stanford later.
If writing alone, without old writing grades, were used for college application, kids could even exploit this crazy variance among graders by applying to all the Ivies with the same paper. Every possible opinion would be held by at least one school, and the one that considered him a genius would admit him.
Post a Comment