Posts
If 80% of student feedback is wrong, what % of teacher feedback is wrong? Are we just assuming that the teacher is 100% correct the 20% of the time they actually provide feedback?
Hah!
I wish ---
Students have long believed (on good evidence) that if the same paper is submitted to two teachers in two different sections of the same course, the paper is likely to receive two very different grades. In 1961, Paul Diederich and his colleagues proved that this student belief is no myth. When 30 student papers were graded by fifty-three graders (a total of 15,900 readings), more than one third of the papers received every possible grade. That is, 101 of the 300 papers received all nine grades: A, A-, B+, B, B-, C+, C, C-, and D. Diederich also reported that
94 percent [of the papers] received either seven, eight or nine different grades; and no essay received less than five different grades from fifty-three readers. Even when the raters were experienced teachers, the grades given to the papers by the different raters never attained a correlation greater than .40. Diederich, P.B., French, J.W., and Carlton, S.T. "Factors in judgments of writing ability." Research Bulletin RB-61-15. Princeton, N.J.: Educational Testing Service, 60 pp.
The Schools We Need and Why We Don’t Have Them
E. D. Hirsch
pages 185-188
7 comments:
Many students also believe that if an identical paper is submitted by two separate students to the same teacher, the paper will get two different grades.
The students are probably correct on that, too.
If it mattered to those who give out grades, all essays would be turned in:
*) Typed
*) With a single identical font
*) With the name on a separate, *last* page stapled to the essay, not on the front page.
-Mark Roulo
Grading in the sciences is probably more objective, but it certainly isn't close to blind justice.
In my large computer science course, we took a lot of measures to counteract our own biases.
For example, the grading methodology was written down at the time the exam was written, which all points written out as corresponding to what piece of the answer, so that acceptable partial credit was already clear before anyone read an actual student's answer. (this helped as long as you wrote the problems well; sometimes, you'd see the answers pour in and realize you'd made a mistake, as kids were getting lots of partial credit for answers you really knew didn't deserve it, but met your specific criteria.)
On a multi problem exam, if at all possible, one grader was responsible for ALL of the submissions for a given problem. That meant that at least there should have been consistency across the pages, but often, as the minutes go on, you'd see a number of answers repeat but credit creep would come in, and you'd either inflate or deflate the partial credit over time (depending on your own biases.)
we encouraged students to not write their names on the papers, but instead their student ID number or other randomly generated identifier. It helped a fair amount, but by the final exam, you still knew who was who.
TAs were encouraged NOT to grade their own students if they recognized them, but that meant violating the "one question graded by one person" rule. If they did grade their own, they were likely to say "oh, I know what he MEANT to do here" and award too much credit. And if they were especially diligent about not doing that, they could ding their student at the expense of others. (At least we didn't have a curve, so relative point differences weren't so important.)
I think the main difference was that we thought such bias was bad. I can still remember a history professor telling me that of COURSE he graded different students differently--they all had different skills, and some were brighter than others, and he expected more of them, so he held those brighter ones to a higher standard, judging their work more harshly. An A from him depended on who *I* was, you see, and he was unabashed about this philosophy. To him, this was the proper role of a professor to mold the students who could be molded.
This is why I fled to the sciences and mathematics.
Incidentally, in NZ for our external high school exams we only put a student ID number on the paper, and the papers were sent to different areas of the country to be marked to minimise the risk of a marker recognising the student's handwriting and this remove bias (markers were often, perhaps always, teachers).
THANK YOU FOR LEAVING THESE COMMENTS.
Fair grading practices are a MAJOR issue here, one that I'm going to have to tackle in the not-too-distant future.
I've got the URL recorded; will get these comments pulled up front...
Many institutions limit access to their online information. Making this information available will be an asset to all.
Anybody notice the the grading practice in their high school or middle school favors honors sections? I've noticed honors will have tons of easy quizzes, while nonhonors will have app. 10% of the amount of quizzes. Each quiz is worth half a test grade in the weighting. Always wondered how students could get a 98 average; now I know.
Post a Comment