Greetings all,
I attended a workshop on problem solving given by the school board's math task force. I was told by the presenter that the text I am using does not contain a single level 4 question. As I scratched my little head in disbelief, I could not help but wonder how my former students are getting such high marks in high school calculus. I guess the two years they spent with my substandard textbooks was not too harmful.
Back in February, I attended a regional professional network committee conference on using data. The meeting itself was a complete waste of taxpayers' dollars, but I was left with a few new questions. I usually hear grumblings from fellow staffers about lousy tests from textbook writers. Since I never taking a course in writing assessments, I tend to use the assessment instrument given to me by the publisher. I figure if I follow the program, then the assessment should be useful. Maybe I am truly misguided. To make a long story short, I'm trying to better my assessment skills in the classroom. Currently, I am reading Frank Baker's book (http://edres.org/irt/) on item response theory.
While working through the calculations for item difficulty and item discrimination, I began to ponder the connection (correlation) between these calculations and the idea of level 1, 2, 3, and 4 questions. What should an Item Characteristic Curve look like for a level 4 question or a level 1 question? I don't have a clue, and to make matters worse, I have no idea who to ask for an answer.
Any ideas?
Statistics! Data! Number crunching! Fun! Thanks!
ReplyDelete(Was that geeky?)
Two years ago I went to the Air Force Testing Center to help write our career fields promotion test.
ReplyDeleteIf I am correct, and I am probably not.
Assuming a Level 4 questions purpose is to discriminate between the abilities at the high range, wouldnt the curve be like relatively flat at lower abilities and then approach 90 degrees towards the left.
In other words, it would discriminate the most at the upper levels, with only very few of the average to poor performers getting it right.
Note: I am assuming by Level 4 question you mean:
Level 4
The student has demonstrated a full and complete understanding of all concepts and processes embodied in
this application. The student has addressed the task in a mathematically sound manner. The response
contains evidence of the student’s competence in problem-solving and reasoning, computing and
estimating and communicating to the full extent that these processes apply to the specified task. The
response may, however, contain minor arithmetic errors that do not detract from a demonstration of full
understanding.
• Contains a complete response with clear, coherent, unambiguous, and elegant explanation
• Includes clear and simple diagram
• Communicates effectively to an identified audience
• Shows understanding of the question’s mathematical ideas and processes
• Identifies all the important elements of the question
• Includes examples and counter examples
• Gives strong supporting arguments
• Goes beyond the requirements of the problem
When I say left, I actually mean the right side of the curve.
ReplyDelete"When I say left, I actually mean the right side of the curve."
ReplyDeleteThat would presumably be the "civilian left". 8-)
I'm confused.
ReplyDeleteThe more capable a student, the higher the probability the student will get the question correct. The more the item discriminates, the steeper the curve.
ReplyDeleteBut is there any statistical standard for what constitutes a level four question?
Is this a level 4 grade 8 math questions?
(-1)^5-8^2