Recently I read a book about standardized testing. The author was a person who worked in the industry for 15 years in a number of different positions, participating in and supervising the scoring of written english/reading tests. It deals with a lot of the flaws within scoring "standardized" tests, pointing out how test scoring is not only designed poorly, but used to evaluate the wrong things at high stakes (i.e. schools under No Child Left Behind).
What I found most interesting about the process was that the criteria used to judge student responses to open-ended questions. Not only can the rubrics be too vague and subjective, they can also be far too strict and end up evaluating unimportant things. As a result, the scoring process can be very inconsistent and may result in matching written responses to the rubric's description rather than the score the written response deserves.
One example was of a 4th grade essay that was graded with a rubric emphasizing the correct use of the 5-paragraph form above any other writing criteria. As a result, an essay that was completely mechanically written ("The most important thing to me is my cat because she is fun. She is also loving. She is soft.") had to be scored higher than an essay that was impressive and very descriptive, but did not follow traditional 5-paragraph form.
What was, I thought, a particularly striking part of the book was when the author wrote:
"While rubrics are written by the best intentioned of assessment experts an classroom teachers, they can never--never!--come remotely close to addressing the million different ways [kids] answer questions. If nothing else, standardized testing has taught me the schoolchildren of America can be one creative bunch" (6).
The last phrase I didn't expect to hear at all. Thinking back to when I've taken standardized tests when I was younger, I never gave open ended questions much thought. I thought the assessments asked for very straightforward things. However, once you give a test to hundreds of thousands of little kids, things get muddy. How is a scorer supposed to get inside the head of the kids taking the tests to know what they were thinking if they took a different approach? If they answer in a creative (or just confusing) way that doesn't quite fit in the rubric, how is it scored? This can be disorienting to scorers. New, unique examples they come across while scoring can end up changing the way they grade several times within the same grading process, because they set a new precedent.
Also, the fact that we put value in to standardized tests (used to evaluate schools, test one before college, and determine who gets their diploma in some states) is rather concerning, considering the scoring of these tests essentially requires scorers to try and get thousands of student responses to fit in convenient little boxes. But the reality is that sometimes peoples' responses don't fit anywhere! As such, some scorers may have to resort to marking down 'incorrect answers' so much as 'different ones,' even different ones that are brilliant. It's interesting to see how literally standardized testing is opposed to diversity and creativity-- and "standardization" itself.
No comments:
Post a Comment