Saturday, May 5, 2012

What do valid and reliable tests look like? IDK

My Opinion

I am busy this weekend trying to understand all I can about Annual Professional Performance Review and reading the inspirational Common Core based units of my grad students who aspire to make a difference in the lives of children; meanwhile, there is a quite furor over a pineapple question that fuels the controversy about teacher evaluations.  Believe me, I think teachers must be on their "A" game all the time because what we say and do every moment of every day impacts our students; however, the proposed teacher AND student evaluations do not reflect the attributes of teaching and learning that matter.

From the NY TIMES

Pearson Says Its Tests Are 'Valid and Reliable'

May 4, 2012, 6:46 p.m.

A national test publisher has defended a controversial question involving a talking pineapple on one of its reading tests for New York State, saying it was confident that its tests were “valid and reliable.”
In a letter to the State Education Department disclosed on Friday, the company, Pearson, said the passage about a pineapple that races a hare, included in a test administered to eighth graders, was used to measure students’ understanding of character traits, motivation and behavior.
The company said the question had been used in several other states without complaints. But Pearson did not mention that the question was mocked on a Facebook page established in 2010 and on a blog starting with a 2007 post from a parent in Illinois, who was soon joined by a long string of others in Alabama, Arkansas, New Mexico and now, New York.
The company said it had been field-tested among students in New York and vetted by a local testing committee to ensure it was valid.
“Pearson is confident” that the tests it prepared in both reading and math “have been developed to support valid and reliable interpretations of scores for their intended uses,” the letter said.
The pineapple passage, a parody of Aesop’s fable about the tortoise and the hare, was part of a reading test administered on April 17. It caused an immediate uproar among students, parents and teachers in New York, who said it was nonsensical and unfair. Three days later, the state announced it would not count it in children’s scores.
New York’s decision to void the question’s results was a huge embarrassment for Pearson, which has a $32 million testing contract with New York and contracts with other states.
Soon after, the state threw out questions on two math tests because of errors in the questions or answers, and warned that another math question had two possible correct answers.
Pearson’s letter, which was first reported Friday by Time magazine online, was dated April 22, two days after the state nullified the pineapple question. Pearson sent the letter unsolicited, according to Dennis Tompkins, a spokesman for the State Education Department.
But the letter did not mollify some educators.
Sharon Emick Fougner, a principal in Great Neck, N.Y., who wrote a letter on Tuesday to the state education commissioner, John B. King Jr., asking for a review of the math tests, said in an e-mail on Friday: “The only thing more absurd than ‘The Hare and the Pineapple’ is Pearson’s lengthy, verbose and ludicrous defense of this passage. If the tests had not been so damaging to children, this memo would be comical.”
In the test, the pineapple story was attributed to Daniel Pinkwater, a popular writer of funny, often absurd tales for children. But Mr. Pinkwater’s original story, about an eggplant and a rabbit, had been so thoroughly bowdlerized that when the controversy erupted in New York, he disowned the version that appeared in the test.
Ms. Fougner said “even noneducators” recognized that test questions about which animal in the story was the wisest and why the animals ate the pineapple when it lost the race “required students to render an opinion that could not be validly substantiated by the text.”
Pearson disagreed, saying the answers could be derived through “evidence” from the text. The wisest animal was the owl, Pearson wrote in its letter: “The owl declares that ‘Pineapples don’t have sleeves,’ which is a factually accurate statement. This statement is presented as the moral of the story, allowing a careful reader to infer that the owl is the wisest animal.”
In interviews after taking the test, however, many New York schoolchildren said it was hard for them to believe that the owl was the wisest because the statement that pineapples don’t have sleeves — which later became a running joke at some schools — was so dumb. (The moral of Mr. Pinkwater’s original story was, “Never bet on an eggplant.”)
Some children said they found the story funny, but were disturbed by the humor because they had learned from past experience that test questions were not supposed to be funny.
Ms. Fougner, echoing the anger of many teachers, said she could only assume the purpose of such questions was to “deceive, demoralize, exhaust and frustrate students.”
The Pearson letter said the pineapple story had been used since 2004 in six other states and three large cities, Chicago, Fort Worth and Houston, without incident. But the letter did not address informal complaints, like those in a blog called In the Break Room.
Merryl H. Tisch, chancellor of the New York State Board of Regents, said Friday that she thought the pineapple question made sense in context, but that in hindsight, the passage should not have been used because it added fuel to arguments by those who oppose using testing in teacher evaluations.
“Listen, I’m not going to defend the pineapple story,” Dr. Tisch said. “I don’t like excuses. When you need to execute something, you need to do it. I don’t believe this chorus of, ‘Well, these mistakes happen all the time.’”
But she said the furor had been amplified by the antitesting movement and had given ammunition to opponents of using testing to evaluate students and teachers.
“It gives them momentum, and I believe that’s what’s going on here,” she said.
Maria Newman contributed reporting.
Anemona Hartocollis covers health issues in New York City for The New York Times.

No comments: