Earlier this week, state Superintendent Joy Hofmeister said she was dropping the state’s writing test from the A-F grading equation for schools this year because the test and results were unreliable.
The test vendor wasn’t an issue, she said.
Oklahoma Watch, however, learned that school officials are questioning the qualifications of people grading the tests. The graders were hired by Kelly Services, a temp agency under contract with Oklahoma’s new test vendor, Measured Progress. That agency was also used by the state’s previous test vendor, CTB/McGraw-Hill, which was criticized last year for improper grading of the writing exams.
This marks the second year in a row that the writing scores for fifth and eighth graders have been thrown out of the letter-grade calculation by the state Department of Education.
Education officials have said too many of the scores were similar or had glaring discrepancies, such as students who do poorly in reading scoring higher on the writing test than those who read well.
According to a job posting on the Kelly Services website, the pay for test graders needed in a number of states is $11.50 an hour. Applicants must have a bachelor’s degree in English.
The company benefits include free coffee and cocoa, a large break room and casual dress.
The headline for the company’s ad reads: “Welcome! You have taken the next step towards putting your college degree to work!”
Officials with Kelly Services and Measured Progress said any training or education requirements are set by the state Department of Education, and that extensive training is provided. The scorers are also evaluated daily, and each test is scored by two people.
The graders work out of scoring centers in Dover, New Hampshire, Longmont, Colorado, and Menands, New York.
Kelly Services has also been criticized for recruiting on Craigslist.
Clinton Public Schools Assistant Superintendent Tyler Bridges, a former eighth grade English teacher, said he was shocked by the minimum requirements and the fact that Kelly Services would recruit on Craigslist.
Bridges said he had concerns about the qualifications of test graders, especially if they are fresh out of college and have no experience.
“That just kind of blows me away,” he said.
CTB/McGraw-Hill and Pearson have faced similar criticism over the hiring of test graders.
Measured Progress was awarded two contracts in December worth a combined $48 million through fiscal year 2019 to handle Oklahoma’s testing.
Judith Rubenstein, a spokeswoman for Measured Progress, said in an email that Kelly Services recruits, screens and trains the scoring staff. Measured Progress then directly supervises and operates the scoring process. The scores are based on the state’s grading rubric.
Writing tests are more difficult and expensive to grade than standard multiple -choice tests because they a person must review and score each student’s passages.
Department spokesman Phil Bacharach said this year’s concern is that the scores don’t accurately reflect students’ abilities.
Bacharach said he did not know if department officials knew Measured Progress uses the same company to hire testers, but added the state does not have any concerns with the testing company or the test graders.
“Bottom line is, the first thing the superintendent wanted to address is specifically with the issues with the results of the test itself,” Bacharach said. “This is not an issue with the people.”
However, several school officials said the similarity of so many writing scores indicates a problem with the grading, not just the test.
Rick Cobb, assistant superintendent of curriculum and instruction at Moore Public Schools, said he believes poorly trained or qualified test scorers are part of the problem. Too many of his students had similar grades on the exam, he added.
“The whole process is flawed from start to finish,” Cobb said. “We need to do a better job at training teachers – and that’s on the schools. We (also) need to do a better job picking the tasks for the test. And the test vendors – they need to not do this on the cheap.”
Cobb said he understands the grading job will be temporary because of the nature of testing, but he said the sheer volume of exams companies have to grade each year means they have to invest in the right people.
“If they (Measured Progress) are going to keep paying people on the cheap, we’re going to keep getting the same results,” he said. “We need to see a scoring company not try to gouge the state for profits or skimp on their employees."
Oklahoma Parent Teacher Association President Jeffery Corbett also criticized the writing test and the amount of time students spent in class preparing for the exam, which takes away from instruction.
"It is time to take our classrooms out of the hands of the billion-dollar testing industry. Our children deserve the opportunity to become innovate, critical thinkers, not just great test takers," Corbett said.
