Pottstown School District recently found out that it’s standardized tests may have shown some kind of irregularity. The irregularity is consistent with possible cheating.
There are several different kinds of irregularities when it comes to tests. There are tests where all the students appear to have the same answers- in that case the assumption is that they were given the answers either ahead of time, or during the test. Or maybe a teacher arranged seating so that smart children were seated next to dullards and everyone was invited to look at the smart kids work.
These types of cheats are usually identified by the people involved with them. It’s hard to believe that a large group of kids could have been directed to cheat without at least one of them mentioning that experience to someone who had a clue.
Another flag is raised when students answers are changed. John Armato pointed out that he’d be interested in seeing how many changes students made on tests that were right answers to wrong answers- because the flag that alerted officials to possible cheating was WTR answers, or wrong to right answers. In order for a large number of students to change answers from wrong to right, they either have to be coached, or educators might have been responsible and changed the answers themselves to make it appear that the students had a better grasp of the material.
I just read about Atlanta’s problem with cheating. They have named teachers and administrators who have cheated. They have gotten admissions from some of those teachers that cheating occured at parties in which a group of people got together and “fixed” tests.
That is nuts!
The wildest thing for me to read is that the measurement companies that check the tests have had the ability to check things like suspicious erasures for years. In Atlanta this started in 2008 I think…in PA we are looking at results from 2009.
That is wild!
I’m not really comfortable with the evidence that suggests cheating was happening. For example, tests assume that each child may have up to 3 legitimate changes on their test of wrong answers to right answers. This is just an example, but follow me- now let’s say a child is answering on a scantron system and they got their answers mixed up. That child may well erase 5 or 6 answers and correct them. That child may now be included in that 3 standard deviations from the mean that would flag a cheater. Although, in our example no cheating happened!
Now let’s imagine that the tests were created so that that wouldn’t happen, in otherwords, they expected that children might erase 4 or 5 questions in a row and fix them. So the test may not be looking at blocks of adjusted answers- but we don’t know that! And this is one of the reasons that John Armato might have been asking about how many questions students adjusted that went from right to wrong!
These tests for cheating just show general overuse of the WTR changes. They aren’t testing particular students results and they aren’t testing the same tests over again necessarily. They are saying ” these are our margins” When we see any deviations we flag them as possible cheating violations. When we see a standard deviation of 3 or more then we are almost sure it is evidence of cheating.
Now let’s talk about those deviations. While it sounds like a sure bet that there has been cheating going on it only works because there is a norm that gets created when most students take tests. If the average student changes 3 questions then our flagged test might only have 5 questions changed, or 6 questions changed.
So let’s imagine again…Let’s say a test was created by a company using all the standard methods. But let’s say that no one checked the actual questions. Let’s say a group of students in one particular school was advised during their instruction about a particular connection between some of their material. So when they answer question 14, they suddenly realize that question 4 and question 8 are wrong. These students just hit the standard for correcting questions that were wrong to make them right.
If they make any further changes on their tests, which they might be more likely to do now that they have noticed what they consider might have been a trick- then their tests will be flagged as if there was some cheating going on in their tests.
Yes, there is a lot of science involved in test taking and test grading. Yes, I’m sure it works very well for the most part. But there are also some problems! And schools hire companies to test their systems and their states hire testing facilities and tests. All of these systems are products. They all have a cost.
My concern is that we are blaming teachers and administrators for doing something that they haven’t been shown to have done. Again, remember that there is just a question of cheating- no one has been accused of it. The only thing that will happen is schools that have been accused of cheating will have a “forensic audit”.
I don’t imagine that means they’ll test the students again. What will happen is they’ll test their machines. And they’ll re-grade the tests. But none of this addresses problems with the answers on the test. Oh, that is probabaly because we’ve already agreed that the test is perfect- but that is another silly assumption. How can a test be perfect for students from diverse communities? How can a test be perfect for Latino students and Black students and Caucasian students and Asian students from different communities?
If a child is familar with the phraseology of a question or with the language- will they not have more time to respond and then go over and make changes to their booklets?
For a child who is struggling with the language and who might feel confused by certain questions- will they have time to check and recheck answers?
When you really think about it what we have here is a problem with our system as a whole. We created a test to make it easy to test large groups. *( that is sort of cheating) We then say we aren’t going to leave the grading to teachers, we’ll use a machine ( that is also sort of cheating). Then we say our machines are pretty accurate. But they are machines- so we can’t ask them to understand why students make certain choices, all we can do is calculate how all the other students have answered those questions ( that is sort of cheating).
We are in this problem of education because the numbers have gotten away from us. And they have gotten away from us because we have mistakenly assumed that we are all the same and therefore we can use a one size fits all system.
Guess what? We are the same and more importantly we are NOT!
Our tests have been created to prove that each child has assimilated the material they’ve been presented with. If the child doesn’t answer the test accurately then we fault the teachers or the administrators. ( This is silly. It’s cheating our kids.)
It’s the easy way to handle the problem of educating students. Let’s not see if kids really know what we are demanding they should know. Let’s just test them on what we think they should have absorbed.
Folks? That is cheating too!
We have a problem alright but it isn’t going to go away if we fire some “bad teachers or administrators”. That is a bandaid on a much larger problem.
Just remember…I told you!
love,
mo