The Testing Charade: Pretending to Make Schools Better by Daniel Koretz

The first things schools do is to reallocate teaching time from nontested subjects to those tested. Next, they reallocate time within the tested subjects by only teaching material that has shown up on previous tests. It isn’t unusual for districts to hand out lists of material tested. Finally, we have the subject of coaching that can help students score better without understanding the material involved. Teachers are encouraged to give unit tests that mimic the state tests in every way depriving them of encountering anything unexpected. This also results in fraudulent score gains.
Students can get multiple choice questions correct by only knowing that the wrong answers are wrong as good distractors are difficult to compose. Tests are like polls in that the only deal with small samples of the course content. Coaching that focuses on how to score and not real learning involves bad teaching practice and it is common. Many teachers only know the current system that promotes this. Such coaching does nothing to prepare students for the real world either. The tests define the curriculum and teachers tend to reverse engineer their teaching. Perhaps the worst fallout is due to the fact that high performing students are exposed to little of this bad teaching while poor kids get the biggest doses.

The mantra for the current reform movement is that all children can learn to high levels. Now, more specifically, it’s all should be college and career ready. Clearly, these policies have failed. Due to the range in cognitive ability, there is no reason to expect high performance for all. In order to evaluate schools, arbitrary standards and targets are set. Daniel summarizes a number of ways this is done, and they all involve a lot of guesswork. This results in some being too low while others are impractically high.
Performance targets can also be manipulated to give the desired results. These reforms were supposed to reduce inequality, but they have only made it worse as kids in poor schools engage in more test prep, which features bad teaching practice. Also, teachers in poor schools face problems that make it harder to produce the desired gains. It seems that reformers aren’t aware of the bell curve distribution of study ability as they create tests designed to sort kids into a bell curve of performance. Pretending all kids are the same makes no sense as they come from very different environments.

In the current system a teacher has to throw self-interest to the wind if she wants to make math interesting, engaging, and fun. Many good activities fall outside of the range that standardized tests can sample well. Good teaching won’t increase scores on a specific test as lessons aimed at the test itself. The state tests, for example, don’t require students to write a research paper. Would you rather see research papers or test prep?
At the heart of the problem is that fact that identifying effective teaching isn’t as easy as the reformers think as it relies on subjective judgments of administrators based on their observations. Test-based evaluations are unstable. This is due in part to the fact that a teacher’s classes can vary from year to year, the number of students is small by statistical standards, and what happens to kids outside of school has a greater impact.
Daniel goes on to explain why Value-Added Models don’t work and explains bias and reliability. A reliable test generates similar scores when it’s given, while a test with bias gives scores that are too high or too low and may or may not be reliable. What we have seen is that a teacher’s average test scores are unreliable. It isn’t unusual for a teacher to be in the bottom 20% one year and in the top 20% the next.

Book Summaries