Noise: A Flaw in Human Judgement by Daniel Kahneman, Oliver Sibony, & Cass Sunstein

Part II: Your Mind Is a Measuring Instrument

4. Maters of Judgmen

  • A judgment is a decision where the instrument used is the human mind. The most common measure of variability is the standard deviation, which the authors use here. Matters of judgment occupy a place between known facts and taste. There are two types of reliability. They are within-person where the same person judges a situation more than once over time and between-person where multiple people judge the same thing at the same time. The authors provide a scenario involving the hiring of a CEO for you to experience the process of judging.

5. Measuring Error

  • The main idea here is that being very high one time and very low another time does not mean that on average you are right. The key equation is Bias + Noise = Error. Bias is simply the average of errors. If your average error is zero your high judgments are equal to your low judgments. The less noise, the smaller the standard deviation. That means your bell-shaped curve has less spread. Here we are introduced to the concept of mean squared error. It is equal to bias squared plus noise squared. This means that bias and noise are independent of each other and equally weighted when it comes to determining error.

6. The Analysis of Noise

  • This chapter deals with a noise audit from 1981 of a federal judicial system where 208 judges each judged sixteen cases. The measure of noise within each case is the standard deviation of the terms assigned in that case. The mean sentence was 7.0 years and the standard deviation around that mean was 3.4 years. This is considered the system noise. When we compare judges to each other we find some more lenient than others. These deviations are called level errors. In this sample, the level errors were 2.4. The deviation within the sentences for a single judge is called pattern error. The pattern errors for each judge add up to zero.
  • The three types of noise are compared to each other with this equation. System Noise2 = Level Noise2 + Pattern Noise2.

7. Occasion Noise

  • Occasion noise is the noise we see when looking at the same person making repeated judgments about the same thing. We do not always produce identical judgments when faced with the same facts on two occasions. Measuring occasion noise is not easy. If you present someone with the same problem a second time they may recognize it and simply reproduce their earlier answer.
  • The wisdom-of-crowds effect comes into play here. If the same person makes an estimate on two occasions, the average is more likely to be closer to the correct answer. The average of two judges, however, is likely to be even more accurate. This gives a rationale for the notion that you should sleep on it and think again. Some other sources of occasional noise are mood, stress, fatigue, weather, time of day, and the order in which a specific judgment appears. Occasion noise is usually less than the difference among individuals. In other words, you are less different from the you of last week than you are from someone else today.

8. How Groups Amplify Noise

  • The first few people in a group who express an opinion can sway the group. The best way to make your group “wise” is to have people express opinions independently without knowing what others think. Aggregating independent judgments is a good way to reduce noise. The opinion of an initial speaker can cause an information cascade. When it comes to politics when members of one party hear the opinion of the other party they are likely to reflexively disagree rather than do their own thinking. This was especially apparent when Trump was president as Democrats were quick to disagree with anything he did or said.
Share this:
Share this page via Email Share this page via Stumble Upon Share this page via Digg this Share this page via Facebook Share this page via Twitter Share this page via Google Plus     If you like the summary, buy the book
Pages: 1 2 3 4 5 6

Leave a Reply