Nate Silver
The Signal and the Noise: Why So Many Predictions Fail-but Some Don't
Signal and noise https://fivethirtyeight.com/
“Standard & Poor’s, for instance, told investors that when it rated a particularly complex type of security known as a collateralized debt obligation… at AAA, there was only a 0.12 percent probability—about 1 chance in 850—that it would fail to pay out over the next five years… In fact, around 28 percent of the AAA-rated CDOs defaulted, according to S&P’s internal figures… This is just about as complete a failure as it is possible to make in a prediction: trillions of dollars in investments that were rated as being almost completely safe instead turned out to be almost completely unsafe” (loc. 379).
How could the ratings agencies—“whose job it is to measure risk in financial markets” (loc. 417)—have failed to account for the possibility of a housing crash? Well, when you consider how much money these agencies were making off of their ratings (loc. 445-66), the whole conundrum becomes a little easier to unravel. As Silver puts it, “the possibility of a housing bubble, and that it might burst… represented a threat to the ratings agencies’ gravy train. Human being have an extraordinary capacity to ignore risks that threaten their livelihood, as though this will make them go away” (loc. 466).
Compounding the danger in this scenario was the fact that American financial institutions were, at the time, very highly leveraged. As Silver notes, “Lehman Brothers, in 2007, had a leverage ratio of about 33 to 1, meaning that it had about $1 in capital for every $33 in financial positions that it held. This meant that if there was just a 3 to 4 percent decline in the value of its portfolio, Lehman Brothers would have negative equity and would potentially face bankruptcy. Lehman was not alone in being highly levered: the leverage ratio for other major U.S. banks was about 30 and had been increasing steadily in the run up to the financial crisis” (loc. 633).
Bayes Thomas Bayes was an English minister who lived in the 18th century. Though Bayes was elected as a Fellow of the Royal Society and did publish during his lifetime, he did not achieve a good deal of influence until after his death; and today his influence is stronger than ever. Bayes’ influence comes mainly from a paper of his that was published after his death called ‘An Essay toward Solving a Problem in the Doctrine of Chances,’ which “concerned how we formulate probabilistic beliefs about the world when we encounter new data” (loc. 4123).
The paper was intended as a response to the famous philosopher and skeptic David Hume, who argued that we could not truly predict anything with any amount of certainty. This is the case, according to Hume, because all of our information about the world comes from past experience, and just because something happened in the past (even with great frequency) does not mean we can logically deduce that it will happen again in the future.
A famous example here is one involving breast cancer. About 1.4% of women develop breast cancer when they are in their 40’s (loc. 4196). One way to detect breast cancer is with a mammogram, but these tests are not foolproof. Specifically, if a woman has breast cancer, a mammogram will detect it about 75% of the time. If, on the other hand, she does not have breast cancer, a mammogram will still come up positive 10% of the time (loc. 4199). Let’s say a woman in her 40’s has a mammogram and it comes up positive. What are the chances that she has breast cancer? The answer is a lot less that what you might think. It’s actually 10% (a number that Bayes’ theorem accurately comes up with [loc. 4201]).